Introduction
Autonomous driving (ADAS) technology has improved steadily over the last few years. Almost every new car sold today has a driver assistance system built into it, and developers work hard every day to achieve higher levels of autonomy. Most publicly available vehicles are at levels one and two, which still require a human driver to make the decisions but assist with specific tasks such as steering, braking, and acceleration. A few level-three vehicles are also commercially available. This level allows the driver to focus their attention elsewhere for a short period at lower speeds (e.g., in a traffic jam). The highest levels are still in the development and testing phase.
To reach levels four and five, developers must tackle various challenges to ensure high performance and public safety. Working groups, such as IEEE-P2020, are working towards publishing an internationally recognized standard aimed at defining KPIs and outlining test methods that can guide developers around the world when testing the performance and safety of their AV systems.
Current challenges
The AV industry faces many challenges in its push to level five autonomy. There are camera challenges, such as poor lighting conditions (nighttime driving, tunnels, garages, etc.), extreme contrasts (direct sunlight), poor weather conditions, contamination of the cameras, and the general complexity of the scene that a camera system must overcome to produce accurate adjustments.
In addition, there are other challenges, including establishing rules, regulations, and liabilities for AVs while still prioritizing innovation and ensuring that the safety of the hardware and against cybersecurity threats is robust. In addition, it's vital to answer the ethical questions about what to do in unavoidable crash scenarios.
Finally, developers and manufacturers must address the issue of testing their system KPIs in a test lab environment while using test methods that ensure test repeatability accuracy. These challenges must be addressed before the public legalizes and accepts level-five autonomous driving.
The IEEE-P2020 international working group is confronting many of these challenges. After years of work and commitment by a variety of engineers, academics, and automotive industry leaders, the P2020 standard has been submitted for final review and approval. Publication is set for the fall of 2024. With its publication, P2020 will become the first major internationally recognized standard for measuring the image quality of automotive imaging systems. Companies worldwide can use this standard to guide their image quality testing methods and parameters during development as they push AVs forward.
The P2020 standard outlines various key performance indicators (KPIs) that have been identified as crucial for evaluating the performance and safety of automotive camera systems. One of the main KPIs is contrast transfer accuracy or CTA (formerly known as contrast detection probability), defined by the standard as the accuracy of reproducing the scene contrast within a specified area and under various imaging conditions.
Contrast Transfer Accuracy (CTA)
Contrast Transfer Accuracy refers to the probability that a camera system will accurately interpret the contrast of an object in its field of view. For example, advanced driver assistance systems (ADAS) must be able to identify the contrast of different objects in their field of view. Without proper identification, the camera system may make an incorrect adjustment and create a dangerous situation. With that in mind, it is imperative to test CTA performance to ensure high quality and safety.
Testing CTA performance
As the IEEE-P2020 standard progressed, it became clear that new, more advanced test devices would be required to fulfill the testing specifications of the standard. For example, a typical image quality test setup in a test lab uses a test chart, and illumination devices can only offer a limited number of data points in a synthetic scene, i.e., it isn't easy to replicate a real-world scene that an automotive system would encounter with such a setup.
One way to test a dynamic automotive camera situation is to perform test drives. However, while test drives can be a practical test method, they also require extensive manpower, time, and space. In addition, it is often very challenging to recreate the exact conditions of the test drive, e.g., how do you simulate the same weather conditions and sun brightness? Even more problematic, how do you ensure test repeatability accuracy in different test labs or even your own? These issues made it clear that accurate CTA measurements required new innovative test equipment that could be utilized in the comfort of a test lab.
Versatile Light System (VLS)
With our first-hand knowledge of the IEEE-P2020 standard (many of our engineers are actively contributing to its publication), we began developing new solutions to meet the testing demands. Our latest innovation is the Versatile Light System (VLS). The VLS is an illumination solution that efficiently measures CTA and other P2020 KPIs, including modulated light mitigation probability (MMP) and contrast signal-to-noise-ration (CSNR).
The VLS is a system of six Vega devices, each connected to an individual movable arm*. All Vegas are linked via the arms to a tripod, offering high adaptability. The flexibility of the arms allows each Vega to be manually moved around to quickly generate randomized setups and angles during testing, allowing you to create a dynamic test scene.
The VLS has control software that allows you to generate test sequences, increase or decrease the brightness, and activate flicker mode for MMP testing. In addition, the VLS-Evaluation software enables you to evaluate your test results.
The VLS offers a powerful solution for those looking to evaluate the KPIs outlined in the IEEE-P2020 standard accurately. It can also be utilized to test camera systems with high intensity and very short exposure times from other industries.
VLS in a test lab
One of the VLS's significant benefits is that it is optimized for test labs of almost any size. Each Vega light source is connected to an arm, and all arms are connected to a traditional tripod, making it easy to move around and store. In our iQ-Lab, we are performing numerous tests using the VLS, and we are using the flexibility of the Vegas and the arms to simulate different test scenes and illumination angles more efficiently as required. This robust flexibility provides a comprehensive overview of how the camera system performs in multiple challenging real-world situations typically encountered by automotive camera systems.
Testing in a test lab is ideal, but what about simulating different weather conditions? In the iQ-Lab, our testing team combines the VLS with the iQ-Climate Chamber to test camera performance in various weather climates. The device under test is placed inside the iQ-Climate Chamber, and the test temperature is set. The VLS is then placed in front of the window for testing.
We are excited to add the VLS to our product portfolio and believe it will be a game changer in the market for those looking to evaluate their camera systems performance, particularly those following the test methods and KPIs outlined in the IEEE-P2020 standard.
Contact us
Our iQ-Lab offers numerous camera testing services, including CTA evaluations, according to IEEE-P2020. If you would like to send your cameras to us for testing, don't hesitate to contact us for further information at .
For more information on purchasing the VLS or other solutions for your lab, please contact our sales team at .
*We use third-party products to enable our customers to create flexible setups. The arm and tripod are third-party products, and the setup shown is an example of how the user can set up the light sources using them.