Image Quality Factors

Contrast Detection Probability (CDP)

Image Quality Factors
  1. Introduction
  2. The background of CDP
    1. Why CDP over SNR?
  3. How to test for CDP
    1. Dynamic Test Stand (DTS)
    2. Benefits of the DTS and its CDP evaluation
  4. Conclusion
  5. References

Introduction

Contrast detection probability refers to the camera systems ability to correctly identify the contrast of an object in its field of view. For example, advanced driver assistance systems (ADAS) must be able to identify different objects in its field of view. Without proper identification, the cameras system may make an incorrect adjustment and create a dangerous situation. With that in mind, it is extremely important to test the CDP of a camera system to ensure high performance and safety.

CDP
Image 1: An ADAS system must be aware of its surroundings.

The background of CDP

Until recently, the automotive industry lacked international standards for testing and evaluating the image quality of these camera systems. To combat this problem, the IEEE-P2020 working group was formed to establish parameters that will ultimately lead to the publication of an international standard on automotive image quality.1

At this time, the group has defined certain key performance indicators (KPIs), including CDP, and published a white paper that outlines the groups work towards an international standard for automotive image quality testing and specifications.2

CDP is the most important key performance indicator (KPI) that the group is currently working on. First presented by Dr. Marc Geese in 2018, CDP is designed to be independent from the components tested so it can be used to perform the same performance tests on different subsystems (e.g., lenses, sensors, windshields, etc.).

Why CDP over SNR?

Traditionally, SNR is used to describe the performance of a camera when reproducing the contrast of a scene. However, for a scene with extremely high dynamic range, such as those often encountered by ADAS (e.g., a scene with flickering light), the SNR value is not the best indicator of full system performance in these situations.

SNR is a well-established metric that measures both the intended signal and the background noise that obscures the signal. Unfortunately, it is hard to link the SNR values directly to an application.

So, while most engineers are familiar with the required SNR for their industry, the values themselves still cannot be directly linked to the task of an imaging system to detect a specific contrast under a specific condition. CDP, however, can provide this missing information and thus, vastly improve the design requirements that are now based around fact as opposed to a “best guess.”3

SNR drop graph
Image 2: A typical SNR curve due to HDR rendering showing various SNR drops.

SNR drops are another issue that often appear in ADAS cameras. These drops are a phenomenon of a multi-exposure HDR algorithm that results in a non-linear SNR curve. This issue is especially prevalent in luminance regions where multiple images are combined often leading to an SNR drop from an acceptable high value to a critical low value.

SNR OECF
Image 3: An example of an SNR drop. Noise is more common in the light parts of the image as opposed to the dark.

Additional issues with SNR also arise when trying to predict how an ADAS camera system will perform using an SNR value. The SNR drops, for instance, do not account for other elements in the scene such as fog or dust in the air between the camera and the object(s) in the field of view.

SNR CDP
Image 4: Various aspects such as dust or fog are not taken into account when using the SNR. For the automotive industry, this can quickly become a safety problem.

CDP was developed as a result of SNR being an inadequate performance measurement for the automotive industry. Unlike SNR, CDP directly links the system under test and can be used to describe its performance regardless of the natural conditions. CDP can also provide an overview of multiple image quality components using the same metric.

Numerous tests are currently underway to support CDP as an internationally recognized test procedure for automotive camera systems.

How to test for CDP

As CDP is a relatively new concept, there are limited tools on the market that can measure and provide accurate results. For example, a typical image quality test setup using a test chart can only provide a limited number of data points in a synthetic scene. Dynamic scenes, however, such as those experienced by automotive cameras are better analyzed using numerous data points from the scene. To recreate dynamic scenes in a test lab, our engineers have developed the DTS.

Dynamic Test Stand (DTS)

Essentially, the DTS can test an extensive dynamic scene as opposed to a synthetic scene in a test lab environment. In the test lab, it can precisely recreate driving conditions that would be much harder to duplicate in an unpredictable outdoor setting or with a single test chart.

The DTS tests for CDP using six separate illumination boxes each with 36 gray patches for a total of 216 different intensities with a dynamic range of 120 dB. Each box can show different levels of intensity at the same creating a more dynamic testing situation similar to what an automotive camera will face in a real environment. One image set can provide numerous data points for CDP giving you much more insight into the full performance of the system under test.

DTS device
Image 5: The full DTS device.
DTS contrast boxes
Image 6: Contrast detection boxes from the DTS for evaluating the CDP performance of a camera.

The DTS has the only integrated analysis software specifically for evaluating CDP on the market. The software creates a heat map based on the data points collected (see images below) from the device under test. These heat maps show each data point plotted depending on the systems performance at a wide range of luminance and contrast. Yellow shows where the system performs highly, while blue indicates lower performance. It is important to note that each system under test has different requirements and so even a map that appears to have lower performance may in fact still be acceptable for the specific camera under test.

CDP results
Figure 1: Results for a single exposure system that performs well at high contrasts and luminance.
CDP results
Figure 2: Results from an HDR system that increases its performance by staggering three exposures into one.

Benefits of the DTS and its CDP evaluation

The DTS and its evaluation software gives automotive companies a reliable way to characterize their systems and use the results to evaluate performance and safety of their systems in a test lab setting.

Other benefits include:

  • Numerous data points for each measurement
  • Eliminating the majority of road testing
  • Validating the specifications of finished products
  • Easily allows manufacturers to make specification requirements via heat maps
  • Output .xml file with CDP, OECF, and PDF (probability density function) results

The DTS also has the ability to generate flickering light sequences for evaluating the camera systems response to changing light intensities. For more information, please see our flicker article.

Conclusion

Contrast detection probability refers to an automotive camera systems ability to correctly identify the contrast of an object in its field of view. Without proper identification, the system may make an incorrect adjustment and create a dangerous situation.

CDP is one of the current KPIs under development by the IEEE-P2020 working group. The group was formed to enact these testing parameters that will eventually lead to an internationally recognized automotive image quality standard.

AThe Dynamic Test Stand (DTS), one of the first CDP testing solutions on the market, has been developed in accordance to the proceedings of the iEEE-P2020 group. The DTS can generate a dynamic testing scene using 216 intensities at 120 dB to test the CDP performance of a camera system. It is also equipped with CDP evaluation software that uses heat maps to show exactly how the system is performing using different contrast and luminance.

The end goal of these automotive ADAS camera systems has always been to improve the efficiency and safety of our daily transportation. But before these systems can be implemented into everyday vehicles, they must first be rigorously tested to ensure the avoidance of dangerous situations.