Image Quality Factors

Contrast Detection Probability (CDP)

Image Quality Factors
  1. Introduction
  2. The background of CDP
    1. Why CDP over SNR?
  3. How to test for CDP
    1. Dynamic Test Stand (DTS)
    2. Benefits of the DTS and its CDP evaluation
  4. Conclusion
  5. References

Introduction

Contrast detection probability refers to the camera system's ability to correctly identify an object's contrast in its field of view. For example, advanced driver assistance systems (ADAS) must have the capability to identify different objects in their field of view. Without proper identification, the camera system may make an incorrect adjustment and create a dangerous situation. With that in mind, it is imperative to test the CDP of a camera system to ensure high performance and safety.

CDP
Image 1: An ADAS system must be aware of its surroundings.

The background of CDP

Until recently, the automotive industry lacked international standards for testing and evaluating these camera systems' image quality. As a result, the IEEE-P2020 working group was formed to establish parameters that will ultimately lead to the publication of an international standard on automotive image quality.1

At this time, the group has defined specific key performance indicators (KPIs), including CDP, and published a white paper2 that outlines the group's work towards an international standard for automotive image quality testing and specifications.2

CDP is the most important key performance indicator (KPI) that the group is currently working on. First presented by Dr. Marc Geese in 2018, CDP is designed to be independent of the components tested so a test team could use it to perform the same performance tests on different subsystems (e.g., lenses, sensors, windshields, etc.).

Why CDP over SNR?

Traditionally, SNR is used to describe a camera's performance when reproducing the contrast of a scene. However, for a setting with extremely high dynamic range, such as those often encountered by ADAS (e.g., a scene with flickering light), the SNR value is not the best indicator of full system performance in these situations.

SNR is a well-established metric that measures both the intended signal and the background noise that obscures the signal. Unfortunately, it is hard to link the SNR values directly to an application.

So, while most engineers are familiar with the required SNR for their industry, the values themselves still cannot be directly linked to an imaging system's task to detect a specific contrast under a particular condition. CDP, however, can provide this missing information and thus, vastly improve the design requirements that are now based around facts as opposed to a "best guess.3

SNR drop graph
Image 2: A typical SNR curve due to HDR rendering showing various SNR drops.

SNR drops are another issue that often appears in ADAS cameras. These drops are a phenomenon of a multi-exposure HDR algorithm that results in a non-linear SNR curve. This issue is especially prevalent in luminance regions where multiple images are combined, often leading to an SNR drop from an acceptable high value to a low critical value.

SNR OECF
Image 3: An example of an SNR drop. Noise is more common in the light parts of the image as opposed to the dark.

Additional issues with SNR also arise when predicting how an ADAS camera system will perform using an SNR value. The SNR drops, for instance, do not account for other elements in the scene, such as fog or dust between the camera and the object(s).

SNR CDP
Image 4: Various aspects such as dust or fog are not taken into account when using the SNR. For the automotive industry, this can quickly become a safety problem.

The development of CDP is a result of SNR being an inadequate performance measurement for the automotive industry. Unlike SNR, CDP directly links the system under test and can describe its performance regardless of the natural conditions. CDP can also provide an overview of multiple image quality components using the same metric.

Numerous tests are currently underway to support CDP as an internationally recognized test procedure for automotive camera systems.

How to test for CDP

As CDP is a relatively new concept, there are limited tools on the market that can measure and provide accurate results. For example, a typical image quality test setup using a test chart can only offer a limited number of data points in a synthetic scene. However, dynamic scenes, such as those experienced by automotive cameras, are better analyzed using numerous data points from the scene. To recreate dynamic scenes in a test lab, our engineers have developed the DTS.

Dynamic Test Stand (DTS)

Essentially, the DTS can test an extensive dynamic scene as opposed to a synthetic scene in a test lab environment. In a test lab, the DTS can precisely recreate driving conditions that would be much harder to duplicate in an unpredictable outdoor setting or with a single test chart.

The DTS tests for CDP using six separate illumination boxes, each with 36 gray patches for 216 different intensities with a dynamic range of 120 dB. Each box can show different intensity levels simultaneously while creating a more dynamic testing situation similar to what an automotive camera will face in a natural environment. One image set can provide numerous data points for CDP, giving you much more insight into the system's complete performance under test.

DTS device
Image 5: The full DTS device.
DTS contrast boxes
Image 6: Contrast detection boxes from the DTS for evaluating the CDP performance of a camera.

The DTS has the only integrated analysis software specifically for evaluating CDP on the market. The software creates a heat map based on the data points collected (see images below) from the device under test. These heat maps show each data point plotted depending on the system's performance at a wide range of luminance and contrast. Yellow shows where the system performs highly, while blue indicates lower performance. It is important to note that each system under test has different requirements. So even a map that appears to have lower performance may, in fact, still be acceptable for the specific camera under test.

CDP results
Figure 1: Results for a single exposure system that performs well at high contrasts and luminance.
CDP results
Figure 2: Results from an HDR system that increases its performance by staggering three exposures into one.

Benefits of the DTS and its CDP evaluation

The DTS and its evaluation software give automotive companies a reliable way to characterize their systems and use the results to evaluate their designs' performance and safety in a test lab setting.

Other benefits include:

  • Numerous data points for each measurement
  • Eliminating the majority of road testing
  • Validating the specifications of finished products
  • It easily allows manufacturers to make specification requirements via heat maps
  • Output .xml file with CDP, OECF, and PDF (probability density function) results

The DTS can also generate flickering light sequences to evaluate the camera system's response to changing light intensities. For more information, please see our flicker article.

Conclusion

Contrast detection probability refers to an automotive camera system's ability to correctly identify an object's contrast in its field of view. Without proper identification, the system may make an incorrect adjustment and create a dangerous situation.

CDP is one of the current KPIs under development by the IEEE-P2020 working group. The group was formed to enact these testing parameters to lead to an internationally recognized automotive image quality standard eventually.

The Dynamic Test Stand (DTS), one of the first CDP testing solutions on the market, has been developed in accordance with the proceedings of the IEEE-P2020 group. The DTS can generate a dynamic testing scene using 216 intensities at 120 dB to test a camera system's CDP performance. It is also equipped with CDP evaluation software that uses heat maps to show exactly how the system performs using different contrast and luminance.

The end goal of these automotive ADAS camera systems has always been to improve our daily transportation efficiency and safety. But before we can implement these systems into everyday vehicles, they must first be rigorously tested to ensure the avoidance of dangerous situations.