Introduction
The security and video surveillance camera industry has experienced an explosion in growth over the last few decades. Most public spaces are now under surveillance for the populace's safety, and more cameras are being installed daily. However, it's crucial to understand that it is not necessarily the number of cameras but the quality of their image processing that matters. Without high image quality, the performance and safety of security camera systems will suffer greatly.
In 2018, the IEC technical committee TC79 officially published the first internationally recognized standard for security camera and video surveillance performance testing. Many of our engineers are members of TC79 and actively contributed to the publication. The standard, IEC 62676-5, provides an in-depth outline of the performance thresholds that the camera and video systems must attain to be considered acceptable for public/consumer use. These thresholds are analyzed by testing the system for various key performance indicators (KPIs) established in the standard.
Overview of KPIs
Security and surveillance cameras are, for the most part, utilized for safety purposes. Thus, all the vital KPIs must be thoroughly tested before the camera systems are brought to market. In the standard, the TC79 committee outlines eight image quality performance tests that evaluate the most important KPIs for security video camera systems, including resolution, minimum illumination/low-light, dynamic range, infrared illumination, geometric distortion, veiling flare, and maximum frame rate. Other factors contributing to the overall camera performance include white balance, signal-to-noise ratio (SNR), image processing, and color output.
IEC 62676-5-1 environmental testing
The TC79 committee is currently working on an addition to the standard that focuses on the impact the surrounding environment can have on these camera systems. Many security cameras are located outside and must naturally contend with varying temperature fluctuations and humidity. These environmental challenges can affect the image and video quality, particularly resolution and noise. IEC 62676-5-1 will outline test methods for evaluating camera performance at different temperature and humidity levels.
KPI testing methods
When testing camera KPIs, it is essential to follow the procedure as described in the standard to ensure the most accurate results. The standard provides a brief overview, camera settings, and the general test methods for each KPI. We will discuss the test methods and the equipment we use for testing and analysis.
Resolution
For resolution measurements, IEC 62676-5 uses ISO 12233:2000 as a basis for the requirements for measuring resolution performance. Security cameras can be tested using two different methods, the first being the visual resolution method. This method uses slanted edge test targets due to the horizontal and vertical resolution requirements of video surveillance cameras. These slanted edge test charts are widely available but must be based on ISO 12233 to comply with IEC 62676-5.
The other method for resolution measurements is the sine-based SFR method (s-SFR). This method was introduced in ISO 12233:2014 and uses Siemens star test targets for evaluating resolution. With a Siemens star, video cameras' horizontal, vertical, and diagonal resolution can be accurately analyzed.
When we measure resolution according to IEC 62676-5, we first determine the geometric distortion of the camera. If the geometric distortion is above 15%, the standard does not recommend using the visual resolution method (using slanted edges) for measurement. In this case, the standard recommends using the SFR measurement. For the slanted edge measurements, we use test chart TE170 or TE095; for Siemens star, we use TE253 or TE253 9x. Please note that for this standard, the Siemens star charts must have stars in all four corners and in the center.
Low-light/minimum illumination
One of the most important KPIs for security camera testing is low-light performance, and ISO 19093 outlines the methods for low-light performance testing. However, this ISO standard was not yet complete when IEC 62676-5 was originally published. As a result, ISO 19093 is not referenced in the central part of the IEC standard but is now an addendum to the standard and is recommended for low-light testing.
The goal of the low-light KPI is to determine the lowest light level where the camera still provides acceptable image quality. The standard references two different options for measuring low light. The basic method uses a reflective grayscale test target to measure noise reduction at the various gray levels. The other method is to use an OECF gray patch chart and then measure and compare the quality of the light patches against the dark patches.
ISO 19093 also references the TE42-LL reflective test chart, produced exclusively by Image Engineering. This chart is optimized for low-light testing and contains various grayscale and other targets for measuring other low-light KPIs, and this chart is what we utilize when measuring low-light performance.
Dynamic range
The dynamic range KPI evaluates how well a camera reproduces light and dark aspects of a scene in the same image. It uses ISO 14524 and ISO 21550 as its baseline for dynamic range measurements of security camera systems. There are two options for measuring dynamic range according to the standard. The first option is a visual dynamic range test that requires a reflective grayscale chart. The second option is more advanced and requires a transparent OECF gray patch chart with preferably more than 24 patches. The reflective chart will show less contrast at just under 60 dB / 1,000:1 while a transparent chart will show a contrast up to 134 dB / 5,000,000:1.
The chart we use for dynamic range measurements of security cameras is the transparent TE269C. This chart has 36 gray patches and a very high contrast of 134 dB. Version C has been adapted to meet the exact specifications of IEC 62676-5 making it one of the top test charts for dynamic range measurements of video surveillance cameras.
Infrared illumination
Many security cameras have built-in infrared (IR) illuminators, primarily for night vision, for measuring the distance between a camera and an object to obtain the video output level in the IR range. The standard requires a white chart with over 70% white pattern reflectance to test the infrared performance KPI. At IE, we use the TE285 test chart adapted to the standard requirements to evaluate the infrared illumination operating view distance.
Line geometric distortion
In IEC 62676-5, the measurements for geometric line distortion are based on ISO 17850. Image distortion occurs when the straight lines of an image appear to be deformed or curved unnaturally. The security standard evaluates distortion using a grid chart and measuring a straight horizontal or vertical line bending at defined distances from the image center. From there, you can calculate the amount of acceptable distortion at varying distances.
When we measure geometric distortion based on the standard, we use the TE251 test chart. This is a reflective test chart for measuring geometric distortion like the one described in the standard. It uses black crosses on a white background for measuring horizontal and vertical lines.
Veiling flare
Many security camera systems are located in an outdoor environment, making them susceptible to capturing stray light, particularly sunlight, in the image or video. This challenge is known as veiling flare and can significantly disrupt the image quality. IEC 62676-5 uses ISO 18844 to measure the flare KPI.
The ISO and IEC standards recommend using a test chart incorporating light traps into its design. These light traps absorb a high proportion of light, creating targets with a higher density than standard test targets. The level of black of these traps will only be affected if the level of flare rises. The camera can then be evaluated for how well it performs at various flare levels.
When we measure flare in our lab, we use the TE281 reflective test chart. This test chart is based on the requirements of the IEC and ISO standard and uses circular light traps to capture light.
Maximum frame rate
The maximum frame rate KPI evaluates the maximum level at which the capture frame rate of the image sensor is adequate under different lighting conditions. The standard describes a measurement method that uses an LED light array that can be switched on and off at set intervals to measure the camera's frame rate.
At IE, we test the maximum frame rate of a security camera system using the LED-Panel. This device uses an array of LEDs that offer various illumination modes and frequencies depending on the measurement parameters. It is based on ISO 15781 but is also suitable for IEC 62676-5. In addition to frame rate, the LED-Panel can also analyze different timing measurements such as shooting time lag, negative shutter release time lag, shutter release time lag, shooting rate, and start-up time.
Setting up a test lab
For each KPI, the standard outlines test setups recommended when testing security cameras. For the most part, when using reflective test charts such as the TE42-LL for resolution or the TE281 for flare, it's essential to ensure the chart is illuminated uniformly. We use the iQ-Flatlights and a chart mount in our test lab to create a working test setup. Our iQ-Flatlights are powered by iQ-LED technology, which allows you to generate custom spectra. Thus, you would only need one light source (for reflective charts) to generate all of the illumination requirements outlined in IEC 62676-5.
For transparent chart illumination, we recommend using the LG3 lightbox. This light source is particularly effective with TE269C for high dynamic range measurements as it can illuminate a high contrast test target with over 150000 lx. We also offer the LE7 lightbox for uniform (transparent) chart illumination. This device uses iQ-LED technology and requires a TE269C or other step chart. We also recommend using the LE7-IR version for the best results.
As previously mentioned, IEC 62676-5-1 is an add-on to the standard that will cover how to test cameras in different weather environments. Currently, the only commercially available device for testing cameras under various temperatures is the iQ-Climate Chamber. We use this device to test how security cameras react to changing environments.
Finally, when analyzing the results, we use the iQ-Analyzer-X analysis software. You can easily upload your image results, and the software will show you where performance can be improved for the KPI under test.
Please see our test lab checklist for more information on test equipment or how to set up a lab for proper image quality testing. You can also contact our support team.
Conclusion
Most public spaces are now under surveillance for the populace's safety, and more cameras are being installed daily. As a result, these cameras must be tested and meet internationally recognized performance standards to ensure high quality and safety.
The primary standard the security camera industry utilizes is IEC 62676-5. This standard provides an in-depth outline of the performance thresholds that the camera and video systems must attain to be acceptable for public/consumer use. These thresholds are analyzed by testing the system for a variety of key performance indicators (KPIs) resolution, minimum illumination/low-light, dynamic range, infrared illumination, geometric distortion, veiling flare, and maximum frame rate.
As security and surveillance cameras become increasingly part of our daily lives, it is necessary to continue establishing performance standards that manufacturers can use to ensure that their products perform at a high level and are safe for public use regardless of the environment.