More resolution for color sensor

Background

The principle of digital image sensors such as the 18 MP onsemi AR1820HS means that they acquire only brightness – but not color – information. As a result, a color filter is applied to each pixel during manufacture of color sensors. This is known as the Bayer matrix.

Arrangement of color filters in the Bayer matrix
Arrangement of color filters in the Bayer matrix
The RGB filter layers only transmit light with a particular wavelength
The RGB filter layers only transmit light with a particular wavelength

Of each four pixels, two pixels are given a green color filter, one pixel a red filter and one pixel a blue color filter. This color distribution corresponds to the perception of the human eye and is referred to as the Bayer matrix. A pixel depicts only the information for one color.

To obtain the complete RGB values for each pixel, the missing primary colors are interpolated from four neighboring pixels using appropriate algorithms. This color interpolation assumes that there are only slight color differences between two adjacent pixels of the same color. Strictly speaking, a sensor with Bayer matrix therefore has only a quarter of the native sensor resolution.

Sensor mono mode

Although the Bayer matrix cannot simply be rendered invisible for mono mode, the following two solutions show how you can achieve the desired result depending on the type of application.

1) … for "gray scenes"

If the 18 MP color sensor is to be used in mono mode for achromatic scenes, note that a broad band light source (white light) must still be used. This is because of the sensor’s Bayer matrix. With this sensor, monochromatic (single color) light would have resulted in the individual pixels transmitting no or less information depending on the wavelength of the light, due to the RGB filter layers used (see Figure 2). This can result in a different brightness perception for the individual pixels. In this case, the RGB enhancement must be calibrated separately for R, G, and B. As a result, you then obtain an identical brightness perception for all pixels as with a mono sensor.

Without calibration of the RGB enhancement, the Bayer matrix is clearly visible (left).<br>After RGB calibration (see RGB histogram, right) there is a homo-geneous brightness perception as with a mono sensor.

Note: This RGB calibration is only valid for this specific light source and a "grey" scene. If the light source (wavelength) changes, the RGB enhancement factors have to be re-adjusted.

How to switch the Bayer matrix to "invisible" using the uEye Cockpit:

  • Illuminate the scene with a white light source.
  • Show the histogram with the option "Show Bayer RGB"
  • Select the exposure time so that no significant parts of the image are overexposed (clipping). This can be seen in the histogram if numerous pixels have the value 255. Counter measure, e.g. use AES ("AES/AGC" tab) to regulate to 128.
  • Then correct the AWB (Auto White Balance) to "Grey World". In the histogram, you can see how the color curves align over one another.
  • Check in 8-bit RAW mode ("Format" tab). The Bayer matrix should not be visible.
  • If it is, adjust the RGB enhancement factors manually ("Image" tab).


After calibration, you will no longer see the Bayer matrix in "monochromatic" scenes. By contrast, if you bring a colored object into the scene, you will only see the Bayer matrix on it.

After the RGB calibration, the Bayer matrix can only be seen in colored parts of the image, as the RGB filter layers transmit no or less information depending on the wavelength of the light. (see Figure 2)

2) … for "color or gray scenes"

If you work with color scenes in your application, the brightness sensitivity of the individual Bayer pixels constantly changes with the variation in colour components. There is also a way to achieve genuine mono mode in this situation. The solution lies in the color spectrum of the 18 MP onsemi AR1820HS.

Above a wavelength of around 900 nm the color filters for the individual pixels have similar spectral properties. Beyond this threshold, all pixels on the sensor respond practically identically to incident light again – exactly as with a dedicated mono sensor. This means that the Bayer matrix can also be made invisible using this method, both for color and also for grey scenes.

The color spectrum of the AR1820HS shows similar spectral proper-ties of the color filters above 900 nm
The color spectrum of the AR1820HS shows similar spectral proper-ties of the color filters above 900 nm

In order to be able to use this spectral property of the sensor as described, you must observe the following:

  • Ensure defined lighting conditions, i.e. seal off light with other wavelengths shorter than 900 nm as far as possible.
  • Order your uEye camera with AR1820HS sensor specifically with GL filter (glass). The HQ filter normally ordered with this sensor would shut out the long-wave light required here. By contrast, the GL filter allows light beyond 900 nm to pass almost unweakened. The highest possible signal strength thus arrives at the sensor.

Suitable lens

Under the conditions described, a uEye camera with AR1820HS sensor in mono mode satisfies all the requirements for a far higher image resolution than in color mode. In the overwhelming majority of cases, however, the camera is used in conjunction with a lens. In this case, with an achievable nominal sensor resolution of 18 MP, the lens is the limiting factor in the resolution capability of the entire system.

The optical resolution of a lens is generally specified in mega pixels. This figure relates to the largest sensor format for which the lens is designed. It determines how fine the structures are that the lens can transmit. In other words, the smaller the sensor format the finer resolution the lens requires. If you are using a lens with an insufficient optical resolution, the sensor pixels are under-scanned. The details that can be depicted then spread over more adjacent pixels. As a consequence, you do not achieve the full image sharpness that the sensor resolution allows. However, you should avoid combinations involving a lens that has a higher resolution on the image side than the relevant sensor. This can make unwanted aliasing / moiré effects visible in the image.

In practice, reproducing the smallest detail with approx. 2 pixels is recommended. Overall, this means that identifying a suitable lens depends on the reproduction scale, but also to a significant extent on the pixel size of the sensor used.