Shutter-less calibration of uncooled infrared cameras

Infrared (IR) cameras based on microbolometer focal plane arrays (FPAs) are the most widely used cameras in thermography. New fields of applications like handheld devices and small distributed sensors benefit from the latest sensor improvements in terms of cost and size reduction. In order to compensate for disturbing influences derived from changing ambient conditions, radiometric cameras use an optical shutter for online recalibration purposes, partially also together with sensor temperature stabilization. For these new applications, IR cameras should consist only of infrared optics, a sensor array, and digital signal processing (DSP). For acceptable measurement uncertainty values without using an optical shutter (shutter-less), the disturbing influences of changing thermal conditions have to be treated based on temperature measurements of the camera interior. We propose a compensation approach based on calibration measurements under controlled ambient conditions. All correction parameters are determined during the calibration process. Without sensor temperature stabilization (TEC-less), the pixel responsivity is also affected by the camera temperature changes and has to be considered separately. This paper presents the details of the compensation procedure and discusses relevant aspects to gain low temperature measurement uncertainty. The residual measurement uncertainty values are compared to the shutter-based compensation approach.


Introduction
The evolution of infrared (IR) thermography including the discovery of new fields of applications is based on the development of small, power-efficient and, most of all, lowcost IR sensors.Microbolometer focal plane arrays (FPAs) are thermal sensors and are also called uncooled sensors because they work at room temperature without any cooling.This cost-reducing benefit compared to photon sensors is the reason why microbolometers are widely used in process control, fire prevention, fire protection and surveillance as well as for research and development tasks.
Each pixel of a microbolometer sensor array acts as a single sensor element.To ensure the required thermal isolation, such a microbolometer pixel consists of a semiconductor bridge and is coated with the bolometer material (Fig. 1).The cavity beneath the micro-bridge works as a λ/4resonator/absorber.The entire sensor has to be under vacuum atmosphere for thermal isolation reasons.The temperature of the bolometer bridge changes due to the absorbed inci-dent radiation.This temperature change affects the electrical resistance of the bolometer material due to its temperature coefficient of resistance (TCR).Common resistor materials are amorphous silicon (a-Si) and vanadium oxide (VOx) with a thin film TCR of about −0.03 and −0.027 K, respectively (Budzier and Gerlach, 2011).The change in electrical resistance is determined using a readout integrated circuit (ROIC) within the silicon substrate generating a radiationproportional voltage signal for each pixel.The raw infrared data set comprises the signal voltages of the entire sensor array and contains temperature information from the observed scene but also from the camera interior due to the huge field of view (FOV) of the pixels.
The state-of-the-art semiconductor fabrication technology enables microbolometer sensors with 17 µm pixel pitch and a wide variety of spatial resolutions from about 80 × 80 to 1024 × 768 pixel (Tissot et al., 2013).This leads to sensor sizes from a few centimeters down to only a few millimeters depending on the pixel resolution of the sensor.Sensor manufacturers recently made a lot of efforts to improve Published by Copernicus Publications on behalf of the AMA Association for Sensor Technology.(Moreno et al., 2012).
the vacuum packaging technology from common chip level packages (CLPs) via wafer level packaging (WLP) to pixel level packaging (PLP).This integration of vacuum packaging technology into the semiconductor processing will dramatically reduce the sensor cost and, therefore, will make microbolometer FPAs more attractive for lowest-cost applications like handheld devices and small distributed IR sensors (Hoelter et al., 2015;Takasawa, 2015).
IR cameras are used for radiometric measurement of temperature distributions.Here, two concerns mainly have to be taken into account: the sensor non-uniformity due to fabrication variations and the influences derived from changing ambient conditions, especially the ambient temperature.In the past such cameras used an integrated thermoelectric cooler (TEC) for keeping the temperature of the FPA constant and preventing the sensor parameters' offset voltage and responsivity to change according to the ambient temperature.But the high power consumption of this stabilization is a big drawback and is the decisive reason for using TEC-less microbolometer FPAs, e.g., for mobile IR devices or distributed sensor networks.
The correction approaches for infrared imagers and radiometric cameras differ in the required calibration effort (Tempelhahn et al., 2015).Radiometrically calibrated infrared cameras typically use optical shutters for runtime recalibration purposes in order to regularly correct thermal drift influences on the measurement.The calibration procedure for the shutter-based compensation approach is presented in detail in Budzier and Gerlach (2015).But the shutter is often the size-limiting component of an infrared camera because it has to cover the entire aperture.Another drawback of the shutter-based compensation approach is the interruption of the measurement during recalibration.Therefore, shutter-less infrared cameras are advantageous, especially for the new fields of application mentioned above.This paper presents a novel shutter-less compensation approach based on a cali-bration procedure for determining correction parameters that allow one to compensate for the disturbing influences of a changing ambient temperature.Relevant aspects to gain low temperature measurement uncertainty are discussed and compared to the shutter-based compensation approach.

Radiation model
In the following, the setup of an IR camera as shown in Fig. 2 is considered.The microbolometer FPA consists of pixels located in line i and column j .The exchanged (incident minus emitted) radiant flux ij of each pixel (ij ) is converted into the raw signal voltage V raw,ij corresponding to the linear relation between radiation and signal voltage, with the two sensor parameters responsivity R V ,ij and offset voltage V 0,ij .Both of them vary over the sensor array due to process variations during the sensor fabrication.Furthermore, both also depend on the sensor temperature ϑ fpa .
Previous investigations have shown that the pixel's field of view covers nearly the entire half space.Therefore, the corresponding pixel's projected solid angle ω pix amounts almost to π (Tempelhahn et al., 2013).Infrared cameras usually comprise infrared optics with an f -number about unity.For that reason, the projected solid angle ω obj related to the object irradiance E obj is about 1/5 of ω pix (Budzier, 2014).The remaining projected solid angle ω cam equals 4/5 of ω pix and is covered by the camera interior.Hence, each pixel also detects the irradiance E cam emitted by the camera housing and depending on the camera temperature ϑ cam .Furthermore, each pixel emits the radiant exitance M pix into the environment due to the sensor temperature ϑ fpa .The exchanged radiant flux ij from Eq. ( 1) can be written as a product of the pixel area A pix and a linear combination of these three radiant den- sities with their corresponding projected solid angles: Figure 2 illustrates this radiation composition and points out the influences of the three temperatures of the object, the camera and the sensor on the exchanged pixel radiant flux ij .The projected solid angles ω obj and ω cam of each pixel depend on the position within the FPA and are rotationsymmetrically distributed relating to the optical axis of the IR optics.
By replacing ij from Eq. ( 1) with Eq. ( 2), it is possible to separate E obj containing the essential object temperature information ϑ obj from all other disturbing influences: The slope gain ij of the pixel-specific linear equation comprises the pixel area A pix , the object-projected angle ω obj,ij and the sensor's temperature-dependent responsivity R V ,ij : The intercept offset ij combines different disturbing parts: the offset voltage V 0,ij , the signal parts V cam,ij and V pix de- rived from camera radiation, and the pixel radiant exitance.

Correction approach
The suggested approach adapts the shutter-based correction procedure presented in Budzier and Gerlach (2015) and considers the different disturbing influences separately.Figure 3 illustrates the flowchart of the calibration procedure to determine the necessary correction coefficients.First, the bias voltages of the microbolometer have to be adjusted according to the required object and ambient temperature ranges.The first bad pixel replacement procedure determines pixels that show no response or have an abnormal behavior.
The main calibration steps are non-uniformity correction (NUC) as well as gain and offset correction.As mentioned before, each microbolometer FPA shows a pixel non-uniformity due to the semiconductor fabrication process.This can be corrected by applying the two-point nonuniformity correction (NUC) based on a slope coefficient a ij and an intercept coefficient b ij : The NUC is bound to a certain ambient temperature ϑ amb,ref .Changes in the ambient temperature ϑ amb are trans-ferred inside the camera housing due to heat conduction and convection, and change the camera's and sensor's temperature with specific delay times and time constants.
The sensor temperature dependency of the pixel responsivity R V ,ij is determined using two switchable blackbodies at different constant temperatures.This ensures a constant incident radiant flux difference .Changes in R V ,ij are measured using the signal voltage difference V in relation to the sensor temperature ϑ fpa via the relation The regression function g V ,ij is based on several supporting points at different ambient temperatures and, hence, different sensor temperatures.Using this pixel gain function, the NUC-corrected signal voltage is weighted: The offset correction considers the temperature influences in Eq. ( 5).The offset voltage V 0,ij and the signal voltage part V pix are compensated for in relation to ϑ fpa .The signal voltage part V cam,ij corresponding to the disturbing camera radiant flux is estimated based on additional temperature information ϑ cam from inside the camera using a polynomial regression function of the second order: The offset correction function o V ,ij is pixel-specific and can be based on more than one camera temperature value, since the camera temperature measurement depends on the position of the thermometer: After this ambient temperature compensation, the signal voltage V off,ij should depend only on the object temperature ϑ obj .Pixels that cannot be corrected appropriately are rejected due to the second bad pixel replacement procedure.
The conversion of the signal voltage level into real temperature values is the last step.This transfer function is based on either a second-order polynomial or a Planck-like regression function that has to be determined using a sufficient number of measurements of known temperatures as supporting data points (Budzier and Gerlach, 2015).
The proposed compensation approach can be easily adapted for infrared cameras using microbolometer FPAs with TEC.If the sensor temperature ϑ fpa is stabilized, then the sensor parameters R V ,ij and V 0,ij stay constant and the intercept offset ij depends only on the camera temperature ϑ cam .For that reason, the gain correction step can be skipped and the offset correction step can be simplified.

IR camera setup
The mentioned calibration procedure will be demonstrated for an IR camera comprising a microbolometer sensor array without temperature stabilization (Table 1).
The sensor temperature ϑ fpa is provided by the detector itself and it is assumed that changes are uniformly distributed over the entire sensor array.The sensor specification describes the sensor temperature dependency of the pixel responsivity with a polynomial of the second order, and the sensor temperature dependency of the pixel offset voltage with a polynomial of the third order, The camera temperature is not homogeneously distributed inside the camera.Electrical losses derived from the digital signal processor (DSP) and the sensor array cause a temperature gradient between positions close to the sensor and in the inner side of the camera housing.For that reason, three temperature probes (LM61, Texas Instruments, USA) are placed inside the camera housing in different positions (Fig. 4).Each of the temperature probes TP 1 -TP 3 shows different temporal step responses to changes in the ambient temperature ϑ amb .The response time of the temperature probes, which indicates how fast temperature changes occur, varies from 39 s for TP 1 and 66 s for TP 2 to 105 s for TP 3 .The same relations occur in terms of time constants that give information about the temperature settling time until the steady state is reached.The raw signal voltage V raw,ij responds to an ambient temperature change, even before TP 1 reacts to those temperature changes, and reaches the steady state together with TP 3 .This demonstrates that each temperature probe provides different temporal information about occurring temperature changes, and should be used for compensation.The offset compensation based on different temperature probe values should benefit from having more information about temperature probes due to the possibility of capturing the cameras' temperature  changes completely with respect to the direction of its propagation.

Calibration
In the following, the experimental results of the proposed calibration method using the IR camera from Table 1 will be presented.First, the calibration will be explained.

Calibration stand
The calibration stand was designed to meet the requirements of the proposed calibration procedure.The changing ambient temperature was simulated by using a heating chamber with the infrared camera positioned inside.A lateral opening allows the camera to be face panel blackbodies or testing scenes (Fig. 5).
For the NUC and the ambient temperature compensation steps, two panel blackbodies were used (Table 2): (1) RCN 300 from HGH Infrared Systems, France, and (2) a self-built water bath blackbody.A self-built four-element blackbody comprising four independently working small Peltier element radiators was used for radiometric calibration and testing purposes.A mechanically moved plate is controlled to open and close the lateral opening of the heating chamber in order to guarantee a homogeneous chamber temperature distribution and to reduce the convection between the heating chamber and the outside.Several thermo-elements were placed inside and outside the heating chamber to observe the temperature changes.The blackbody temperatures were captured using pyrometers.

Non-uniformity correction (NUC)
The NUC is based on raw infrared images of at least two different blackbody temperatures that should be spread over the targeted measurement range.The surface temperature should be uniformly distributed over the entire camera field of view.The pixel-dependent slope and intercept values of the pixel response curves from Eq. ( 3) are equalized, yielding the deviation V ij : The resulting linear system of equations is solved in order to determine the pixel coefficients a ij and b ij (Fig. 6).After the NUC, each pixel shows the same behavior when looking at an object with the same surface temperature since the pixel responses follow the same so-called standard curve.However, this works only for certain ambient temperature conditions when the NUC images were taken.The accuracy of the  NUC can be improved by using more than two raw infrared images and applying the least-squares method to determine the NUC coefficients.

Gain correction
Here the relation between responsivity and sensor temperature pertaining to Eq. ( 11) is determined for each pixel and will be taken as the weighting function g V ,ij in Eq. ( 8).According to Eq. ( 7), different infrared images of two blackbodies with constant temperatures of 20 and 80 • C were used.Since the responsivity depends only on the absolute sensor temperature ϑ fpa and corresponds to a second-order polynomial, it is sufficient to use at least three pairs of infrared images taken at different ambient temperatures.Both blackbodies used (Table 2) cover the entire camera field of view.

Offset correction
The offset correction is based on a multivariate linear regression model comprising -polynomials of the second order for each temperature probe inside the camera (Eq.9), and -one polynomial of the third order for the relation between offset voltage and sensor temperature (Eq.12).
Since all temperature inputs are correlated with each other due to heat propagation and equalization processes, additional information on the temperature distribution is needed.It can be extracted from the cross-correlation of two or more temperature inputs.
The offset correction function o V ,ij in Eq. ( 10) comprises for instance 17 coefficients if all temperature inputs and all As mentioned earlier, all temperature inputs show different temporal behavior, which is used to separate them.An ambient temperature change regime with temperature steps from ±5 to ±20 K is applied to the camera inside the heating chamber, while the camera looks at a uniform blackbody with constant surface temperature.The camera temperature, the sensor temperature as well as other control temperatures were captured periodically.Figure 7 depicts the applied temperature change over time and illustrates the relation of the mean signal voltage in relation to the different temperatures.These relations contain all information about the offset changes due to the ambient temperature change.The coefficients of Eq. ( 14) were determined by applying the least-square fit to a polynomial (Bevington and Keith Robinson, 1992) in the offset calibration data.
Depending on the number and the kind of the used correction inputs, different correction uncertainty values in terms of absolute and spatial deviation can be achieved.Table 3 shows the comparison of different regression models based on the deviation of the mean absolute deviation and the mean spatial standard deviation of the pixel signal voltages over the sensor array.It should be noted that the more correction inputs are used, the lower measurement uncertainty values can be achieved.Figure 8 depicts the mean signal voltage and the spatial standard deviation versus time of the calibration using the offset correction function given by Eq. ( 14).

Radiometric calibration
The relation of the signal voltage V offset,ij after ambient temperature compensation and the observed object temperature ϑ obj can be estimated using a second-order polynomial.The uncertainty due to the regression can be reduced using a Planck-like approximation function based on four coefficients rbf and o, especially for interpolation outside the supporting points (Budzier and Gerlach, 2015).The inverted function defines the calculation rule to convert signal voltages into temperature values:

Comparison of shutter-based and shutter-less correction
The presented correction method for shutter-less IR cameras was compared to the common shutter-based approach using the calibrated IR camera from above and a second equivalent camera with a shutter.The ambient temperature was changed from 25 to 45 • C and back.The cameras were facing the same test scene one after another using the four-element blackbody.Figure 9 compares the results of the two measurements of one blackbody with a temperature of 76.6 • C.
It should be noted that the residual deviation of the absolute temperatures of the shutter-less camera is significantly lower than that of the camera with a shutter.Furthermore, due to the fact that there is no interruption of the measurement, the temporal variation is also lower compared to the thermal drift after the shutter procedure.

Conclusions
The presented calibration and correction approach for shutter-less and TEC-less microbolometer-based infrared cameras shows very promising results, especially compared with the common shutter-based correction.The main drawbacks are the long calibration time and the huge number of pixel-dependent correction coefficients.This correction is only suitable for measurements with ambient temperature changes that affect the infrared camera almost uniformly.In the next study, the long-term stability of the calibration parameters will be studied.

Figure 2 .
Figure2.Radiation components in an infrared camera.E obj radiation derived from the object depending on its temperature ϑ obj covering the projected solid angle ω obj , E cam radiation from the camera interior related to the camera temperature ϑ cam and the solid angle ω cam , as well as the radiant exitance M pix of the sensor depending on the sensor temperature ϑ fpa emitted into the projected solid angle ω pix .

Figure 3 .
Figure 3. Correction procedure for shutter-and TEC-less IR cameras.

Figure 4 .
Figure 4. Positions of the temperature sensor sites inside the camera housing and the detector.TP 1 (red) is placed on the front plate carrying the optics.TP 2 (green) is located on the front side of the optical channel.TP 3 (blue) is placed close to the detector and the signal processing unit on the back side of the optical channel.The fourth site is located inside the sensor array itself (violet).

Figure 5 .
Figure 5. Photograph of the calibration setup.

Figure 6 .
Figure 6.NUC coefficient matrices a ij (a) and b ij (b).The central symmetric shape of a ij is due to the relation to the pixel-dependent projected solid angle ω fov,ij .

Figure 7 .
Figure 7. Temperature courses during the offset calibration regime: (a) temperature sensor signals; (b) mean signal voltage versus the four temperature inputs.The shape of the steady-state curves corresponds to a polynomial of the second or third order.The deviations from these steady-state functions are due to the transient behavior during the temperature change.

Figure 8 .
Figure 8. Mean signal voltage and spatial standard deviation of the entire sensor array after offset correction of the calibration data based on 17 correction coefficients.The mean value shows a constant level except for minor peaks when the ambient temperature changes.The spatial level changes slightly depending on the ambient temperature.

ϑFigure 9 .
Figure 9.Comparison of the correction of (a) the shutter-less and (b) shutter-based IR cameras for ambient temperature jumps from 25 to 45 • C and back.The non-continuous shape of the shutter-based correction is due to the shutter procedure and the thermal drift occurring afterwards.The residual measurement uncertainty of the shutter-less approach is significantly lower due to the continuous online correction.

Table 1 .
Properties of the used infrared camera.

Table 2 .
Properties of the used blackbodies.

Table 3 .
Residual uncertainty after offset correction using different numbers and kinds of temperature inputs.TP 1 −TP 3 ϑ TP m • ϑ TP n possible corresponding cross-correlations are considered: