Qualification concept for optical multi-scale multi-sensor systems

This article describes a new qualification concept for dimensional measurements on optical measuring systems. Using the example of a prototypical multi-scale multi-sensor fringe projection system for production-related inspections of sheet-bulk metal-formed parts, current measuring procedures of the optical system are introduced. Out of the shown procedures’ deficiencies, a new concept is developed for determining the orientations and positions of the sensors’ measuring ranges in a common coordinate system. The principle element of the concept is a newly developed flexible reference artefact, adapted to the measuring task of the fringe projection system. Due to its dull surface, the artefact is optimized for optical measuring systems, like the used fringe projection sensors. By measuring the reference artefact with each fringe projection sensor and aligning the resulting data sets on a digital reference model of the artefact, sensor-specific transformation matrices can be calculated which allow transformation of the sensors’ data sets into a common coordinate system, without the need for any overlapping areas. This approach is concluded in an automated measuring procedure, using alignment algorithms from commercial available software where necessary. With the automated measuring procedure, geometrical relations between individual measured features can be determined and dimensional measuring beyond the measuring range of a sensor became possible. Due to a series of experiments, the advantages of the new qualification concept in comparison with the current measuring procedures are finally revealed.


Introduction
New production technologies, like sheet-bulk metal forming (Schaper et al., 2011), involve new challenges for dimensional measurements of the manufactured parts.In the case of sheet-bulk metal forming, metrological requirements of a production-related inspection arise from the short cycle time, the complex and filigree geometry, and varying surface roughness due to the high, irregularly distributed forming forces (Merklein et al., 2012).The challenges of inspecting complex workpieces can be explained by considering the "golden rule of measuring metrology" (Berndt et al., 1968).In 1968 Georg Berndt developed a rule for selecting appropriate measurement systems.Therefore, the measurement uncertainties of the measurement systems have to be known.Following the recommendation of the golden rule, the measurement uncertainty should be at least less than a fifth, and better less than a tenth, of the tolerance width.If this minimum requirement can be met, it ensures that the measurement results are accurate enough (Loderer et al., 2013).To achieve these requirements, a prototype of a multiscale multi-sensor fringe projection system was developed, designed for a production-related environment (see Fig. 1).
The main parts of the prototype systems are three different types of fringe projection sensors with varying measuring ranges and resolutions (see Table 1).
To get an overview of the workpiece and also to measure large features simultaneously, an exchangeable fringe projection sensor with a measuring range of the size of the workpiece is installed (Ohrt et al., 2012).For the measurement of filigree elements, two other types of fringe projection sensors are used as detail sensors which can be arranged around the workpiece.Each of these two sensors captures only one feature, but at a resolution adapted to the feature's size.Whereas there is only one overview sensor available, there are up to  eight detail sensors 1 and up to four detail sensors 2 (Loderer et al., 2015).
The process of gathering measuring results out of multiscale data sets is divided into four main steps (see Fig. 2): firstly, measurements were done by all selected fringe projection sensors automatically.Then the sensors' data sets have to be transformed into a common coordinate system and combined into one holistic data set by a merging process.For this purpose the data sets are roughly aligned in a coarse registration by selecting corresponding points in each data set manually.Only if there is a large overlapping area with at least one significant feature can an automatic algorithm, e.g.presented in Shaw et al. (2013), be considered, but often this requirement cannot be met.In contrast to this for the following fine registration, various automatic algorithms are available.For the presented procedures, the bestfit algorithm of the commercial available Polyworks IMInspect 2014 software is used, which provides numerous settings for multi-data-set alignments.Next to point cloud data sets, polygonal models can also be aligned.The last step of the standard measuring procedure is the evaluation which can also be performed automatically.That procedure requires at least a small overlapping area.However, even if enough cor- responding points are available, the procedure is neither fast nor accurate enough to benefit from the high accuracy of the fringe projection sensors, as can be seen in the results presented in Fig. 8.
In order to get a reliable geometrical relation between individual measured features and the ability to transform data sets without overlapping areas into a common coordinate system, a qualification concept, adapted to the properties of the prototypical multi-sensor multi-scale measuring system, has to be developed.
For the underlying research the shown measuring procedures have been selected as the most suitable approaches for further adaptations of the multi-scale multi-sensor fringe projection system.Besides the explained four steps, there are other approaches for combining measurement data, e.g.presented in Puente León and Kammel (2003), Komander et al. (2014) and Keck et al. (2014).

Qualification principle
To prove the basic idea of the qualification of optical multiscale multi-sensor systems, a flat reference artefact was used (see Fig. 3).On the reference artefact, surface lines of differing distances as well as radii of differing sizes were milled in, and thereby a unique surface structure was created (Kästner et al., 2013).
The measuring ranges of the considered optical sensors are significantly smaller than the artefact's size.Setting up a multi-sensor measurement, the sensors' measuring ranges are positioned onto a measuring object.Subsequently, the measuring object is replaced by the reference artefact.Each sensor now measures a part of the reference artefact's surface and, due to the unique surface structure, the position of each data set can be allocated.By a manually coarse registration using point alignments and a following automatically fine registration, each data set can be aligned to a CAD (computer-aided design) model of the reference artefact.All necessary transformations to get the data sets in the correct positions can be expressed in transformation matrices.These matrices represent the sensor orientations and have to be saved.Replacing the reference artefact by a measuring object, the data sets of each sensor can be transformed in the correct position again by using the transformation matrices of the sensor.If the measuring range of a fringe projection sensor is changed, the qualification procedure has to be done once more.
With the flat reference artefact, a qualification field size of about 1800 mm 2 can be used which is equal to the surface size.Due to the flat design, only a lateral qualification is possible, whereas all sensors have a similar vertical position.
The important advantage of the qualification principle is the loss of need for corresponding areas.Even data sets that do not overlap can be located correctly, and thereby dimensional measurements with optical multi-scale multi-sensor systems are enabled.Moreover, the time-consuming manual coarse registration has to be done only in the qualification procedure.Once all transformation matrices are available, the steps of the measuring procedure run automatically (see Fig. 4).

Flexible qualification concept
A crucial disadvantage is the flat shape of the reference artefact.Sheet-bulk metal-formed objects and the prototypical multi-scale multi-sensor fringe projection system designed for measuring often are of round shapes with varying diameters (see Fig. 5) (Merklein et al., 2015).With a flat reference artefact, the fringe projection sensors can only be qualified if their measuring ranges are positioned at the same height and oriented similarly.However, complex features like cylinders require differently positioned sensors with differing heights of their measuring ranges.The flat reference artefact is not capable of fulfilling these demands.
Thus, a flexible qualification concept was worked out to also allow dimensional measurement of complex features by using optical multi-sensor systems.This concept is mainly based on a new flexible and, adapted to the demands of sheetbulk metal forming, reference artefact (see Fig. 6).The basic principle of the reference artefact, which is a unique surface structure as well, can be found on cylindrical "reference heads".These heads are mounted on "adjustment arms", which are adjustable in the lateral and vertical directions.Thereby, measuring ranges do not have to be set up at the same height, but rather can be oriented freely.In order to optimize the reference heads for optical measuring systems, the surfaces are glass-blasted to generate dull and very measurable surface structures.
With the flexible reference artefact, a qualification field size of about 15 000 mm 3 can be used.Due to the vertical adjustment of the reference heads, a lateral as well as vertical qualification is possible.
According to the qualification concept, ten main steps have to be considered when setting up a complete multi-sensor measurement (see Fig. 7).Firstly, the measuring ranges of the fringe projection sensors have to be positioned on the measuring object.Then the measuring object is replaced with the reference artefact and the reference heads are positioned into the measuring ranges.At least one reference head has to be inside the measuring range of each sensor.In the next step, measurements of the fringe projection sensors are triggered.To generate a reference polygonal model of the reference artefact, it is digitized by using an optical sensor with a bigger measuring range.The quality of the measurement with the overview sensor is crucial for the qualification concept.The more accurate the overview sensor and the higher the quality of the digitization, the more accurate the qualification procedure's result.In the shown experiments, the overview sensor is used to digitize the complete reference artefact.
Next the data sets of the fringe projection sensors are aligned to the digital reference polygonal model of the reference artefact.From these alignments, transformation matri- ces for each data set are calculated, which express the orientation of each fringe projection sensor in a common coordinate system.The qualification procedure finishes by replacing the reference artefact with the measuring object.With measurements of the measuring object, the following measuring procedure starts.Using the transformation matrices, the data sets of the fringe projection sensors can be transformed into the common coordinate system and combined into one common data set by a merging process.By repeating the measuring procedure, this qualification concept enables holistic dimensional measurements of complex features.

Comparison
To detect the advantages provided by the developed qualification concept, a comparison between the standard measuring procedure and the automated measuring procedure with the new qualification concept is worked out.
Therefore, the height (1 mm) and width (3 mm) of a step height standard have to be measured (see Fig. 8).This measuring task represents the need for multi-sensor measurements: due to optical effects like shadowing and technical limitations, e.g. the maximum thread angle, the measurement of both parameters by using only one fringe projection sensor is not possible.Only by changing the position of the sensor or step height standard and performing more measurements can the features be tediously detected by a single sensor.For a fast and reliable measurement, more sensors with differing measuring positions are needed.In order to compare both measuring procedures, the deviations of height and width of the qualified values are considered as parameters.The calculation of heights and width is done with Polyworks IMInspect 2014.A consideration of DIN EN ISO 5436-1 for calculating step heights is not possible due to software restrictions.Contrary to the standard, Polyworks IMInspect 2014 calculates two Gaussian plains and evaluates the vectorial distance between both as the step height.This approach is not  standardized and only generates valid results when the two plains are parallel.Due to the use of a step height standard, the parallelism is ensured, and thus the Polyworks IMInspect 2014 approach is permissible.
In addition to the deviations of height and width of the qualified values, the time needed for performing all necessary steps is evaluated, too.
The results for the deviation of width show a significant difference between the automated measuring procedure the corresponding new qualification concept and the former standard measuring procedure.Whereas the measuring results, gathered by using the standard measuring procedure, are between 0.7 and 0.8 mm smaller than the qualified value, the deviation averages 0.42 µm using the automated measuring procedure.Considering the deviation of height, there is also a difference between both procedures.The deviation averages 5.0 µm for the automated procedure and −0.3 µm for the standard procedure.Although this difference seems to be small, its statistical significance is proven by using a Student's t test.However, focusing on the results' distributions, the reliability of the automated measuring procedure becomes obvious.The automated procedure provides continuously the same value for results, which is caused by using the same sensors' transformation matrices for all ten data sets, whereas the values of the standard procedure are spread between 5 and 3 µm.
Comparing the duration needed for performing all required steps, the automated measuring procedure takes less time.Even though the detected difference is only about 2 min, the automated process is of benefit the more data sets are used.
When performing the standard measuring process, the needed time increases nearly linearly, and one time step rep- resents one completed data set.In contrast, the most timeconsuming steps of the automated procedure are steps 1 to 3.These steps belong to the qualification procedure and include the positioning and measuring of the reference heads, the digitalization of the reference artefact and the registration of the reference heads' data sets in order to calculate the transformation matrices.Once the matrices are available and a script for an automated measuring process is created, which is done in the fourth step, the needed time for the following data sets is significantly shorter.

Application
With the comparison of the standard and automated measuring procedures by using the qualified step height standard, the advantages of the automated measuring procedure could be shown.In order to prove the advantages in a measuring task similar to a task for which the multi-scale multi-sensor fringe projection system was developed, the automated measuring procedure is applied in an inspection of a sheet-bulk metal-formed part.
With a process of the sheet-bulk metal forming, a multiplegap structure is formed in DC04 sheet metal (see Fig. 9).In order to have precise data for evaluating and further improving the process, a holistic detection of the relevant middle section of the multiple-gap structure is necessary.Such detection is not possible by using only one conventional fringe projection sensor with a large enough measuring range, e.g. the overview sensor.The smooth and thereby highly reflective surface of the formed section in combination with the structure's flank angle leads to missing data points and gaps in the data set (Loderer et al., 2015).Moreover, due to small dimensions of the multiple-gap structure, the fringe projection sensor's resolution would be not accurate enough.Using only one fringe projection sensor with a smaller measuring range but instead an appropriate resolution, the whole relevant section cannot be detected at once.This conflict leads to a multi-scale multi-sensor measuring set-up, similar to the measurement of the step height standard, where one side of the structure is detected by one separate sensor of the type of detail sensor 2 and the gap root's detection is done by a third, more accurate sensor of the type of detail sensor 1.In order to allow dimensional measurements of flank angle, gap root radius and distance between gap tip and gap root, a qualification procedure has to ensure a precise and also robust alignment of the three resulting data sets.
To evaluate the stability and robustness of the automated measuring procedure, one data set of each sensor is run through the procedure 10 times, again using Polyworks IMInspect 2014.This means each data set is aligned manually onto a reference data set of the reference artefact for a coarse registration, and subsequently a fine registration by Polyworks IMInspect's best-fit algorithm is done.The resulting transformation matrices were used to transform the data sets correctly in a common coordinate system and the feature evaluation was done finally.Due to using the same data set for each sensor 10 times, the same results for the evaluated features should always be calculated.If there are any variations, these can be clearly matched as deviations in the automated measuring procedure.
The large spread of the manually coarse registration becomes obvious when comparing the data sets' translations and rotations to the translations and rotations of the bestfit algorithm, which were considered as a reference (see Fig. 10).Picking corresponding points in the sensors' data sets and in the reference artefact's reference data sets in the manually coarse registration leads to differing translations and rotations.Obviously, by manual registration only, a stable alignment is not ensured.Nevertheless, when performing a best-fit fine registration after the manually coarse registration, the spreads can be eliminated.Thereby the best-fit fine registration always provides the same translation and rota- tion in the ten repetitions without any spread, whatever the manual input registration deviation was.
After the fine registration, the data sets were aligned and the features could be evaluated (see Table 2).Due to always having the same values for translations and rotations for each sensor data set of the best-fit algorithm, the results for the considered features are also identical every time without any spread or deviation.In this manner the robustness of the automated measuring process could be proven in a measuring task for which the multi-scale multi-sensor fringe projection system was designed.In a real inspection set-up, the manual registration step is necessary only once in the qualification process for allowing a subsequently fine registration.With the transformation matrices generated then, the data sets of the following measurements are aligned correctly and no manual interaction is required.

Conclusion
In this article a new qualification concept, optimized for optical multi-scale multi-sensor measuring systems, was introduced, which allows dimensional measurements of features larger than the measuring range of the optical sensor.The basic principle was proven by a flat-shaped reference artefact using the example of a prototypical multi-scale multi-sensor fringe projection system.A unique surface structure of the reference artefact ensures that the measured data sets can be aligned on a polygonal reference model, and thus transformation matrices for each fringe projection sensor can be calculated.These matrices contain the positions and orientations of each sensor, expressed in a common coordinate system.Thereby the correct transformation of all measurement data sets in a common coordinate system is enabled in order to generate a holistic data set.The basic principle was transferred to a flexible reference artefact, adapted on the shape of sheet-bulk metal-formed parts for whose inspection the prototypical multi-scale multi-sensor measuring system was designed.Together with the new qualification and measuring procedures, an automated and reliable measurement of complex workpieces is possible now.Comparing the new qualification concept with the former standard measuring procedure by setting up a series of experiments, the gathered advantages become obvious.
In order to test the automated measuring procedure, which contains the new qualification concept, a measuring task of a sheet-bulk metal-formed multiple gap structure was set up.
Here, large variations of the manually coarse registration became obvious, which were corrected by the subsequent bestfit fine registration.By repeating the qualification procedure 10 times with the same data sets, variations in the features' results can be matched to deviations in the procedure itself.However, the final feature evaluation provides the same result for each feature in every trail, and thus the stability and robustness of the qualification procedure could be proven for dimensional measurements with optical multi-scale multisensor measuring systems.

Figure 1 .
Figure 1.Technical sketch of the idea without an overview sensor (left side) and realized set-up of the prototypical multi-scale multi-sensor fringe projection system (right side).

Figure 2 .
Figure 2. Automatic (A) and manual (M) steps of the standard measuring procedure.

Figure 3 .
Figure 3. Flat reference artefact for testing the qualification principle.

Figure 4 .
Figure 4. Automated (A) measuring procedure by using transformation matrices.

Figure 5 .
Figure 5. Workpiece demonstrator of sheet-bulk metal-forming processes with its differing sizes and shapes.

Figure 7 .
Figure 7. Complete steps for the qualification and measuring procedure.

Figure 8 .
Figure 8. Considered parameters for comparing the standard measuring procedure (standard) and the automated measuring procedure (automated) with the new qualification concept (upper left corner).Results for deviations of height (upper right corner), width (lower left corner) and comparison of needed time (lower right corner).

Figure 9 .
Figure 9. Multi-scale multi-sensor measurement of a sheet-bulk metal-formed multiple gap structure (upper side) and the corresponding measuring task (lower side).

Figure 10 .
Figure 10.Ranges of data sets' translations and rotations done by a manually coarse registration (boxplots) in comparison to best-fit fine registration (pointed line).

Table 1 .
Technical specifications of the fringe projection sensors.

Table 2 .
Results of feature evaluation.