·Table of Contents ·Aeronautics and Aerospace | Automatic mosaicing with error equalisation of non-destructive testing images for aerospace industry inspectionBogdan J. Matuszewski , Lik-Kwan Shark , Martin R. VarleyDepartment of Engineering and Product Design, University of Central Lancashire Preston, PR1 2HE, United Kingdom John P. Smith BAE SYSTEMS Warton, Preston, PR4 1AX, United Kingdom Contact |
(1) |
In some cases, there may be no analytical function to model this geometrical transformation with sufficient accuracy, or the function is too complex. In these cases, the movement of each image pixel has to be described separately (known as dense motion representation) [1]. This means that the number of parameters to be estimated can be twice the number of pixels in the overlapping area. The number of parameters to be estimated can be reduced by approximation of the function f using two-dimensional splines controlled by a smaller number of displacement parameters defined on a coarse spline control grid [2]. A further and significant reduction in the number of parameters to be estimated can be achieved by using simpler image transformation models. One of the most common models is the two-dimensional projective transformation with eight parameters for function f. Using homogeneous co-ordinates this function is defined as:
(2) |
The corresponding Cartesian co-ordinates are [x/w ,y/w ]^{ T } , and [x/w ,y/w ]^{ T } . It can be shown that the two-dimensional projective transformation is an accurate model of the geometrical image transformation for arbitrary sensor movements and planar objects [3]. If the sensor motion is restricted (e.g. panning or rotation only), the number of parameters to be estimated can be further reduced. When the component surface is not planar but can be approximately represented as a piecewise planar, the two-dimensional projective transformation model can be used locally [4].
One of the simplest image displacement models is the rigid transformation given by:
(3) |
This model has three parameters, namely, the translation vector co-ordinates T=[Tx,Ty]^{T} and rotation angle a. The meaning of these parameters is illustrated in Figure 1 which shows a rigid displacement between two images.
Fig 1: Rigid transformation model of image displacement (T_{i,j} represents translation vector between images `j' and `i' and a_{i,j} rotation angle of image j in image i co-ordinate system) |
Although simple, the rigid transformation is an accurate image displacement model for many NDT inspection methods, especially when the component to be inspected has a relatively simple geometry. For example, this model can be used for mosaic construction of ultrasonic, shearographic and thermographic images if the component surface is perpendicular (or almost perpendicular) to the optical axis of the camera. The mosaic construction of x-ray images is also possible using this model if the component to be tested is relatively thin and has a simple internal structure.
(4) |
where I _{j} (x _{k} , y _{k}), is the displaced k-th pixel of the j-th image in the co-ordinate system of the i-th image. This method can be used to estimate the parameters of the model described by equation (2) which includes the rigid transformation.
The estimation procedure usually requires performing numerical minimisation in order to determine the values for the unknown parameters [4]. Variations of this method with extra constraints added and images modelled as random fields have also been used to estimate dense motion fields [6].
When image displacement is modelled by the rigid transformation, methods based on the cross-correlation measure are often used. Many different variants of this measure can be employed [7] depending on the image structure, robustness required, accuracy and computation cost. In the work described in this paper, the cross-correlation coefficient function, which measures the similarity between pixels in different images covering the same component area, was used. This function is defined by:
(5) |
where I _{j} (x _{k} , y _{l}) and I _{i} ( x_{k} , y_{l}) denote two images with the respective image mean values _{j} and _{i}. Since a higher
value of the correlation coefficient function indicates a higher degree of similarity between the two images, it was applied to the overlapping areas of neighbouring images to estimate the transformation parameters. The translation vector and rotation angle are estimated iteratively. Following the initial estimation of the translation vector, the rotation angle is then estimated by moving the second image to the position defined by the translation vector found and transforming the overlapping images' sub-areas to polar co-ordinates (as rotation in Cartesian co-ordinates becomes translation in polar co-ordinates). The whole process is repeated until no significant changes in the estimated parameters are detected. To reduce computation cost, the calculations are performed in a hierarchical manner using image pyramid decomposition [8] with the translation vector and the rotation angle found for a lower resolution level used as the initial estimates for a higher resolution level.
A review of other methods available for the estimation of the image displacement parameters can be found in [9]. The estimation yields a set of the parameter values which define displacement transformations (local transformations) for all possible image pairs with sufficient overlapping. These parameters are denoted by
(6) |
Fig 2: Diagram of `rigid' mosaic of five images |
In the diagram shown in Figure 2, the current position and orientation of the images with respect to a common co- ordinate system in the mosaic are denoted by:
(7) |
Using the differences in the positions and orientations between the images, the local transformations required can be computed for the current mosaic and are denoted by:
(8) |
Since these computed local transformations are consistent (because they are obtained from the image positions and orientations in the mosaic), it implies the following relations to be true for the mosaic shown in Figure 2:
(9) |
(10) |
However, the relations (9) and (10) are unlikely to be true for the local transformations previously estimated, because they are obtained based on the similarity existing in an image pair without relating to their positions and orientations in the mosaic, and are also contaminated by some inherent estimation errors. Using the previously estimated local transformations based on image similarity, the images can still be placed into a mosaic as long as no closed loop is formed from them in the process. For example, in Figure (2), a mosaic could be built using the sequence of the estimated local transformations . Errors are accumulated as the local transformations are cascaded to build the mosaic. Consquently, the local transformation estimated using image similarity for images one and five, is usually not equal to the local transformation computed, (T_{1,5}a_{1,5} ) using their positions and orientations of (x_{ 1} ,y_{ 1} ,q1 ) and (x_{5} ,y_{5} ,q_{5} ) in the current mosaic. As a matter of the fact, for large mosaics (containing dozens or hundreds of images) the local transformation estimated based on image similarity can differ significantly from those computed based on image positions and orientations. This means that the accumulated errors of the local transformation can cause a serious degradation in the mosaic quality and the estimated local transformations based on image similarity should not be used directly for mosaic construction.
Let an optimum set of the local transformations to achieve a best alignment of all images by distributing the errors evenly across the whole mosaic be denoted by:
(11) |
The optimal local transformations should not only satisfy the consistency conditions (similar to conditions (9) and (10)), but also be as close as possible to the local transformations estimated based on image similarity (because each one is a good approximation of the true local transformation). The latter constraint can be expressed as
(12) |
The determination of the optimal local transformations is achieved via the minimisation of a cost function constructed based on the sum of the squared differences between the two local transformations with one estimated using the image similarity and the other computed using the image positions and orientations in the mosaic:
(13) |
where
(14) |
(15) |
In the algorithm implementation, the position and orientation of the first image is used as the reference to anchor the whole mosaic (with (x_{1},y_{1},q_{1} )= (0,0,0 )for example). Minimising the cost function of equation (13) yields a set of optimum position and orientation values in p for the images in the mosaic:
(16) |
and the optimal local transformations are computed from using
(17) |
These optimal local transformations satisfy the consistency conditions as they are computed in equation (17) using the image positions and orientations in the mosaic. Furthermore, the optimal local transformations also satisfy condition (12) as the cost function reaches the lowest possible value when these transformations equal exactly to the local transformations estimated based on image similarity.
To simplify the notation, the following mapping operation is employed to convert the two subscripts used in equation (13) to one subscript:
(18) |
where n is the total number of the local transformations in the mosaic. Rewriting equation (13) using one subscript yields
(19) |
As the minimisation defined by equation (16) is a non-linear problem, a numerical method based on the Levenberg-Marquardt (LM) algorithm [10] is used. This algorithm requires the gradient and an approximation of the Hessian matrix of the cost function E(p), they are calculated using the partial derivatives of for 1£ i £ 3n ,1£ j £ 3m and are given by the following equations:
(20) |
(21) |
(22) |
All others entries of the Jacobian matrix are zero. The initial positions and orientations of the images in the mosaic can be obtained either from the NDT inspection system or if this is not possible, they can be computed from the estimated transformations between images in the acquired image sequence.
After the correct alignment of all images, mosaic composition based on the Voronoi diagram construction method is performed to combine the overlapping areas of the neighbouring images into one image. In this method, the values of each pixel in the mosaic is taken from one image only with the image having the image centre closest to the location of the pixel concerned.
Fig 3: Simulated mosaic |
(23) |
Fig 4: Histograms of estimated translation errors from (with errors in horizontal and vertical co-ordinates in blue and red respectively) and rotation error from |
Fig 5: Histograms of translational errors from (with errors in horizontal and vertical coordinates in blue and red respectively) and rotation errors from computed using initial image positions |
Using the algorithm of error equalisation described in the previous section, the histograms of the errors between the equalised local transformations and the estimated local transformations is shown in Figure 6, whereas the histograms of the errors between the equalised local transformations and the exact local transformations is shown in Figure 7. From these error histograms, improvements on the quality of the mosaic is obvious as the maximum translation error is about 0.6 pixel and the maximum rotation error is about 0.5 degree. Comparing Figure 7 with Figure 4, the errors from and j are seen to be smaller than the local transformation estimation errors from
Fig 6: Histograms of equalised translational errors from (with errors in horizontal and vertical co-ordinates in blue and red respectively)and rotation errors from |
Fig 7: Histograms of equalised translation errors from (with errors in horizontal and vertical co-ordinates in blue and red respectively)and rotation errors from |
Fig 8: Three ultrasonic sub-images of an aircraft component | Fig 9: Constructed ultrasonic mosaic for images shown in Figure (8). | Fig 10: Mosaic of 132 x-ray images. |
Figure (10) shows the mosaic of 132 x-ray images of an aircraft component captured by using a real-time radiographic system. In this case, translation only was used to model the image displacement with the parameters estimated using the cross-correlation coefficient function. The initial positions of images were acquired from the real-time radiographic machine. Although the rigid transformation is only an approximation of the exact image displacement model, the developed method was able to produce the satisfactory quality mosaic in this case also.
© AIPnD , created by NDT.net | |Home| |Top| |