·Home ·Table of Contents ·Aeronautics and Aerospace

Region-based wavelet fusion of ultrasonic, radiographic and shearographyc non-destructive testing images Bogdan J. Matuszewski, Lik-Kwan Shark, Martin R. Varley, Department of Engineering and Product Design, University of Central Lancashire Preston, PR1 2HE, United Kingdom John P. Smith, BAE SYSTEMS Warton, Preston, PR 1AX, United Kingdom Contact

Abstract

This paper describes image fusion techniques developed for integration of multi-modal non-destructive (NDT) testing images. The paper starts with a brief description of the image registration technique used to match each NDT image to the corresponding component CAD model. Three fusion techniques are then presented in detail and applied to real ultrasonic, radiographic and shearographic images acquired from an aircraft component. From the fused images obtained, the region-based fusion approach is shown to offer a superior quality for defect interpretation, because the fusion parameters are optimised according to the unique geometrical and material structure contained in different parts of the component under inspection.

1. Introduction

Ultrasonic [1], radiographic and shearographic inspection techniques are three complementary non-destructive testing (NDT) methods available for operators to determine the structural integrity of an aircraft component. As the image features generated by each type of inspection technique are very different from one another, each inspection method requires its own highly trained operators to accurately classify the observed image features as component defects or image artefacts inherent in that particular inspection method. Consequently, the use of these three techniques, as independent and separate processes for the detection of different types of defects, significantly increases the overall manufacturing cost due to the requirement of a significant amount of inspection time (especially in the case of inspecting large primary structures) and needs highly trained staff. One possible method to reduce inspection time while maintaining full defect detection capability has been previously proposed by the authors [2], whereby the three different NDT images are combined by wavelet fusion to enable defect interpretation to be carried out on one fused image instead of three separate images. However, as an aircraft component is likely to consist of different material structures, improvement of defect/structure visibility in one part of the aircraft component in the wavelet fused image is likely to be accompanied by deterioration of the defect/structure visibility in another part of the aircraft component. This leads to the region-based wavelet fusion method proposed in this paper.
After introduction of the necessary pre-processing operations to align ultrasonic, radiographic and shearographic images based on the component wire-frame CAD model, the paper presents three fusion techniques with one of them being the proposed region-based wavelet fusion method. Based on the structural and material information available in the CAD data, the proposed method divides the NDT images into several regions having the same material structure thereby enabling the wavelet fusion to be performed with its fusion parameters optimised according to the unique characteristic patterns in each decomposed region. The effectiveness of the proposed method is demonstrated by combining three real ultrasonic, radiographic and shearographic images acquired from a practical aircraft component to show a significantly better fused image for reliable defect interpretation and component sentencing

2. Image alignment

Due to different resolutions resulting in different image sizes for the same component and perspective distortions inherent in image acquisition, it is not possible to directly fuse NDT images without first performing image alignment. One possible way to correctly align NDT images is to use the component wire-frame CAD model as a geometrical reference model. By correcting perspective distortions and accurately mapping each NDT image acquired from an aircraft component to its corresponding wire-frame CAD model, the geometrical relationships between the NDT images are established for image fusion. An automatic image registration method for aligning NDT images with the CAD model has been developed by the authors [3], and consists of three main processing stages, namely:
1. Feature detection in the NDT image of the component under inspection according to those features selected from its corresponding CAD model. This is achieved by using the Hough transform operated in an iterative mode.
2. Feature matching between the NDT data features detected and the corresponding CAD model features selected. This involves generation of a better correspondence between the NDT data features and the CAD model features using a numerical combinatorial optimisation method, and minimisation of the matching error by selecting the best transformation to rotate, translate and scale the NDT data features to reduce their distances to the corresponding CAD model features.
3. Matching refinement to achieve high precision image registration of the NDT image with an accuracy of ±1 pixel. This is based on a numerical minimisation method to estimate the position and orientation of the component in 3D space projecting to the image plane.

Using a practical aircraft component as an example, Figures 1 and 2 show the final results of aligning ultrasonic and radiographic images with the corresponding wire-frame CAD model. From these figures, while the internal structure of the component is seen to be clearly visible in the radiographic image due to the very high resolution inherent in the radiographic technique, two defects corresponding to the two white patches are apparent in the ultrasonic image.

Fig 1: Registered ultrasonic image

Fig 2: Registered radiographic image

3. Structure decomposition

With the NDT images correctly mapped to their corresponding CAD model by automatic image registration, it is a relatively easy matter to utilise the structural and material information available in the CAD data to divide the NDT image into several viewing layers having the same material structure, thereby enabling fusion to be carried out on component sub-structure in a hierarchical manner. An important advantage of structure decomposition for image fusion is to facilitate different fusion strategies for different material structures to improve fusion quality. Additionally automatic defect detection and segmentation based on region and texture can be carried out in the wavelet domain. This can be achieved by taking the advantage that a recognisable regular pattern should be present in a particular viewing layer with the same material structure if it does not contain a defect.
From the CAD data of the component with the ultrasonic and radiographic images shown in Figures 1 and 2, the three material structures, namely, a thin carbon-fibre layer, a thick honeycomb structure with carbon-fibre skin, and transition regions between these two are shown as red, blue and green respectively in Figure 3.

Fig 3: Component structure mask derived from CAD data

4. Wavelet domain fusion

The aim of image fusion methods described in this section is primarily to provide human observers multi-technique NDT data in an integrated form for easier and better defect interpretation. To achieved this aim, a fusion technique should
1. preserve all relevant information contained in the input data,
2. introduce no artificial objects particularly if they are not consistent with the input NDT image data,
3. eliminate irrelevant information and noise, and
4. represent different multi-modal data features in a way that human operators can easily distinguish between them.
The difficulty of making the salient features contained in different NDT data sets simultaneously visible leads to the use of the discrete wavelet transform (DWT) As different image features can be compactly represented by a small number of wavelet coefficients at different resolutions in the wavelet domain, the important features of each NDT data set can be preserved without significant loss in the fused image constructed by using the inverse DWT (IDWT), if the wavelet coefficients of the images to be fused are correctly combined.
Figure 4 shows the general scheme of the wavelet domain fusion.

Fig 4: Wavelet domain fusion

As shown in Figure 4, the first stage of wavelet fusion is to perform the two-dimensional (2D) DWT on NDT images. By using a two-channel filter bank consisting of a lowpass filter and a highpass filter with impulse responses derived from a particular wavelet basis function, the 2D DWT is implemented as two one-dimensional(1D)DWTs,The first 1D DWT is performed along the horizontal direction of each NDT image with each row being treated as a 1D signal.By performing lowpass and highpass opearations on each row followed by a 2-to-1 down-sampling operation to discard every other column, the first 1D DWT produces two outputs for each NDT image, namely: the horizontal lowpass output and the horizontal highpass output.The second iD DWT is performed along the vertical directionof the outputs produced by the first1D DWT with each column of the horizontal low pass output and the horizontal high pass output being treated as a 1D signal . By performing lowpass and high pass operations on each column followed by 2-to-1 down sampling operation to discard every other row, the second 1D DWT produces four first level wavelet coefficients for each NDT image, namely: the vertical lowpass of the horizontal lowpass output, w^k_l,a(where superscript k denotes the type of input images with 1 for ultrasound, 2 for X-ray and 3 for shearographic image), the vertical highpass of the horizontal lowpass output,w^k_l,hand the vertical lowpass of the horizontal highpass output,w^k_l,vand the vertical highpass of the horizontal highpass output,w^k_l,dwhile w^k_l,a corresponds to the first approximation of each NDT image,w^k_l,h,w^k_l,d,w^k_l,v correspond to the horizontal,diagonal and vertical detail respectively.Applying the whole process again to w^k_l,ayields wavelet coefficients w^k_2,a,w^k_2,h w^k_2,d, and w^k_2,v at the second resolution level as shown in Figure 4,and the entire process can repeated until the desired decomposition level is reached.
Figure 5 shows the ultrasonic,radiographic and shearographic sub-images of the smaller honeycomb area with the defect visible on the bottom left of the ultrasonic and shearographic images and their corresponding DWT with 4 decomposition levels based on the symlet wavelet of order 5.

Fig 5: (a)-(b) Ultasound image and its DWT,(c)-(d)X-ray image and its DWT,(e)-(f) shearographic image and its DWT

From Figure 5 the ultrasonic defect feature is seen to dominate the coarsest approximation to its low frequency nature, the internal structures in the radiographic image are seen to dominate the horizontal, diagonal detail levels to their high frequency nature ,and the shearographic defect feature is seen to dominate the coarse level detail coefficients due to the relatively low frequency 2D oscillations (visible as an elliptical pattern in Figure 5(e)) After the wavelet decomposition of each NDT image to produce multiple sets of the different wavelet coefficient sets denoted W¹to W ^p, they are combined together to form a single set of the wavelet coefficients for the fused image. This process can be represented by the formula:

(1)

where F_m,o,i,j(·,...,·,p) is the function representing the fusion rule for the wavelet coefficients w^f_m,o(i,j) of the fused image (see Figure 4), and vector p represents additional parameters used by the fusion rule. From this general formula, it follows that the fusion rule theoretically can depend on decomposition level denoted by m, orientation (namely a, h, d, or v) denoted by o, and coefficient location denoted by i and j. In the following, three different fusion methods are described and applied to the NDT images shown in Figure (5). For the first two methods, fusion rules do not depend on the location (i,j) of the fused coeffi cient w^f_m,o(i,j),whereas for the third method fusion rule depends on the coefficient location by utilising the information of the image structure decomposition described in section 3.

(a) Fusion by averaging
In this method coefficient of the fused image are computed according to the formula:

(2)

where A(k,m,o) are weighting factors selected for image k, at decomposition level m with orientation o. This fusion method simply performs a weighted averaging operation of the wavelet coefficients. The weights A(k,m,o) are selected before fusion based on typical image contents. For example, if it is usually true that the DWT coefficients at level m and orientation o computed for image k are significant for the representation of some important image features, then A(k,m,o) will have a relatively high value; if they represent mostly noise or irrelevant information from the defect detection point of view, then A(k,m,o) will have a small value. This fusion method does not take into account the spatial variation of the image contents or the actual coefficient values of the images. For the fusion results shown in the next section, A(k,m,o)=1/P for all k,m and o except that A(1,1,o)=A(3,1,o)=0 and A(2,1,o)=1 for all orientations.

(b) Fusion by energy comparison
In this method, the wavelet coefficients of the fused image are computed according to the formula:

(3)

where e^k_m,o(i,j) is the local energy computed for the DWT coefficients over a window of size (d+1)X(d+1) with the window centre located at (i,j) at level m and orientation o:

(4)

Using this method the wavelet coefficient from image k is included in the IDWT to construct the fused image if its local energy is greater by a specified threshold value, T, than the local energy computed for the corresponding wavelet coefficients of other images. If the difference between the local energy levels computed for any image pairs is less than or equal to T, then the average of the corresponding wavelet coefficients is used as the fused wavelet coefficient. This method does not utilise the structure decomposition information, but the fusion rule depends on the actual coefficient values of the images. For the fusion results shown in the next section, a window of size 5X5 (d=2) was used for the local energy computation and T=1 was selected as the threshold value.

(c) Region-based fusion by multi-resolution feature detection
In this method the wavelet coefficient of the fused image are computed according to the formula:

(5)

where:

(6)

and q^k_m,o(·,·) is the mask matrix obtained as the result of binary morphological `clean' and `close' operations [5] applied to binary matrix defined by:

(7)

where th^k_m,o are threshold values.

The principle of this method is to detect any irregularities (usually representing defects) from the NDT images and preserve them in the fused image. This is achieved by analysing the consistency of the DWT coefficients of each decomposed layer having the same material structure to yield a predefined distribution. Any departure from the expected distribution is treated as a representation of a potential defect and the corresponding DWT coefficient should be included in the IDWT to construct the fused image. The detection is based on a thresholding operation performed separately for each image type, each decomposed layer, each decomposition level and each orientation. Apart from the DWT coefficients responsible for the representation of suspected defects, the DWT coefficients which represent the most predominant features in the each decomposed structure region should be also preserved (e.g. radiographic information for the honeycomb structure or ultrasonic information for the thin carbon-fibre area).

The thresholds values th^k_m,o(i,j)for a given image type at a decomposition level and orientation are computed for all DWT coefficients corresponding to the same decomposed layer according to the structure decomposition mask. The threshold value computation benefits from the fact that after the structure decomposition, analysis can be performed for each layer separately by expecting a certain regularity in its texture if no defect is present in this area. This means that the DWT coefficient distribution for each image data type, component layer, decomposition level and orientation can be predicted and used to detect any abnormalities. Figure 6 shows the normalised histogram of the computed DWT coefficients related to the decomposed honeycomb layer of the ultrasonic image at decomposition level three,w¹_3,a(i,j). In the same figure, the red line represents the normal probability distribution which is obtained by scaling along the horizontal axis to fit the part of the histogram consistent with the normal probability distribution, and can be treated as a good approximation of the true probability distribution of the DWT coefficients in the defect free case. From the normal probability distribution, the distribution parameters were estimated with the mean value µ=1299 and dispersion s =56. The threshold value is computed as th¹_3,a(i,j)=5 s + m = 1579 and shown as the black vertical line in Figure 6. Only one threshold value is shown in the histogram as the approximation coefficients for all levels are always positive. Another example of this procedure is shown in Figure 7 for the shearographic image corresponding to the honeycomb area. The histogram shown is for the computed detail coefficients at level four with vertical orientation,w³_4,v(i,j). Following the procedure described before with additional constraint that µ=0, the threshold is computed as th ³_4,h(i,j) = 5s = 5 ´52 = 260.

Fig 6: Histogram computed for ultrasonic DWT coefficients, w13,a(i,j)

Fig 7: Histogram computed for shearographic DWT coefficients w34,v(i,j)

To construct the fused image, the IDWT is performed on the fused wavelet coefficients by transposing the DWT structure with a 1-to-2 up-sampling operation preceding the two-channel filter bank.

5. Fusion results

The results of fusing two NDT images, namely, the ultrasonic and radiographic images of the honeycomb area shown in Figures 5(a) and 5(c), using the three fusion methods, are shown in Figure 8. The region-based fusion based on multi-resolution detection of the features in the honeycomb area is seen to produce the best image for defect interpretation, as Figure 8(c) shows simultaneously a sharp high contrast image of the internal honeycomb structure from the radiographic image and the defect extracted from the ultrasonic image. The quality of the other two fused images is acceptable with the worst one produced by the averaging fusion method, because Figure 8(a) is seen to have the lowest contrast and the defect is the least prominent.

Fig 8: Fusion of two NDT images: (a) fusion by averaging, (b) fusion by local energy comparison, (c) region-based fusion by feature detection

The results of fusing three NDT images, namely, the ultrasonic, radiographic and shearographic images of the honeycomb area shown in Figures 5(a), 5(c) and 5(e), using the three fusion methods, are shown in Figure 9. Similar to the results obtained from the fusion of two NDT data images, the region-based fusion is seen to produce the best image for defect interpretation, as Figure 9(c) not only shows clearly the internal structure of the honeycomb but also preserve fully the defect information from the ultrasonic and shearographic images. Although the defect information from both ultrasonic and shearographic images is also preserved in the fused image produced by the fusion method based on local energy comparison as shown in Figure 9(b), it is not as clear as that in Figure 9(c), and the internal honeycomb structure is not well represented with some artificial black patches which may lead to misinterpretation during defect evaluation. Figure 9(a) shows the poorest fused image produced by the averaging fusion method, where the internal honeycomb structure is seen to be poorly represented and the defect information from the ultrasonic image is not visible.

Fig 9: Fusion of three NDT images: (a) fusion by averaging, (b) fusion by local energy comparison, (c) region-based fusion, (d) region-based fusion using RGB colour space

Although the defect information from both ultrasonic and shearographic images was well preserved in the fused image produced by the region-based fusion, it is not easy to distinguish between them as both of them are represented using grey levels. This is particularly true when the defect patterns produced by the two methods overlap as it is the case shown in Figure 9(c). To improve discrimination between the information produced by different NDT images, the region-based fusion is modified by using three primary colours in the RGB colour space to represent the three information extracted from the three NDT images. Let w^fk_m,o(i,j) represent the wavelet coefficients of the fused image in the RGB colour space with k=1 denoting the ultrasonic image being associated with colour green, k=2 denoting the radiographic image being associated with colour blue and k=3 denoting the shearographic image being associated with colour read, the wavelet coefficients of the fused image are given by the formula:

(8)

Figure 9(d) shows the fused image in RGB colour space with the information from the three different NDT image types easily distinguishable.

6. Conclusions

In this paper, a novel region-based wavelet fusion approach based on multi-resolution feature detection is presented to combine three separate NDT images into one image. The proposed method is shown to preserve the most important features in each data set, thereby enabling defect interpretation and classification to be carried out more rapidly, accurately and consistently.

Acknowledgement

This work is supported by the UK Engineering and Physical Science Research Council (EPSRC) under Innovative Manufacturing Initiative (Grant Ref: GR/L34464).

References

1. J. P. Smith, L.-K. Shark and T. J. Terrell, `Detection of aircraft component defects by low voltage excitation of ultrasonic transducers', Review of Progress in Quantitative Nondestructive Evaluation, Vol.16A, pp935-942, Plenum Press, 1997.
2. B. J. Matuszewski, L.-K. Shark, M. R. Varley and J. P. Smith, Wavelet domain fusion of multiple NDT data sets for industrial inspection, Proceedings of NDT'99 and Corrosion'99, Poole, Dorset, 14-16 September 1999, pp. 19-24.
3. B. J. Matuszewski, L.-K. Shark, J. P. Smith and M. R. Varley, `Automatic fusion of multiple non-destructive testing images and CAD models for industrial inspection', IEE 7^th Int. Conf. on Image Processing and Its Applications, IPA'99, Manchester, pp661-665, 1999.
4. I. Daubechies, `Ten lectures on wavelets', CBMS, SIAM, 61, 1994.
5. W. K. Pratt, `Digital Image Processing', Wiley,1991.

© AIPnD , created by NDT.net

|Home|    |Top|