·Table of Contents ·Aeronautics and Aerospace | Region-based wavelet fusion of ultrasonic, radiographic and shearographyc non-destructive testing imagesBogdan J. Matuszewski, Lik-Kwan Shark, Martin R. Varley,Department of Engineering and Product Design, University of Central Lancashire Preston, PR1 2HE, United Kingdom John P. Smith, BAE SYSTEMS Warton, Preston, PR 1AX, United Kingdom Contact |
Fig 1: Registered ultrasonic image | Fig 2: Registered radiographic image |
Fig 3: Component structure mask derived from CAD data |
Fig 4: Wavelet domain fusion |
As shown in Figure 4, the first stage of wavelet fusion is to perform the two-dimensional (2D) DWT on NDT images. By using a two-channel filter bank consisting of a lowpass filter and a highpass filter with impulse responses derived from a particular wavelet basis function, the 2D DWT is implemented as two one-dimensional(1D)DWTs,The first 1D DWT is performed along the horizontal direction of each NDT image with each row being treated as a 1D signal.By performing lowpass and highpass opearations on each row followed by a 2-to-1 down-sampling operation to discard every other column, the first 1D DWT produces two outputs for each NDT image, namely: the horizontal lowpass output and the horizontal highpass output.The second iD DWT is performed along the vertical directionof the outputs produced by the first1D DWT with each column of the horizontal low pass output and the horizontal high pass output being treated as a 1D signal . By performing lowpass and high pass operations on each column followed by 2-to-1 down sampling operation to discard every other row, the second 1D DWT produces four first level wavelet coefficients for each NDT image, namely: the vertical lowpass of the horizontal lowpass output, w^{k}_{l,a}(where superscript k denotes the type of input images with 1 for ultrasound, 2 for X-ray and 3 for shearographic image), the vertical highpass of the horizontal lowpass output,w^{k}_{l,h}and the vertical lowpass of the horizontal highpass output,w^{k}_{l,v}and the vertical highpass of the horizontal highpass output,w^{k}_{l,d}while w^{k}_{l,a} corresponds to the first approximation of each NDT image,w^{k}_{l,h},w^{k}_{l,d},w^{k}_{l,v} correspond to the horizontal,diagonal and vertical detail respectively.Applying the whole process again to w^{k}_{l,a}yields wavelet coefficients w^{k}_{2,a},w^{k}_{2,h} w^{k}_{2,d}, and w^{k}_{2,v} at the second resolution level as shown in Figure 4,and the entire process can repeated until the desired decomposition level is reached.
Figure 5 shows the ultrasonic,radiographic and shearographic sub-images of the smaller honeycomb area with the defect visible on the bottom left of the ultrasonic and shearographic images and their corresponding DWT with 4 decomposition levels based on the symlet wavelet of order 5.
Fig 5: (a)-(b) Ultasound image and its DWT,(c)-(d)X-ray image and its DWT,(e)-(f) shearographic image and its DWT |
From Figure 5 the ultrasonic defect feature is seen to dominate the coarsest approximation to its low frequency nature, the internal structures in the radiographic image are seen to dominate the horizontal, diagonal detail levels to their high frequency nature ,and the shearographic defect feature is seen to dominate the coarse level detail coefficients due to the relatively low frequency 2D oscillations (visible as an elliptical pattern in Figure 5(e)) After the wavelet decomposition of each NDT image to produce multiple sets of the different wavelet coefficient sets denoted W^{1}to W^{ p}, they are combined together to form a single set of the wavelet coefficients for the fused image. This process can be represented by the formula:
(1) |
where F_{m,o,i,j}(·,...,·,p) is the function representing the fusion rule for the wavelet coefficients w^{f}_{m,o}(i,j) of the fused image (see Figure 4), and vector p represents additional parameters used by the fusion rule. From this general formula, it follows that the fusion rule theoretically can depend on decomposition level denoted by m, orientation (namely a, h, d, or v) denoted by o, and coefficient location denoted by i and j. In the following, three different fusion methods are described and applied to the NDT images shown in Figure (5). For the first two methods, fusion rules do not depend on the location (i,j) of the fused coeffi cient w^{f}_{m,o}(i,j),whereas for the third method fusion rule depends on the coefficient location by utilising the information of the image structure decomposition described in section 3.
(a) Fusion by averaging
In this method coefficient of the fused image are computed according to the formula:
(2) |
where A(k,m,o) are weighting factors selected for image k, at decomposition level m with orientation o. This fusion method simply performs a weighted averaging operation of the wavelet coefficients. The weights A(k,m,o) are selected before fusion based on typical image contents. For example, if it is usually true that the DWT coefficients at level m and orientation o computed for image k are significant for the representation of some important image features, then A(k,m,o) will have a relatively high value; if they represent mostly noise or irrelevant information from the defect detection point of view, then A(k,m,o) will have a small value. This fusion method does not take into account the spatial variation of the image contents or the actual coefficient values of the images. For the fusion results shown in the next section, A(k,m,o)=1/P for all k,m and o except that A(1,1,o)=A(3,1,o)=0 and A(2,1,o)=1 for all orientations.
(b) Fusion by energy comparison
In this method, the wavelet coefficients of the fused image are computed according to the formula:
(3) |
where e^{k}_{m,o}(i,j) is the local energy computed for the DWT coefficients over a window of size (d+1)X(d+1) with the window centre located at (i,j) at level m and orientation o:
(4) |
Using this method the wavelet coefficient from image k is included in the IDWT to construct the fused image if its local energy is greater by a specified threshold value, T, than the local energy computed for the corresponding wavelet coefficients of other images. If the difference between the local energy levels computed for any image pairs is less than or equal to T, then the average of the corresponding wavelet coefficients is used as the fused wavelet coefficient. This method does not utilise the structure decomposition information, but the fusion rule depends on the actual coefficient values of the images. For the fusion results shown in the next section, a window of size 5X5 (d=2) was used for the local energy computation and T=1 was selected as the threshold value.
(c) Region-based fusion by multi-resolution feature detection
In this method the wavelet coefficient of the fused image are computed according to the formula:
(5) |
(6) |
and q^{k}_{m,o}(·,·) is the mask matrix obtained as the result of binary morphological `clean' and `close' operations [5] applied to binary matrix defined by:
(7) |
where th^{k}_{m,o} are threshold values.
The principle of this method is to detect any irregularities (usually representing defects) from the NDT images and preserve them in the fused image. This is achieved by analysing the consistency of the DWT coefficients of each decomposed layer having the same material structure to yield a predefined distribution. Any departure from the expected distribution is treated as a representation of a potential defect and the corresponding DWT coefficient should be included in the IDWT to construct the fused image. The detection is based on a thresholding operation performed separately for each image type, each decomposed layer, each decomposition level and each orientation. Apart from the DWT coefficients responsible for the representation of suspected defects, the DWT coefficients which represent the most predominant features in the each decomposed structure region should be also preserved (e.g. radiographic information for the honeycomb structure or ultrasonic information for the thin carbon-fibre area).
The thresholds values th^{k}_{m,o}(i,j)for a given image type at a decomposition level and orientation are computed for all DWT coefficients corresponding to the same decomposed layer according to the structure decomposition mask. The threshold value computation benefits from the fact that after the structure decomposition, analysis can be performed for each layer separately by expecting a certain regularity in its texture if no defect is present in this area. This means that the DWT coefficient distribution for each image data type, component layer, decomposition level and orientation can be predicted and used to detect any abnormalities. Figure 6 shows the normalised histogram of the computed DWT coefficients related to the decomposed honeycomb layer of the ultrasonic image at decomposition level three,w^{1}_{3,a}(i,j). In the same figure, the red line represents the normal probability distribution which is obtained by scaling along the horizontal axis to fit the part of the histogram consistent with the normal probability distribution, and can be treated as a good approximation of the true probability distribution of the DWT coefficients in the defect free case. From the normal probability distribution, the distribution parameters were estimated with the mean value µ=1299 and dispersion s =56. The threshold value is computed as th^{1}_{3,a}(i,j)=5 s + m = 1579 and shown as the black vertical line in Figure 6. Only one threshold value is shown in the histogram as the approximation coefficients for all levels are always positive. Another example of this procedure is shown in Figure 7 for the shearographic image corresponding to the honeycomb area. The histogram shown is for the computed detail coefficients at level four with vertical orientation,w^{3}_{4,v}(i,j). Following the procedure described before with additional constraint that µ=0, the threshold is computed as th ^{3}_{4,h}(i,j) = 5s = 5 ´52 = 260.
Fig 6: Histogram computed for ultrasonic DWT coefficients, w^{1}_{3,a}(i,j) | Fig 7: Histogram computed for shearographic DWT coefficients w^{3}_{4,v}(i,j) |
To construct the fused image, the IDWT is performed on the fused wavelet coefficients by transposing the DWT structure with a 1-to-2 up-sampling operation preceding the two-channel filter bank.
Fig 8: Fusion of two NDT images: (a) fusion by averaging, (b) fusion by local energy comparison, (c) region-based fusion by feature detection |
The results of fusing three NDT images, namely, the ultrasonic, radiographic and shearographic images of the honeycomb area shown in Figures 5(a), 5(c) and 5(e), using the three fusion methods, are shown in Figure 9. Similar to the results obtained from the fusion of two NDT data images, the region-based fusion is seen to produce the best image for defect interpretation, as Figure 9(c) not only shows clearly the internal structure of the honeycomb but also preserve fully the defect information from the ultrasonic and shearographic images. Although the defect information from both ultrasonic and shearographic images is also preserved in the fused image produced by the fusion method based on local energy comparison as shown in Figure 9(b), it is not as clear as that in Figure 9(c), and the internal honeycomb structure is not well represented with some artificial black patches which may lead to misinterpretation during defect evaluation. Figure 9(a) shows the poorest fused image produced by the averaging fusion method, where the internal honeycomb structure is seen to be poorly represented and the defect information from the ultrasonic image is not visible.
Fig 9: Fusion of three NDT images: (a) fusion by averaging, (b) fusion by local energy comparison, (c) region-based fusion, (d) region-based fusion using RGB colour space |
Although the defect information from both ultrasonic and shearographic images was well preserved in the fused image produced by the region-based fusion, it is not easy to distinguish between them as both of them are represented using grey levels. This is particularly true when the defect patterns produced by the two methods overlap as it is the case shown in Figure 9(c). To improve discrimination between the information produced by different NDT images, the region-based fusion is modified by using three primary colours in the RGB colour space to represent the three information extracted from the three NDT images. Let w^{fk}_{m,o}(i,j) represent the wavelet coefficients of the fused image in the RGB colour space with k=1 denoting the ultrasonic image being associated with colour green, k=2 denoting the radiographic image being associated with colour blue and k=3 denoting the shearographic image being associated with colour read, the wavelet coefficients of the fused image are given by the formula:
(8) |
© AIPnD , created by NDT.net | |Home| |Top| |