Matching of CAD model projections and X-ray projection images for shape inspection of metal assemblies

X-ray computed tomography (CT) facilitates non-destructive inspection of the interior of mechanical assemblies. However, industrial materials often contains heavy metals; which causes in CT artifacts in CT images and prevents accurate segmentation or shape measurement. To overcome this issue, in this study, we proposed a new inspection method that uses X-ray projection images and CAD meshes. We extracted characteristic features as edges and curved surfaces as feature images and used those for component alignments and shape measurements. The idea behind this is that these characteristic 3D features are captured as crease pixels in 2D projection images; thus, they can be extracted and compared with CAD meshes directly without conducting actual CT reconstructions. Furthermore, we conducted preliminary tests with some metal parts, which indicated that our approach can effectively inspect multi-material mechanical assemblies.


Introduction
X-ray computed tomography (CT) is widely used for inspecting industrial parts because it allows non-destructive inspection of various components [1].By capturing and reconstructing hundreds to thousands of X-ray images, X-ray CT can acquire 3D CT images of the object that is to be measured.One of the most promising applications of CT is measurement of assemblies of industrial products because of its ability to inspect assemblies without disassembling them.However, many industrial products are composed of metal parts, especially heavy metals, which strongly attenuate X-rays and cause artifacts in CT images [2].CT artifacts can result in CT images with incorrect contours or blurred boundaries, thereby making it difficult to obtain an accurate surface shape of the objects [3].To adress this problem, we propose a novel shape inspection method that uses 2D X-ray projection images and computer-aided design (CAD) to obtain the initial shape.The CT values on X-ray projection images are different from the theoretical values because of the strong attenuation of X-rays, which is the main cause of CT artifacts.This problem can be avoided via direct measurement by using X-ray projection images [4].The contours and boundaries of the measured object in X-ray projection images are considered to be accurate.Therefore, these features can be extracted from X-ray images and used for registration and shape measurements.This approach can also reduce the number of images used for positioning and inspection, which would shorten the computation time.11th Conference on Industrial Computed Tomography, Wels, Austria (iCT 2022), www.ict-conference.com/2022

Method
The proposed method uses X-ray projection images for the assembly of several components along with their 3D CAD models (triangular meshes) as the input.We want to scan objects that contain heavy metals and could have CT artifacts in their reconstructed CT images.Accordingly, to minimize the effects of CT artifacts, we focused on measuring characteristic features, such as edges and curved surfaces instead of measuring all surface shapes of the objects.These charactaristic features were extracted from both X-ray projection images and CAD meshes, to perform registration and shape measurement.Our shape inspection method can be explained in the following four steps: (1) Compute feature images from the original Xray projection images, (2) reconstruct feature volume from feature images and conduct assembly registration, (3) Compute the contours from CAD mesh, and (4) measure the shape with mesh contours and feature images.The dimensions of the object can be measured from the projections of the CAD mesh and feature images extracted from the actual X-ray images.

X-ray feature image extraction
First, we computed the feature images representing the edges and curved surfaces from the X-ray projection images [5].These features comprised of pixels wherein the gradient changed sharply.We used normal tensor voting to determine these pixels.The advantage of using normal tensor voting is that it is robust against noise; thus, it is suitable for detecting sharp edges.Further, we computed the normal of the X-ray projection images beforehand, and 3 × 3 matrix V of normal tensor voting for each pixel.
where i represents a pixel in a kernel, n n n i is a normal vector, k is the kernel size, and P u i ,v i is the projection value of the i-th pixel.Images of the normals were created beforehand from the projection images by computing the gradient of the projection values.To determine which pixel is a feature points, three eigenvalues λ 1 , λ 2 , and λ 3 ( λ 1 ≤ λ 2 ≤ λ 3 ) of V were computed.These eigenvalues represents the magnitude of the principal component of normal vectors in the kernel; thus, λ 2 /λ 3 could be used as a feature value.This value varies from 0 to 1, which suggests gradient changes.If the medium and max eigenvalues, λ 2 and λ 3 are almost equal, λ 2 /λ 3 is close to 1, and the pixel could be detected as a feature pixel.Binary feature images were also computed by applying non-maximum suppression to the images.For better shape inspection, we also need sub-pixel accurate shape of the features; therefore, we compared the feature values of the binary feature pixels with the neighboring pixels, thereby computing sub-pixel accurate X-ray feature points projections.11th Conference on Industrial Computed Tomography, Wels, Austria (iCT 2022), www.ict-conference.com/2022

CT reconstruction and registration
In this subsection, we reconstructed a CT volume using feature images with the Feldkamp method using the Shepp-Logan filter [6].In this step, only a small number of images are needed.We used around 100 to 200 feature images to reconstruct the CT volumes.This CT volume only contained 3D feature shapes, such as edges and curbed surfaced, because feature images were used for reconstruction.For assembly parts registration, the CT volume was converted to 3D point clouds and was used for aligning the CAD meshes to the measurement coordinate system.Each component CAD mesh of the assembly was sequentially aligned to the 3D feature model using the iterative closest points algorithm [7].Alignment was performed using the outer components, and the point clouds used for alignment were automatically removed.The remaining point clouds were used for the next CAD component alignment.The point clouds include only the feature shapes of the assembly; thus, it is also easy to segment and register each individual component of the CAD meshes.

Mesh feature vertices extraction
We computed the feature vertices of the CAD mesh to compare the CAD mesh and actual shape on the 2D images.In our work, we used contours of the CAD mesh for shape inspection.CAD mesh feature vertices, which comprised of contours were extracted and used as measurement vertices.These vertices were also projected on to the 2D images and compared with the X-ray feature images.The mesh vertices that comprises of contours are defined as a set of points satisfying the following equation: where p p p is a point on the surface, n n n(p p p) is the unit normal vector at p p p, and c c c is a perspective viewpoint.In other words, the contours comprise the set of points on the surface whose normal vectors are perpendicular to the viewing direction.To detect these vertices for two adjacent faces on the mesh, we calculated the angle between the normal vector of the face and the viewing direction, θ 1 , θ 2 .If θ 1 < π/2 < θ 2 , the two vertices that are contained in the two faces are the contours.These mesh contours and feature vertices were computed for every direction of the X-ray feature images.When all projections are combined, these vertices will include all the characteristic features of the CAD mesh, such as edges and curved surfaces.These feature vertices were assigned to measurement points, and were used for shape inspection.

2D-3D shape inspection
In this step, we compared the X-ray feature points projections and mesh feature vertices, and conducted 2D-3D shape inspection for the each mesh feature vertex.Further, we computed the errors between X-ray feature points projections and mesh feature 11th Conference on Industrial Computed Tomography, Wels, Austria (iCT 2022), www.ict-conference.com/2022vertices for every projection direction, which were combined as errors in a 3D space.First, for the projection in only one direction, the X-ray feature points projection and the feature mesh vertices projection were compared.Each projected mesh feature vertex d d d m was paired with the closest point d d d x in the X-ray feature points projection, and the difference between the paired points ε det = |d d d x − d d d m | was computed.Then, these errors ε det were backprojected onto the corresponding mesh vertices.The back-projection error of each silhouette point can be computed as ε = ε det L/D , where L is the distance between the X-ray and the object, D is the distance between the X-ray and the detector.To conduct a more accurate 2D-3D inspection, the errors calculated from multiple directions of the projection images were combined.θ i (i = 0, 1, ...k) is the angle by which the X-ray source (or the part) has been rotated between each scan of a projection image, d d d x (θ i ) is the vector pointing from the source to the projected points on the detector.The 3D locations of the X-ray feature points projections p p p x (θ all ) is the point at which all 2D vectors d d d x (θ i ) cross.The resultant vector p p p x (θ i ) will have some errors.Thus, the point at which all 2D vectors cross is determined by the least-squares method.Any number of projected images could beused; however, reducing the number of projections reduces the computation, while increasing the number of projection images increases the inspection accuracy.

Experiments
The proposed algorithm was applied to inspect multiple metal assemblies.X-ray projection images of 1024 × 1024 pixels with a pixel size of 0.4 mm were used.The scanned data was obtained using the Carl Zeiss METROTOM 1500 G1 cone-beam X-ray CT scanner.The experiment conditions are listed in Table 1. Figure 8 shoes the scanned objects, i.e., their X-ray projections, and reconstructed mesh models.All three scans show intense CT artifacts; thus, accurate shape inspection could not be conducted via conventional surface mesh reconstruction.Figure 9 shows the registration and feature shape inspection results of the simple assembly components.A 3D feature model was successfully generated from the feature images, and the CAD mesh of each component was aligned to the model correctly.
Figure 9b shows the deviation from the original mesh vertices.The edge vertices of the metal plate were transformed based on the X-ray feature images.
For the aluminum plate, detection of edges from the mesh of the object was successful.The object feature lines were clearly detected in the feature images, and the 3D shape of the real object was recovered from the CAD mesh using the 2D projection images.However for the resin and titanium rods, there was a lot of noise in the shape inspection result.This was because all surface features were detected as feature lines, and could not be reliably corresponded between the mesh projections and the feature projection images.Figure 10 shows the feature shape inspection results of the metal hinge assembly.A 3D feature model was successfully generated from the feature images, and the CAD mesh of each component was aligned to the model correctly.The shape of the CAD mesh of the object is slightly different from the actual object.In particular, the shape of the corners of the hinge plate is square in the CAD, whereas the actual one is rounded.The left side of Figure 10 shows X-ray projection image and the corresponding projection image of the CAD mesh.The difference in shape can be clealy observed.The right side of Figure 10 shows the errors from the CAD vertices to the actual shape.The proposed method corrected the edge shape by using only projection images.Figure 11 shows the shaoe inspection results of the metal L-shaped brackets.A 3D feature model was generated from the feature images, and the CAD mesh of each component was aligned to the model.The upper side of Figure 11 shows the X-ray projection image and corresponding projection image of the CAD mesh.There is a slight deviation in the alignment.The lower side of Figure 11 shows the errors from the CAD vertices to the actual shape.The lower side of the edge of the metal brackets was corrected and the edge vertices were transformed by the proposed method.However, the proposed method did not correct the upper side of the edge of the metal brackets and the corrected shape became slightly thicker than the actual shape.A possible reason is that in the projected image of the thick metal object, the feature lines tend to be detected a bit thicker than the actual shape.This is because the projection value changes sharply for high density objects; the maximum change in the gradient is detected slightly outside the object's contour.This caused a misalignment in the registration, and the correction performed using the feature image was not sufficient enough to obtain the correct shape.

Conclusion and Future work
In this study, we developed a novel inspection method for objects that have severe CT artifacts.We mitigated the effects of CT artifacts by using feature images and feature models; and evaluated characteristic shapes, such as edges and curved surfaces.Further investigations are required to apply this inspection algorithm for the measurement of other dimensions and shapes, such as measuring the thickness of plane-shaped objects.

Figure 1 :
Figure1: Flowchart of the proposed method.The feature images are created for each X-ray projection image, which are used for creating the 3D feature model and registrations.Further, shape inspection is performed by comparing the feature images and CAD mesh projections.

Figure 3 :
Figure 3: Extract feature points in sub-pixel accuracy.

Figure 6 :
Figure 6: Back-projection of the error.

Figure 7 :
Figure 7: 3D crossing vectors from top view

Figure 8 :
Figure 8: Scanned metal objects used in experiments

11th
Figure 9: Feature shape inspection results of the metal plates and sticks.CAD meshes and feature images are compared, and the deviations of the measurement vertices are computed.

Figure 10 :
Figure 10: Feature shape inspection results of the metal hinges.CAD meshes of the hinges are aligned to the feature model.CAD meshes and feature images are compared, and the deviations of the measurement vertices are computed.

Figure 11 :
Figure 11: Feature shape inspection results of the L-shaped brackets.CAD meshes are aligned to the feature model.CAD meshes and feature images are compared, and the deviations of the measurement vertices are computed.

Table 1 :
Experiment conditions