CAD-based defect inspection with optimal view angle selection based on polychromatic X-ray projection images

X-ray 3D computed tomography (CT) is a non-destructive method that allows inspection of internal components of an object. In the conventional approach, the comparison between the measured projections and the 3D model is performed by processing a CT reconstruction from projection data. The accuracy of the inspection analysis strictly depends on the reconstruction, which can suffer from numerous artifacts. To overcome this problem, we propose a new method that performs inspection directly in the projection space, simulating realistic projections from the computer aided design (CAD) model. Our method bases its strength on the prior knowledge from the CAD data and knowledge about the inspected object itself. When inspecting for defective or missing components, it is not restrictive to assume that the position of a potential deviation from the nominal geometry is approximately known. Therefore, we define regions of interest (ROIs) for feature extraction by simply projecting a component or a volume around it. Furthermore, based on the nominal geometry, we can identify the projection angles for which a certain component is most visible, to restrict the inspection analysis to projections obtained at these angles. Our procedure for quality control is composed of two main steps: the first one must be performed prior to in-line inspection and consists of i) an accurate alignment to calibrate the system geometry and ii) building of libraries of simulated projection data. These libraries are used in the second, in-line step to perform a fast 3D alignment with respect to the position and orientation of the sample. The knowledge of the sample’s orientation allows one to select only a few projection angles for which a potential defect is most visible and perform classification by extracting features and measures from merely the projections at this limited set of angles, for a predefined ROI. In this paper, we motivate and describe our procedure for limited view in-line inspection of defects and validate our method with experiments on real data.


Introduction
In the last few years, the make industry is getting increasingly interested in 3D CT.This non-destructive method permits to reconstruct a detailed profile of the volumetric internal structure of a complex object starting from a collection of 2D X-ray projection images.Its main, but not unique, application in the manufacturing process is inspection, to detect possible material defects or dimensional deviations.For this purpose, the acquired projection images can be compared to the nominal geometry using CAD models.Conventional CAD-based X-ray inspection requires 3D reconstruction and voxelization of an object, starting from its radiographs, before comparing it to the nominal geometry [1,2].With this methodology, the quality of the final reconstruction is crucial to guarantee an accurate analysis (e.g metrology).For high quality reconstructions, typically hundreds of X-ray projection images from a large number of viewing angles are required.Even so, the 3D reconstructed images may suffer from numerous artefacts due to misalignment in the measurement setup, beam hardening, undersampling, etc.Part of these problems can be overcome by doing the analysis in the native projection space instead of the reconstruction domain.With this aim, we recently developed a CAD projector [3] capable of simulating projection images from the CAD model.The CAD projector is efficiently implemented on the GPU and integrated with our flexible, open-source reconstruction software, the ASTRA Toolbox [4].With respect to other similar state-of-the-art tools, as [5,6], our CAD projector is meant as a forward projector, rather than a toolbox to create simulated data.In this paper, we propose a CAD based inspection directly performed in the projection space, comparing the 2D simulated projections directly to the measured ones.Our method avoids the need of reconstructing the volume and correcting the artifacts, and performs well using a very limited amount of projections over a limited angle range.This meets the needs of in-line inspection where usually the time for checking a sample is narrow and the structure of the object itself can impede 360 • rotation.Also, thanks to a GPU implementation and parallelization, the complete procedure requires an execution time in the order of a second, making it applicable directly in-line.The novelty of our method lies in the smart usage of the CAD model for extracting precious information: since the position and the extent of the possible defects that may occur during assembly are assumed to be known a-priori, we can easily define the ROI where to perform inspection based on the CAD model.Also, after a fast 2D-3D registration, we define a method to select only the angles where the component to be checked is most visible (we refer to these angles as visibility angles), based on image contrast.To do this, we pre-compute projection data for the expected object and we associate to each discrete orientation view of the CAD model around its rotation axis a contrast value, so that the knowledge of the orientation for each projection angle guides the choice of the visibility angles.At the end of our workflow, we classify the samples by extracting some useful task specific features from only the projections at the visibility angles.Results on real data are shown to demonstrate the effectiveness of such an approach.The paper is structured as follows: section 2 provides a general view of the methodology, presenting first our procedure to simulate realistic radiographs from the CAD model and to align them to the ground truth ones.We then explain the details of our proposed procedure for in-line inspection, which includes some prior non in-line calculation.In section 3 we show the results of the angle selection procedure and of the classification on real data, both for missing and for distorted components.Section 4 is dedicated to the time evaluation of the entire procedure and the improvements on implementation necessary for in-line application.Finally, in section 5, we draw our conclusions, focusing on future improvements.

Methodology
As previously mentioned, the CAD projector is a tool to simulate radiographs of the reference CAD object.For this purpose, polychromatic X-rays are virtually cast from a source, penetrate the CAD model and hit a virtual detector.After detecting the points of intersection of the X-ray beam with the triangle mesh of the CAD model, the line length that a ray travels inside the model is calculated, and, knowing the material of each component of the model, polychromatic projections are calculated using the polychromatic form of the Beer-Lambert law.In order to compare the simulated projection images to the measured data, it is necessary that the acquisition parameters, such as the position of the source, the detector and the rotation axis with respect to the reference system centered in the center of mass of the sample are known.Also, to adequately simulate the behavior of the beam when intersecting the sample, the materials of all the components must be known in advance.Two optimization processes are needed: (1) estimation of the spectrum of the source and (2) geometric calibration.The X-ray source spectrum estimation was developed by Marinovski et al. [3] and is performed by minimizing the discrepancy between the intensity values of the measured and simulated data.In [3], the authors showed the consistency of the method through experimental results on simulated and real data.In the present work, we extend this approach to allow geometric calibration using CAD data of a calibration phantom in terms of position and orientation of the rotation axis, the detector, the phantom and the position of the source (see Figure 1).These parameters are optimized using the Hill climbing algorithm [7].Although this alignment procedure returns an accurate result, it is not applicable to in-line quality control due to its long execution time.Therefore, our methodology for in-line inspection has been split in two main steps.The first step consists of a set of procedures that have to be performed prior to in-line inspection.The second step is the actual in-line inspection, which is performed in real time.A flowchart of the proposed approach, that will be explained in depth in section 2.1 and 2.2, is shown in Figure 2.

Prior to in-line inspection
Our proposed method starts by estimating the spectrum of the source and geometry parameters of the system once, prior to the in-line analysis.Based on these estimates of the spectrum and geometry parameters, we built two libraries, with simulated images from different projection angles.The span angle range in the library can be the total projection angle used to acquire the real images, or only a fraction of it if the component to inspect has an internal symmetry.The step angle between two consecutive images in the library depends on the accuracy required by the application.The last step in the prior stage is to compute a contrast measure associated to each projection image in the library.In-line, features will only be extracted on a selection of high contrast projections.
Spectrum estimation Estimating the source spectrum is necessary to realistically simulate real polychromatic X-ray projections.
It guarantees that the range of the intensity values of the simulated images corresponds to that of the measured ones.It is performed iteratively together with the geometry estimation.We estimate the percentage of the number of photons in each energy bin by minimizing the discrepancy between simulated and real data.
System geometry calibration In this step, we only estimate the geometrical parameters related to the relative position of the detector, the source and the rotation axis, that is the source-object-distance (SOD) and the source-detector distance (SDD).
There parameters can be pre-calibrated and can be assumed to be the same for all in-line scanned objects.
Building the following libraries: (a) ROI for alignment The first library contains a ROI of the simulated projections, that will be used during inspection for a fast alignment on the detector plane.It can be manually defined and should contain a portion of the object that is big enough to guarantee a good performance of the alignment algorithm.
(b) ROI for inspection This library contains a ROI where to perform the quality control.If the goal is to inspect for a missing component, the ROI can easily be defined from CAD data by projecting only this component along with components whose contribution needs to be avoided.Otherwise, supposing that the position of the defective part is known, simple CAD volumes can be defined around the identified position and projected.In the latter case the ROI can be optimized in order to discriminate optimally between defective and non-defective samples.
Contrast evaluation For each projection angle, the contrast is calculated in the first library (a) on the smallest rectangle containing the component, masking out the contribution of other components that are too close or can overlap with it, to exclude their contribution.In literature, many measures of contrast are defined.In this work, we use the root-mean-squared contrast (RMSC), defined as: where I is the projection image to be evaluate, with N × M pixels and I is its mean intensity.The values of the contrast can be precalculated and stored in a vector: in the inspection phase, the visibility angles will be chosen as those with the highest contrast values.

During in-line inspection
In-line inspection requires a fast alignment procedure between the simulated projections from the CAD model and the real projections.We consider a ROI of the real radiographs corresponding to the one in the first library (a).First, one of the simulated radiographs is rigidly registered to each measured projection.At this step, to avoid to process all the images of the library, the estimated rigid transformation is applied directly to the real projections.Once the images are transformed, we need to find the orientation of the samples: to do so, we perform a bruteforce search on the library, using the mean square error (i.e. the sum of squared differences between the real and simulated projections) as a criterion.Each projection angle corresponds to a contrast value, calculated using Eq. 1.Among these, only the 15% most contrasted views are considered on which to perform the final feature extraction and classification, using the library (b) containing the ROI for inspection.

Experiments
We validated our methodology on in-line scans of syringes, composed of metallic and plastic parts.The goal of the study was to distinguish between intact and defective products.For each scan, X-ray images were acquired from 21 equiangular projections within an angular range of 200 degrees.Eight defects were induced under a controlled situation.Four of these eight defects originated from the metal components (e.g., spring, needle) of the syringe.These four defects could easily be inspected by image processing, without the need to apply the proposed approach.Indeed, by exploiting the fact that intensity values of metallic parts are clearly different from those of polymers, the edges of the metallic components could easily be identified, after which simple calculations were performed for classification.For example, by determining the coordinates of the pixels that belong to the needle, it could easily be determined if the needle was not correctly positioned inside the device.Two of the four defects that were inspected with the proposed approach were located on a completely symmetric object: it is obvious then that in this case there are no angles more significant than others.For these cases, we reduced the features' dimension using the visibility angles related to the other component that required inspection, that instead had preferable view orientations.
The entire object was symmetric with respect to a 90 • rotation and we focused on only one component to determine its orientation and the visibility angles.Because of the symmetry, the library contained images oriented from 0 to 89.9 degrees, with an angular step of 0.1 • .In Figure 3, the variation of the RMSC, calculated on the simulated images of the library, is shown in function of the projection angle.It is evident that the component under inspection is mostly visible from the angles in the neighborhood of 0 • and 90 • .Three of the defects required to detect if a component was missing or shifted: in this case the library with the ROI for inspection was defined by projecting the component.For the other one, a possibly distorted component, we projected a small box manually instead, defined around a strategic part of the component were the defect should be visible.
The result of the rigid alignment between real and simulated data is shown in Figure 4, and the component that determines the angle view selection is highlighted.This is one of the components that can be missed or distorted.Each of the 21 measured projections was aligned and then the best orientation angle was found performing a bruteforce search in the library.Among these projection images, the three projections with the highest contrast were selected.
Regarding the (b) library, for three of the defects the ROI contains the projection of the component to inspect, for one it consists of the projection of a a small box around a strategic part of the component were the defect should be visible.
For our experiment we had 52 labeled samples to use as training data, 20 of them were defective.A final blind test was performed on a test set composed of 600 samples, 120 of them were defective.Based on training data, we calculated correlation and signalto-noise ratio between the simulated and the real projection in case of a missing component.In case of a distorted component, a binary gradient image from the original radiographs was also extracted, and a quantitative measure was calculated summing all nonzero pixels.For this case also the root mean squared error was useful for the final classification.For each defect, thresholds on the measures were manually established and the samples were labeled as defective if the statistics simultaneously satisfied a set of logical clauses.When applying the same classification procedure to the test set, the result was a 100% success rate.This means that we obtained no false positives (objects that are defective but labeled as not defective) nor false negatives (samples wrongly classified as defective), demonstrating the effectiveness of our strategy.

Analysis of time execution
Industries are getting increasingly interested in automated 3D X-ray inspection, although in most cases the conventional reconstruction approach is too slow to be applied in an in-line inspection process.
Our methodology was designed under the constraint that the total inspection time per sample should be approximately 1 second.
To achieve this goal, the alignment process and statistics calculation were implemented on the GPU, using the OpenCV library for image registration.Image acquisition was performed placing four samples on a rotatory stage: in a total acquisition time of 4 seconds, so still corresponding to 1 second of scan time per sample.Moreover, those four object and each of the 21 projections were inspected simultaneously using multiple GPUs.Registering images and calculating statistics on the GPU required respectively 1.335 and 0.255 seconds, that, with the previous considerations, makes it feasible to inspect four samples in 1.59 seconds.Our tests ran on a GTX 680 GPU.

Conclusions
Opposed to conventional inspection based on 3D CT images, our methodology performs the 2D/3D alignment and inspection directly in the projection space, without the need of reconstructing the volume from the set of measured projections.As demonstrated, it is applicable to cases of both a limited number of projections and a limited angular view.Our methodology meets the need for fast in-line industrial inspection of each individual product, ensuring that those introduced in the market meet certain quality standards.The strength of our method lies in the smart usage of the reference model to create libraries for ROI-based inspection at carefully selected viewing angles.Choosing a targetted ROI for inspection is a fundamental step to guarantee the success of classification.If the location of a possible defect is approximately known, we define the libraries containing the inspection ROIs by projecting the component to inspect or small volumes around it.Often in literature defining a good region of interest requires a lot of effort, applying for example many filters for edge extraction and segmentation.Our approach allows to simply identify the ROI based on the CAD model's structure.Moreover, the ROI position is determined after each projection image is aligned with those in the libraries, resulting on a different region depending on the view angle.With the prior knowledge based on the CAD model, we defined the visibility angles, i.e. those angles with highest contrast.In our workflow, features are extracted only on those projections, where the defect is supposed to be most visible, resulting in a final better recognition of defective samples.Moreover, restricting the number of projections where to extract features evidently gives an improvement on the final computational time.
We believe that the view angle selection can make a big step forward in industrial inspection, which is why as future work we intend to make this optimization process more generic and widely applicable.

Figure 1 :
Figure 1: Starting from a default configuration with the object rotating around the z axis, we can estimate translation and rotation of both the object and the rotation axis.

Figure 3 :
Figure 3: RMS contrast in function of the projection angle.

Figure 4 :
Figure 4: The figure shows the comparison between the real projection (left) and the selected simulated projection after alignment (middle).The difference image is on the right.