Thickness-Driven Sheet Metal Segmentation of CT-Scanned Body-in-White

Large-scale and high-energy X-ray computed tomography (CT) scanners, such as XXL-CT, can fully digitalize car-sized industrial assemblies. For utilizing the data collected using these scanners in digital engineering applications, a part segmentation method is needed. Hence, we propose a segmentation method for a body-in-white primarily composed of sheet metal. The proposed sheet metal segmentation is driven by the sheet thickness, which is estimated from the CT value profile through the sheet structure. The thickness information is assigned to a triangle mesh representing the medial surface of the sheet metal. Then, a simple region-growing algorithm is used to cluster the mesh triangles with comparable sheet thicknesses. Lastly, the resulting clusters are merged manually to obtain the final part segmentation


Introduction
X-ray computed tomography (CT) scanning can completely digitize a product, including its internal structure, so it has become a common technique for analyzing and inspecting industrial assembly.Figure 1 displays a CT volume obtained by scanning the body-in-white (BiW) of a real car.We used XXL-CT [1], a pioneer system for scanning large-scale objects, to scan such a massive construction.To utilize CT-scanned assemblies in digital engineering applications, part disassembly of digital data, called part segmentation [2], is necessary for part-by-part data processing; rebuilding geometric models including part information (CAD), dimensional metrology of each part (CAT), and simulation model generation combining the parts (CAE).A large assembly comprises many parts and often has a CT volume with enormous data size (for car-sized objects, several hundreds of gigabytes to a few terabytes).Therefore, it is challenging to segment the CT volume using conventional manual or semiautomatic segmentation systems.Researchers have thoroughly investigated this problem, suggesting solutions such as automation using machine learning and scalability for enormous quantities of data [3,4].
In this research, we proposed a CT volume segmentation method for a BiW primarily composed of sheet metal.Sheet metal segmentation is a procedure that separates sheets along welding areas.Therefore, it is crucial to extract welding areas where multiple sheets overlap.However, as seen in the center of Figure 1, it is challenging to directly identify and separate multiple sheets of sheet metal in the welding areas because of the resolution limitations of large-scale CT scans, which typically only correspond to a sheet metal thickness of about 1 mm in two to three voxel layers.To address this issue, the overlapping sheets were considered as a single sheet with high thickness, and the segmentation was performed using the thickness information.A key point of our proposal is shown in the right three graphs of Figure 1, which plot CT values through the sheet structure with three different thicknesses.We can see that the width of the CT value profile does not entirely reflect the sheet thickness.Instead, the sheet thickness, which is roughly equivalent to the actual thickness, is well represented by the peak values.Width and peak values were used as feature maps for sheet metal segmentation of the CT-scanned BiW.

Method of Sheet Metal Segmentation
An overview of the proposed segmentation method is provided in Figure 2. First, the medial surface of the BiW sheet metal was extracted as triangle mesh.Then, the feature maps used in the mesh clustering were assigned to the mesh.Finally, part segmentation was performed after the automatic mesh triangle clustering.As mentioned in Section 1, the most important contribution of this study is the use of two feature maps obtained from CT volumes for segmenting a sheet metal structure.Thus, we start by analyzing a CT value profile estimated through a sheet structure in Section 2.1.Then, the detailed algorithm is described in Sections 2.2 and 2.3.

Analysis of CT Value Profile through Sheet Structure
In the ideal case, where the CT scanning process has no blurring effects, the CT value  estimated through a sheet structure at position  can be expressed as follows: where  is the thickness of the sheet and  is the linear attenuation coefficient.In this case, we can easily measure the thickness by simple thresholding   /2 as shown in the left image in Figure 3.However, in real CT scanning situations, the blurring effect prevents an accurate thickness measurement.Herein, we consider a simple blurring effect induced by convolution with Gaussian function   : where  is the standard deviation of   and represents the amount of blurring effect, which depends on the focal spot size of the X-ray source, the pixel size of the detector, and the voxel size of the CT volume.Figure 3 shows a specific case with  2 and  1.As illustrated in the right image, the thickness of the sheet is inaccurately estimated as  2.8, which is calculated using the standard adaptive thresholding technique with half of the peak value  0 .
To analyze more general cases, we introduced scaled  with  ≡ /.The blurred CT value can be rewritten in a simpler form as follows: Note that   takes  if  /2   /2, where  / represents the ratio of the thickness to the amount of blurring.The left image of Figure 4 shows the graphs of   for various settings of parameter  0.5| 1,2, … ,10 .In the center image, we plot the change of the estimated thickness  with respect to  .For the estimation, we used the adaptive thresholding shown in the right image of Figure 3, i.e.;   2|  0 /2 |.From Figure 4, while the thickness can accurately be estimated for thick enough sheets (in the red dotted ovals), but the accuracy is low when  and  are close.Further, because of the slow increase rate of  in the inaccurate range, it is hard to distinguish sheets with different thickness values.In contrast, the peak value, which is denoted by     0 , significantly increases with the increasing thickness.The above analysis indicates that the use of the peak CT value  in addition to the width  is effective in segmenting sheet structure according to sheet thickness.In the following sections, the algorithmic detail of the proposed method is described.

Generation of Medial Surface with Thickness Features
To avoid processing the enormous CT volume data of a BiW directly, we initially create a triangle mesh that represents the medial surface of the sheet metal.Figure 5 illustrates a 2D version of the procedure steps.As shown in the left image, points  |  1,2, … on the isosurface of the input CT volume are extracted.We need to set the isovalue to low enough value to extract the thin sheets.Each point  is located between the six-connected neighboring voxels similar to grid-based polygonizer such as Marching cubes algorithm [5].The peak point  is found along the line segment directed in the CT value gradient ∇  .We calculated the gradient using the central difference method.As illustrated in the center-right image, sampling of points  |  1,2, … is redundant because the peak points are derived from the starting positions on the front and back sides of the sheets.To solve the redundancy, we apply a Poisson disk sampler [6] to perform spatially uniform down-sampling, as shown in the center-right image.The medial surface mesh is then generated by setting mesh vertices as the down-sampled points  |  1,2, … .We used GOM Inspect to obtain the triangles on the mesh.Figure 6 shows an example of the medial surface mesh.Generally, algorithms generating triangle meshes from point-clouds are designed to output two-manifold meshes.In the medial surface mesh case, as shown in the right image of Figure 6 the mesh is not appropriately generated along the function parts meeting more than two sheets, where we need to generate non-manifold edges shared with more than two triangles.A possible solution for this problem is to use algorithms specially developed for meshing medial axis of solid objects such as [7].In this research, we do not care this problem since segmentation method tends to separate the mesh along the junction parts.
After generating the medial surface mesh, two feature values are assigned to the vertices of the mesh.As illustrated in Figure 7, for each vertex  , the peak CT value  is simply the CT value at the vertex, i.e.,    .The width value  is the length

Segmentation of Medial Surface Mesh
Given the medial surface mesh that is assigned the two feature maps  and  , a simple region-growing algorithm is firstly applied to automatically cluster the mesh triangles.For each triangle  on the mesh, feature values  and  are simply determined by averaging the values at the triangle's three vertices.Then, the triangle with the maximum value among  in the unclustered triangle set is iteratively chosen as the seed triangle  for the region expansion.By thresholding, the weighted sum of the squared differences of two features from the values of the seed triangle, i.e.,      , the region growth is halted.The final mesh, as seen from the top and bottom of the BiW, is shown in Figure 2 bottom.The average of the peak values of the triangles is used to color-code the triangles that belong to the same cluster.
After clustering, the final segmentation is obtained by manually merging clusters.The final merging step was operated on mesh editing software POLYGONAL meister.Figure 8 shows an example of an editing process by a user.In this example, the user selected several clusters on the mesh and then eliminated them iteratively.After the merging step, to create a segmented volume,   the segmented surface mesh can be mapped to the original CT volume.As illustrated in the left image of Figure 9, we search the nearest mesh vertex in each voxel with a value greater than the isovalue and then assign the label of the vertex to the voxel.The right images in Figure 9 show the resulting volume segmentation with its cross-sectional view.Using the segmented volume, we can generate the isosurface mesh representing a part of the BiW, as shown in Figure 10.

Results and Discussion
The main parts of the BiW were successfully retrieved from the segmentation result of the BiW, which is shown in Figure 11.Because of the limitation of RAM, we resampled the original CT volume to the half-resolution and applied surface nets algorithm [8] to polygonize the segmented parts.The following points are planned to modify the proposed method in terms of automation, accuracy, and efficiency.
 To extract each sheet metal with the current approach, clusters must be picked up one by one; additional automation would make the system more effective.Additionally, we intend to create a system that automatically separates multiple-sheet areas and extracts spot-welding marks. Currently, we use the single thresholding to label the voxels in the sheet metal, which does not reflect the thickness feature values used in the medial surface mesh clustering.Adaptive thresholding method should be used to obtain more accurate geometry for thickness of the sheets. Loading fully sized CT volume is intensive to computational resources.However, once we compute the segmented medial surface mesh, spatially adaptive loading is possible to save the resource and achieve the handing more high-resolution volumes using the standard PCs.

Figure 1 :
Figure 1: Isosurface in the CT volume of a BiW (left), the cross-section image of the side-sill of the CT Volume (middle), and plots of the CT values estimated through the sheet structure (right).The values of width and peak are represented by blue dotted arrows and red solid arrows.

Figure 2 :
Figure 2: Proposed clustering method using two feature maps on the medial surface mesh of a CT-scanned BiW.

Figure 3 :
Figure 3: A simple bullring model of a CT value profile through a thin sheet structure.

Figure 4 :
Figure 4: CT profiles of various sheet thickness (left) and their feature values (center and right).

Figure 5 :
Figure 5: Computation for generating a medial surface mesh.

Figure 6 :
Figure 6: A cut-view of the medial surface mesh.The boundary edges of the mesh are colored in red.

Figure 7 :
Figure 7: Computation of the two feature values at a mesh vertex.

Figure 8 :
Figure 8: Example of an editing process for clustered medial surface mesh.