Comparison campaign of XCT systems using machined standards representative of additively manufactured parts

Additive manufacturing enables the production of complex geometries, both internally and externally. This poses a challenge to quality control. Non-destructive volumetric testing methods are required. Among these methods, X-ray computed tomography is currently the most promising. It enables not only to detect inner and outer defects but also to perform dimensional measurements. However, the method lacks of metrological tracability. To fill this gap, an EMPIR project 17IND08 AdvanCT involving several European countries was proposed. In this frame, DTU Mekanik and LNE have conducted a comparison campaign of XCT systems using machined material measures representative of additively manufactured parts. The purpose of the comparison was to investigate the performances of industrial X-ray computed tomography with respect to dimensional measurements on external but also on internal features.


Introduction
A growing number of industrial sectors are considering the potential of additive manufacturing (AM) as an asset to improve their production capabilities.Indeed, AM enables the fabrication of very complex shapes and inner cavities that cannot be manufactured with conventional techniques.Nevertheless, AM has been shown to produce specific types of defects in as-built parts [1].Thus, before they can be used in safety-critical areas, such as aeronautics, medical sector, etc, the integrity of the AM parts and their compliance with specifications must be ensured to be certified.This requires stringent quality controls to be implemented.With such complex geometries, inner cavities, and high surface roughness, it is particularly challenging to characterise the parts' geometry, the dimensional accuracy and the inside of the parts in terms of flaws, without damaging them.
The characterisation methods need to be non-destructive as well as volumetric and unaffected by surface conditions.X-ray computed tomography (XCT) meets these requirements and is presently the most suitable method [2].However, the traceability and the evaluation of the uncertainty for XCT dimensional measurement are still critical tasks, above all in the case of internal features and structures.
The overall objective of the EMPIR project 17IND08 AdvanCT for "Advanced Computed Tomography for dimensional and surface measurements in industry" [3] was to improve the metrological quality of dimensional XCT measurements.In the frame of this project, DTU Mekanik and LNE designed and manufactured a number of material measures (also called transfer standards in the following) to lead a comparison campaign of XCT systems.As both laboratories are involved in quality control of AM parts, one decided to develop transfer standards representative of AM parts in terms of flaws and material, meeting the needs of the industry.These transfer standards as well as the chosen measurands were fully described in the previous iCT2020 paper [4].
Several comparison campaigns of XCT systems have been conducted and the results published.Their aims as well as the investigated transfer standards were different.A. du Plessis et al., in 2019 [5], aimed to advance the efficient use of micro-XCT facilities for process optimization and quality inspections for AM products in order to provide confidence in their use but also to indicate the need for further development of transfer standards, protocols and image analysis workflows for quantitative assessment, especially for direct and quantitative comparisons between different laboratories.Whereas, the different comparison campains presented by M. Bartscher et al., in 2018 [6], focused on dimensional measurements.However, none of these interlaboratory comparaisons aimed to investigate specifically the performance of industrial XCT with respect to dimensional measurements on external but also on internal features.
After a summary of the previous iCT2020 paper, with a brief review of the transfer standards and of the measurands, this paper will present the metrological calibration of the transfer standards and the statistical analysis of the results of the comparison of XCT systems.The XCT campaign involved several laboratories and companies from all over Europe: Zeiss (DE), Novo Nordisk (DK), DTI (DK), NPL (UK), Messtronik (DE), CEA-List (FR), MTC (UK) and the University of Nottingham (UK).

Physical standards for the comparison campaign of XCT systems
Three material measures, such as the ones presented in Figure 1, designed with the purposes of being representative of the AM parts and of evaluating the capability of XCT to detect internal features in mono-or multi-materials, were machined.In order to be representative of AM parts these standards contain external but also internal spherical calottes, of different diameters, simulating internal AM flaws, internal hollow cylinders simulating internal structures and external grooves simulating cracks.In addition, they are in acrylonitrile butadiene styrene (ABS), stainless steel and aluminium, materials used at lot in the AM industry [4].Internal plugs allow for evaluating the capability of XCT to detect internal features in mono-or multi-materials, and, as they are removable, their metrological qualification with surface reference methods such as coordinate measuring machine (CMM) is possible.

Measurands for the comparison campaign of XCT systems
The chosen measurands are presented in Figure 1 and described in Table 1.Table 1: Measurand definition (cf. Figure 1).

M2
Width of the external insert groove 1 on step 3 i.e. distance between Point 1 and Point 2. Point 1 corresponds to the intersection of the x-axis and edge 1: intersection of planes (GG) XY1 and YZ1.Point 2 corresponds to the intersection of the x-axis and edge 2: intersection of planes (GG) XY2 and YZ2.The coordinate system is defined as follows: 0 is the centre of the hole used for the screw holding the external insert, the y-axis is parallel to edge 1, the x-axis is in plane XY1 and the z-axis perpendicular to XY1 plane.

M3-1 to 4
Diameters of the 0.6 mm bore at 4 heights on the internal insert on step 4.These diameters correspond to the diameters of the circles fitting (GG) the bore at Z=-0.1 mm (M31), Z=-0.58 mm (M32), Z=-0.82 mm (M33) and Z=-1.3 mm (M34) M4 Diameter of the sphere which fits (GG) the calotte P3 on step 4 internal insert

Metrological calibration of the transfer standards
The reference measurements of M1, M3 and M4 were performed at DTU with a tactile CMM, and the measurements of M2 were performed at LNE with an optical CMM.They were carried out either before or after circulation of the standards.
The DTU machine is a mechanical CMM equipped with a static probe placed in a temperature controlled room (T = 20 °C ± 1 °C).This CMM is of the type Zeiss PRISMO (Erreur !Source du renvoi introuvable.2) with a VAST XT-GD1 measuring sensor.The calibration equipment was used with CALYPSO 6.8 software.
The measurements to evaluate M1 were performed with a 0.8 mm ruby sphere, a 0.6 mm tungsten shaft, a 8 mm shaft length probe and a measuring force of 50 mN.The position of each calotte was measured by scanning 4 circles picking up 32 points on each circle at 0.125 mm/s.The measurements to evaluate M3 and M4 were performed with a 0.3 mm ruby sphere, 0.2 mm tungsten shaft, 3 mm shaft length probe and a measuring force of 50 mN.The bore circles related to M3 were measured by scanning 2 circles of 32 points at 0.125 mm/s.The calotte related to M4 was measured by scanning 2 circles of 32 points at 0.125 mm/s.Traceability of the reference measurements was documented by the general condition of instruments and repetition of the measurements, which showed good consistency.

Comparison campaign of XCT
Ten laboratories and companies, listed in Table 2, participated to the XCT measurement's campaign.These measurements were performed with different XCT systems, and post-processed by different software tools, also given in Table 2.
A technical XCT measurement protocol was drafted and sent to all participants beforehand.However, no strict instructions were given for the scanning such as the XCT parameters and the number of scans.The transfer standards were to be measured by selecting all XCT parameters according to the experience of the participant to give the best of his XCT system in term of dimensional measurement uncertainty in order to be placed in the conditions of XCT service providers.In addition to the protocol, a template table, specifying the information to give on the XCT systems, XCT scans, etc, was also sent.
The comparison was based on the circulation of three transfer standards, one in ABS (ABS1 then ABS2-severe cracks in ABS1 forced to switch to ABS2), one in aluminium (Al1) and one in stainless steel (SS1).According to the available energies of the systems, the stainless steel standard was measured with a few systems and according to the chosen XCT parameters, some measurands could not be determined.Those, which were measured, are listed in Table 2.

Statistical analysis
Firstly, the results were analysed in accordance with ISO 5725 parts 2 and 4 [7,8].These ISO standards provide practical guidance for evaluating the accuracy of a measurement method.Although the accuracy is a qualitative characteristic, in these ISO standards, and generally in the validation method field, it is designated by the two properties precision [7] and trueness [8].According to these documents, trueness refers to the bias between the arithmetic mean of a large number of test results and a reference value, whereas precision evaluates the closeness of agreement between test results.
Secondly, the z scores and the zeta scores, defined in ISO 13528 [9], were calculated for each laboratory.Using the bias of one laboratory, these criteria allow to evaluate the overall performance of the laboratory.Moreover, zeta scores may reveal underestimated uncertainties.

Application of the ISO 5725-parts 2 and 4
ISO 5725-part 2 is dedicated to precision and ISO 5725 part 4 to trueness.

Precision: repeatability and reproducibility
Precision is quantified by two uncertainty contributors, viz.repeatability and reproducibility.Repeatability represents the dispersion of the results obtained under unchanged measurement conditions.Reproducibility represents the dispersion of the results where at least one change has been introduced [10].
The variances of the results were initially analysed with a Cochran test.If the test failed, the individual results associated to the maximum variance were successively analysed with a Grubbs test for the detection of outliers in the data set [7].
After the outlier's removal, the repeatability Sr was evaluated as weighted mean of the standard deviation of each laboratory's results Sri, according to Eq. 1 where dofi represents the number of degree of freedom associated to the standard deviation Sri.
Eventually, an ANOVA test (ANalysis Of VAriance) was perfomed on the participant's mean values to highlight systematic differences among laboratories.Measurement results affected by a systematic behaviour could be accounted for as laboratory effect   2 evaluated according to Eq. 2 where   2 corresponds to n times the variance of the mean-n is the number of repeated measurements of each laboratory.
Finally, the reproducibility standard deviation SR was obtained using Eq. 3 (3)

Bias of the method
Trueness is normally expressed in terms of bias [8].
According to ISO 5725-4, an overall mean value xmean representative of the comparison results (method in the ISO standard) was evaluated for each measurand after applying Cochran and Grubbs tests.Thus, the mean value of each measurand was compared with the corresponding reference value xref to evaluate the bias of the method (see Eq. 4) The reference value xref is discussed in § 5.4 of ISO 5725-4 [8] and in § 7 of ISO 13528 [9].
The standard uncertainty of the bias was evaluated according to Eq. 5 (see § 5.5.3.2 of ISO 5727-4) The two components in the equation are: the precision of the method estimated with standard deviation SR and the measurement uncertainty u(xref) of the reference value.It should be noted in Eq. 5 that the number p of participating laboratories plays a central role, in particular for the significance of the estimate.

Application of the ISO 13528
The proficiency test result of a laboratory can be assessed by a z score according to Eq. 6 Zeta scores can also be useful to evaluate a laboratory's capability to have results close to the reference value within its stated uncertainty.It was computed using Eq.7 Z score indicates agreement among the results (or the lack of it), wheras zeta score shows coherence of the measurement uncertainty assessment with respect to the reference uncertainty (e.g. an underestimated uncertainty).
Z scores and zeta scores lying outside the interval [-2,2] are considered questionable, while those lying outside the interval [-3,3] need further analysis for explaining the causes, and possibly correcting the deviation from the reference value.

Statistical analysis of the results of the comparison of XCT systems
LNE analysed the measurement results provided by the participants using ISO 5725 parts 2 and 4 to evaluate the performance of XCT systems, and some criteria of ISO 13528 to evaluate the performance of each participant.
In this study, to guarantee the confidentiality of the participants, laboratories were identified by numbers.
When an outlier was found using Cochran and Grubbs tests, the interested laboratory was contacted for explanation, and to discuss the possibility of removal.
The uncertainties given by the laboratories were compared with those associated with the reference values, to check that the first ones were higher than the last ones.When possible, we analyzed the uncertainty factors identified by the laboratories.
For the reference values, it was recommended to measure the transfer standards with the CMM reference method before and immediately after the inter-laboratory test ( § 7 of ISO 13528 [9]).However, this could not be possible because of the Covid pandemic.Therefore, we did not follow the standard procedure, which proposes to estimate additional uncertainty components associated to these instabilities.
When, in this study, too few laboratories provided results and repeated measurements for some of the transfer standards, we did not combined SR and u(xref) to compute the uncertainty associated to the bias, U(B).Instead, we only compared the magnitude of these two components to the bias.For example, a bias lower than the reference value uncertainty may be considered as nonsignificant.Also, a bias higher than the reference value uncertainty but lower or similar to SR may be considered as nonsignificant.
The analysis results for the standard Al1 are given below in Tables 3 to 6. Laboratory 9 has been eliminated with the Grubbs test for measurands M1, M31 and M32.Thus, its results were not taken into account to compute the general mean and the reproducibility standard deviation SR.
Table 3 gives the overall mean values xmean of all laboratories' measurements, the reproducibility SR, the reference values xref as well as their uncertainties U(xref) and bias and Table 4 the uncertainty associated with the mean value for Al1.
Table 3: General means and bias of the comparison for Al1.
Table 4: Uncertainty associated with the mean value for Al1.Note: the uncertainty associated to reference value M1 is given with k = 2.4 but, to compute U(B), k = 2 was used.
The z scores and zeta scores evaluated for each laboratory are given in Tables 5 and 6 for the aluminium transfer standard Al1.
In addition, the zeta scores are represented graphically in Figure 4.     SS1: due to the high density of the material, only three laboratories were able to scan the transfer standard and provide the measurements.Consequently, it is difficult to draw conclusions.However, we noticed that the method seems biased for measurand M4 due to a high bias and a reference value lying outside the range of the laboratory's results.For the other measurands, the method provided results more dispersed than biased.Laboratory 5 has shown good performances with zeta scores in the interval [-2,2].

Conclusions
This paper presents the statistical analysis of an XCT dimensional measurement campaign involving several participants from all over Europe: Zeiss (DE), Novo Nordisk (DK), DTI (DK), NPL (UK), Messtronik (DE), CEA-List (FR), MTC (UK) and the University of Nottingham (UK).Three transfer standards were designed and machined with the purpose of being representative of AM-manufactured components.Particular relevance was given to the capability of XCTs to detect internal features in monoor multi-material elements.A technical XCT measurement protocol was drafted and sent to all the participants beforehand.No strict instructions were given concerning the scanning, such as the XCT parameters or the number of scans.The participants were to measure according to their experience, selecting all XCT parameters for the best performance in terms of dimensional measurements and uncertainty, i.e. in usual conditions of XCT service providers.
Most of the laboratories stated values and uncertainties consistent with the reference values.Nevertheless, mainly considering the different available maximum voltage of the instruments involved in the comparison, this study also highlighted the following:  The number of results available for SS1 was insufficient to provide statistically meaningful results.
 The XCT systems used exhibited different performances.This led either to the elimination of some laboratory's results, or to a high reproducibility.
 Except few cases, no repeated measurements were available.Thus, it was necessary to adapt the calculations described in the series ISO 5725, resulting in larger estimated uncertainties for the bias and the reproducibility.This suggests that a minimum number of XCT scans should have been indicated explicitely in the technical protocol, to be sufficient for a rigorous statistical analysis.
 Very heterogeneous uncertainties were stated by the participants, which in some cases were also underestimated.Even though the sources of uncertainty were explicitely requested, they were provided by only few participants.Therefore, it is not possible to add specific information in this regard.

Figure 2 :
Figure 2: CMM of the type ZEISS PRISMO placed at DTU.The LNE machine used was an optical CMM.It is of the type MICROVU EXCEL502UM (Figure 3) equipped with an optical zoom from x25 to x600 and digital zoom up to x1800.The CMM is placed in a temperature controlled room (T = 20 °C ± 0.5 °C) with a hygrometry of 55 % ± 10 %.The calibration equipment is equipped with INSPEC 2.97.2 software.The groove widths of the external inserts were measured in episcopy at x300 magnification on the upper edge of the groove at mid-length.
The zeta scores of the laboratories 7, 8 and 10 are out of the interval [-3,3] denoting an underestimation of their uncertainty.Other six laboratories (1, 2, 4, 5, 6 and 9) have zeta scores included in the interval [-2,2].The results given by these laboratories were considered in compliance with the reference values.Thus, the overall comparison outcome could be considered satisfactory.

Table 2 :
Participants, countries, XCT systems and post-processed softwares, and evaluated measurands according to the transfer standards of the comparison campaign of XCT.
Al1: the method seems biased only for M2.In addition, there is a strong dispersion of the results because the relative standard deviations are greater than the biases.Six laboratories out of nine have zeta scores in the interval[-2,2].The XCT method can be considered well controlled with the Al1 transfer standard.ABS2: the method seems biased only for measurand M2.The results are more dispersed than biased.The laboratory 5 has shown good performances with zeta scores in the range [-2,2].