Tools for the Analysis of Datasets from X-Ray Computed Tomography based on Talbot-Lau Grating Interferometry

This work introduces methods for analyzing the three imaging modalities delivered by Talbot-Lau grating interferometry X-ray computed tomography (TLGI-XCT). The ﬁrst problem we address is providing a quick way to show a fusion of all three modalities. For this purpose the tri-modal transfer function widget is introduced. The widget controls a mixing function that uses the output of the transfer functions of all three modalities, allowing the user to create one customized fused image. A second problem prevalent in processing TLGI-XCT data is a lack of tools for analyzing the segmentation process of such multimodal data. We address this by providing methods for computing three types of uncertainty: From probabilistic segmentation algorithms, from the voxel neighborhoods as well as from a collection of results. We furthermore introduce a linked views interface to explore this data. The techniques are evaluated on a TLGI-XCT scan of a carbon-ﬁber reinforced dataset with impact damage. We show that the transfer function widget accelerates and facilitates the exploration of this dataset, while the uncertainty analysis methods give insights into how to tweak and improve segmentation algorithms for more suitable results.


Introduction
Recent advances in X-ray computed tomography (XCT) hardware enable the application of the principles of Talbot-Lau grating interferometry (TLGI) in laboratory XCT scanning devices for industrial applications [1,2]. TLGI-XCT devices provide three modalities in one scan: the attenuation contrast (AC) resulting from attenuation of X-rays through a specimen, the differential phase contrast (DPC) containing information about refraction of X-rays, as well as the dark-field contrast (DFC) providing information about X-ray scattering. DPC is highly sensitive regarding changes in the material, for example, delamination or fractures. DFC shows high responses for X-ray scattering, such as from micro-pores or fiber bundles. The combination of all three modalities opens new possibilities for the characterization of material features that cannot be resolved by conventional absorption-based XCT alone. Analyzing these three modalities only side by side is typically insufficient for understanding where the single modalities complement each other. Domain experts, therefore, often want to investigate one fused image of all modalities. For this purpose, we evaluated widget designs that allow the user to specify a combined transfer function (TF) for slices of all modalities of a TLGI-XCT, leading to one simultaneous fused visualization. In addition, when processing multiple modalities in an image segmentation algorithm, it becomes even more crucial in comparison to a single modality to be able to inspect the segmentation process. For this purpose, we propose methods to compute uncertainty values for single probabilistic segmentation algorithm results as well as collections of segmentation results. We also provide methods to investigate these uncertainty types uncovered in the segmentation step of the image processing pipeline. Our contributions in this work are • a design study on supporting the initial inspection and exploration of TLGI-XCT data through a custom fusion widget, • a system for computing and analyzing the uncertainty in the segmentation step of TLGI-XCT data, • as well as case studies on TLGI-XCT data from a fiber-reinforced polymer dataset with which we show the usefulness of our methods.

Background
This section introduces existing work for both aspects of analyzing TLGI-XCT data addressed in this paper. We look at previous interfaces for designing multimodal transfer function widgets, as well as previous work on analyzing segmentation uncertainty.

Transfer functions for multiple modalities
An essential aspect to consider when designing a multimodal transfer function (TF) is the dimensionality of the data available, as having more than one dimension drastically increases the complexity, posing a challenge concerning the user interface. Ljung et al. [3] identify this issue and list a number of techniques (e.g., brushing or volume/slice view interactions) used in multidimensional (nD) TFs to overcome the limitations of a 2D display device. The classic 1D color-opacity TF widget, containing a histogram of attribute data in the background, already requires two dimensions to be displayed. The horizontal axis represents intensity values, the vertical axis represents opacity, and each control point contains a color attribute. Kniss et al. [4] use the gradient magnitude in addition to the intensity of a voxel to enable a more precise classification, but lose precision when manipulating color and opacity, which must be defined through regions such as boxes and pyramids. Patel et al. [5] extend this concept to using the first and second statistical moment. Combining different TF interfaces to achieve even higher dimensionalities is also possible. Zhou et al. [6] integrate a 2D TF such as the one presented by Kniss et al. [4] with a set of 1D TFs to define a separable 3D TF, capable of using three variables to classify the data. This is achieved by defining regions withing the 2D TF, selecting voxels by their primary and secondary attributes. For each region, a 1D TF is provided for further selection by the tertiary attribute.
To enable the interaction with TFs of even higher dimensions, Zhao and Kaufman [7] and Guo et al. [8] resort to the parallel coordinates plot, an attractive method due to the unlimited number of dimensions supported. However, relationships between attributes are lost, as each axis of this plot can only connect to at most two other axes. The interfaces mentioned, while presenting higher dimension solutions for the display and interaction of TFs, focus on multivariate datasets. These widgets do not, however, necessarily deliver satisfactory results for multimodal datasets, such as those produced by TLGI-XCT. Bramon et al. [9] take a step in this direction and present a fully automatic scheme to visualize bi-modal volumetric data. The colors and opacities of the TF are set given a predefined 1D TF for each modality. The authors suggest the possibility to generalize this technique for more than two modalities, but even so, the automatic nature of this method is inadequate for this work's goal of facilitating exploration. We focus on creating a single fused image. If in contrast a comparative analysis of the modalities is required, we refer the reader e.g. to the work of Malik et al. [10].

Segmentation uncertainty analysis
Segmenting an image is a challenge, even for a single modality. There is a multitude of algorithms available for this purpose, but so far, none has been found to be optimal, typically not even for a specific application scenario. In the analysis of multimodal TLGI-XCT data, it is even harder to find the most suitable segmentation. The information from the different modalities needs to be combined for the segmentation, and this combination step requires additional parameters. When optimizing segmentation algorithms for multimodal data and their parameters, a task which would fall into the Visual Debugger Challenge as identified by Heinzl and Stappen [11]. Multiple algorithms are typically executed, each one multiple times with varying parameters. To systematically explore the parameter space, methodical sampling of one or more algorithms is required [12], resulting in a set of slightly different segmentations, which we refer to as segmentation ensemble. For the case where segmentation quality can be judged by visual inspection and optimal parameters shall be found for a single algorithm, Pretorius et al. [13] suggest to analyze segmentation ensembles using tree-based visualization. Torsney-Weir et al. [14] target finding suitable parameters for the case when an objective quality measure is available. Fröhler et al. [15] propose methods for exploring the variation in a segmentation ensemble via hierarchical clustering and aggregated visualizations. For specific domains, there are specialized methods available, such as the work of Weissenböck et al. [16] for quantifying the pore volume in computed tomography data. These tools allow the user to explore the variation in the ensemble, however, they do not utilize uncertainty information. Saad et al. [17] show that uncertainty can provide valuable insights in the analysis of segmentation results. They propose a metric based on the relation of highest and second-highest label probability, and provide methods for editing the segmentation in regions of high uncertainty. Their tool is limited to analyzing a single result of a probabilistic segmentation algorithm. More recently, Summa et al. [18] have proposed a method to find alternative segmentations using uncertainty information, however only for specific graph-based segmentation algorithms. Potter et al. [19] introduced techniques for exploring a 2D field of probability distribution functions. Our probability probing technique extends this to three dimensions and provides information about correlations between different uncertainty measures.
Measures for uncertainty based on information theory have recently gained attention in the visualization of probabilistic segmentation algorithms, such as the work by Potter et al. [20], who introduce entropy as a metric for uncertainty, and propose several visualization techniques based on this metric. Al-Taie et al. [21] propose additional metrics based on the Kullback-Leibler divergence and the total variation divergence. These allow the incorporation of prior knowledge and introduce visual encodings, where uncertainty is color-coded on top of the gray-scale-encoded input image, attenuating or highlighting the uncertainty information over the maximum-probability labeled image. These methods deal with the outcomes of probabilistic segmentation methods. They do not consider the uncertainty information contained in the variation of a segmentation ensemble. Al-Taie et al. [22] introduce an ensemble segmentation method that utilizes the variation in the ensemble for a combined classification, but they do not provide visualizations for this information. The methods presented in this work are based on our previous work [23].

Methods
The analysis of TLGI-XCT data typically starts with the exploration of fused images. Prototypes of our methods for this purpose are described in Section 3.1. Segmentation is required for a more quantitative analysis. Our methods for analyzing the uncertainties involved in this step are laid out in Section 3.2.

Tri-Modal Transfer Function Widget for TLGI-XCT Datasets
As new extraction methods and equipment emerge, such as TLGI-XCT, more profound analysis of volumetric data is made possible. Yet, despite the available scanning equipment and techniques, modern visualization programs (e.g., VGSTUDIO MAX, Amira-Avizo and MeVisLab) do not provide an efficient way to explore multimodal volumetric datasets. Our goal is to test the feasibility of a tri-modal TF design for the exploration of TLGI-XCT datasets, with a focus on usability, taking into consideration the needs of material analysis experts.
The developed concept consists of three individual TFs, each corresponding to one modality, and one mixing function that receives the outputs of the TFs and combines them to create a fused visualization. This concept was implemented in a software prototype, shown in Figure 1. The prototype takes three two-dimensional scalar images (modalities) as input and produces an RGBA image based on the defined functions. Standard 1D color-opacity TF widgets built on top of modality histograms are arranged on the edges of a triangle and allow the user to manipulate the TFs of each individual modality. The 1D TF design was chosen due to its familiarity to domain experts. However, any other design can be used instead, including those mentioned in Section 2. Next to each TF widget, and rotated accordingly, are the results of the TFs and their respective modalities to provide extra guidance to the user. The triangular area formed by the histograms is used to define the weights of the individual TFs. It contains one control point that can be moved by the user (with a mouse click or drag) to any position inside the triangle. The on-screen position (i.e., Cartesian coordinates) of the control point is used to calculate its barycentric coordinates α, β and γ with α ≥ 0, β ≥ 0, γ ≥ 0, α + β + γ = 1. These conditions allow the interpretation of the barycentric coordinates as interpolation parameters, which makes it possible to control the mixing of the outputs of the TFs without having to adjust each TF individually. Furthermore, aiming to improve intuitiveness, the vertices of the triangle are defined in such a way as to increase a modality's weight (α, β or γ) proportionally to the proximity of the control point to the individual rendering of a modality. In other words, the closer the control point is to the individual rendering of a modality, the stronger is the influence of this modality on the final rendering.

Analyzing Uncertainty in the Segmentation of TLGI-XCT Datasets
Uncertainty has recently gained importance in understanding the image segmentation process [17,18]. Our goal is to extend the understanding of uncertainty information in the highly complex image processing pipelines for multimodal datasets such as those  from TLGI-XCT. When multiple modalities are available, finding the most suitable segmentation according to all modalities becomes even harder, as the modalities might hold contradicting information. This leads to a high variation in results when utilizing different segmentation algorithms, and thus in high uncertainty regarding the most suitable segmentation. The tools proposed in this work provide ways to quantify, explore and investigate this uncertainty.
To gather uncertainty information, we start by computing multiple segmentation results. We employ multiple probabilistic segmentation algorithms, such as fuzzy C-means based segmentation methods (FCM) [24], probabilistic support vector machines (SVM) [25], as well as the extended random walker algorithm (ERW) [26]. We sample the parameter space of these algorithms with the help of our tool for sampling arbitrary image segmentation algorithms [27], as shown in Figure 2(a). The output is a collection of different results, which we refer to as an ensemble. We identified three types of uncertainty information we could leverage in such an ensemble, resulting from: • probabilistic segmentation algorithms, • neighborhood variability, and • ensemble variability For each voxel in each segmentation result, a probabilistic segmentation algorithm provides the probabilities that this algorithm assigns to each label, which can be interpreted as a label probability distribution, as shown in Figure 2(b1). Also, high variability in the neighborhood of a voxel can be understood as a measure for uncertainty. For this purpose, we create a distribution out of the label assignments in the neighborhood of the voxel, as visualized in Figure 2(b2). The variability over the ensemble of results, computed with different algorithms or parameterizations, provides clues to the uncertainty of the segmentation. We build a probability distribution for each voxel across the ensemble, as displayed in Figure 2(b3). To retrieve uncertainty values from these distributions, we employ the entropy as a measure for uncertainty, as proposed by Potter et al. [20], this is depicted in Figure 2(c). There is a distribution for each voxel in each segmentation result for the probabilistic segmentation algorithms, as well as for the neighborhood variability. Therefore, we can compute an uncertainty value for each voxel and each segmentation result. We call these values algorithm uncertainty and neighborhood uncertainty. In the case of the ensemble variability, we only have a distribution for a voxel over all segmentation results in the ensemble. We thus only compute a single uncertainty value for the whole ensemble for each voxel, referred to as ensemble uncertainty. After computing these uncertainty values, we employ the visualizations as shown in Figure 2(d) to analyze them. The full exploration interface, with TLGI-XCT data loaded, is shown in Figure 3. For each uncertainty type, we provide an aggregated uncertainty image. For the algorithm uncertainty and neighborhood uncertainty image, where we have one uncertainty value per voxel and segmentation result, we compute the value for each voxel as average value over all segmentation results for each voxel. For the ensemble uncertainty image, we can simply utilize the ensemble uncertainty for each voxel, as we only have a single value per voxel over the whole ensemble. The uncertainty images are color-coded such that black indicates low uncertainty and white indicates high uncertainty. Slice images of these uncertainty images are shown alongside individual segmentation results in the spatial view, as shown in Figure 3(a). An average uncertainty is also computed for each segmentation result and displayed in a bar chart, sorted in descending order, as can be seen in Figure 3 correlations between uncertainty types and links these uncertainties back to their spatial locations in the datasets.

Evaluation
We evaluate our methods on scans of a carbon fiber reinforced polymer (CFRP) object with impact damage. Slices of the reconstructed volume of all modalities acquired by the TLGI-CT device are shown in Figure 4(a-c). First, we inspect the scan with the tri-modal TF widget, as described in Section 4.1. Then, we segment the dataset to enable a quantitative characterization of the damage, and we analyze this segmentation using the uncertainty exploration methods in Section 4.2.

Fusion Results
To evaluate the efficacy of the prototype, features of interest to domain experts are highlighted (e.g., cracks or pores). These features are emphasized differently in each TLGI-XCT modality. The prototype was loaded with the slices in Figure 4 and it enabled the exploration of all three modalities simultaneously in one visualization, as shown in Figure  Selecting voxels with high algorithm uncertainty and low neighborhood uncertainty in the scatter plot (e) highlights the corresponding voxels in the algorithm uncertainty image (f). One result from the ensemble where the ring artifacts were labeled as damage is selected in the average uncertainty bar chart (h) and shown in the spatial view (g). Selecting regions of low ensemble uncertainty in the scatter plot (i) highlights background regions in the spatial view (j.) results in the former two having a stronger influence, whereas the latter is effectively invisible. Additionally, domain experts were consulted on the intuitiveness of the tool. The 1D TFs used for each modality and the weighting based on the distance to the modalities were considered intuitive. However, the rotation of individual renderings was criticized, as it hinders its comparison with the input slices. Having the control point increase the weight of a modality corresponding to the closest rendering was also considered suboptimal, and corresponding it to the closest histogram (1D TF) was suggested as an alternative.

Segmentation Uncertainty Analysis
We first apply the probabilistic segmentation algorithms described in Section 3.2 to create an ensemble of segmentation results from the multimodal input. We then load these results into our uncertainty analysis interface, as shown in Figure 3. The algorithm uncertainty image in Figure 6(a) shows a generally high level of uncertainty due to the medium gray colors over the whole image, but it is highest along the edges of the specimen as well as at the location of damage inside the specimen. The neighborhood uncertainty image in Figure 6(b) only shows high uncertainty along the edges and along cracks. This is to be expected, as this uncertainty type only considers the uncertainty within each single segmentation result. The ensemble uncertainty image in Figure 6(b) shows high uncertainty in the whole specimen. This is due to the fact that the algorithms could not agree on a common label for the specimen. Some would label it with label 1 while others would assign label 2, even though most algorithms (SVM as well as ERW) are given prior information on what labels to assign to certain gray levels. This supports our initial problem statement that it is hard to find a suitable segmentation for such a multimodal dataset. There are also ring artifacts, highlighted in red in the attenuation image in (d). We select voxels with high algorithm uncertainty and low neighborhood uncertainty in the scatter plot, the selected voxels are marked yellow in Figure 6(e). The spatial visualization of this selection in Figure 6(f) shows that this affects mostly those regions of the image where ring artifacts are located. When inspecting multiple segmentation results, we would see that indeed many algorithms labeled the ring artifacts as damage. In our tool we can do so, by selecting the results in the average uncertainty bar chart (h), where one segmentation result showing these ring artifacts segmented as damage is selected, it is shown in Figure 6(g). We can conclude that we need to apply additional ring artifact correction in order to achieve more consistent segmentation results. When plotting the ensemble uncertainty versus the algorithm uncertainty, as shown in Figure 6(i), we can see two clusters. The cluster of voxels with low uncertainty is selected in the scatter plot. In the spatial view in Figure 6(j), we see that this highlights nearly all background voxels. The algorithm uncertainty, as we can see in the scatter plot, does not separate these voxels, their values cover the full range from low to high. We can conclude: While the single algorithms are about as uncertain about the background as about the rest, the whole ensemble still agrees much more on the segmentation of the background than it does on the segmentation of the object.

Conclusion
In this paper, a system for analyzing and processing multimodal images from a TLGI-XCT device is presented. The methods enable users to explore and experiment with a slice image fusing all modalities. Furthermore, we provide methods for analyzing the uncertainty when segmenting these modalities. The presented case study shows that the developed methods can facilitate the tasks of domain experts working with TLGI-XCT data and provide additional insight. We demonstrated the feasibility of a trimodal TF design for TLGI-XCT images, and that the introduction of a weighting widget (the triangle) is sufficient to accelerate and facilitate the highlighting of features across modalities. We showed that considering uncertainty can provide interesting insights into the inner workings of segmentation algorithms, and can provide hints as to additional image processing steps (e.g., artifact correction) or adaptations to the segmentation algorithm that are required. In the future we plan to test a tri-modal TF design working with volumetric visualizations instead of slice views. We also plan to redesign the widget for further usability improvement. We are also thinking about utilizing the uncertainty information directly in improving the segmentation, for example by merging segmentation results based on their uncertainty.