Table of Contents ECNDT '98
Session: Reliability and Validation
Copenhagen 26 - 29 May 1998
Modelling of NDT Reliability (POD) and applying corrections for human factorsM. Wall, F. A. Wedgwood, S. Burch
AEA Technology plc, National NDT Centre, Culham Science Centre, Abingdon, OX14 3DB United Kingdom
Tel: +44(0)1235 464097 Fax: +44 (0)1235 463799
EMAIL: email@example.com, firstname.lastname@example.org
|TABLE OF CONTENTS|
By understanding and quantifying this reliability it is possible to assess what importance and value to put on inspection findings and also to improve the reliability and performance. In the last twenty years it has become common practice to quantify reliability in terms of two factors: the probability of detection (POD) and probability of false indication (PFI) as illustrated in Figure 1.
||Figure 1: Schematic diagram illustrating (a)POD/PFI and (b) relative operating characteristic (ROC) approaches to quantifying inspection reliability
In the last five years there has been a change of emphasis from experiment towards the development of computer models for predicting POD most notably within the UK National NDT Centre [2,3] and also in the USA . The increased power of computers, PC's and workstations, means that simulations of inspection processes are possible. A number of models are now available for POD using a range of approaches; these models are being increasingly used and validated. Such models mimic the inspection process and allow for statistical uncertainties. Methods are evolving to correct model predictions for human error.
This paper reviews these developments. The values of POD now being obtained by modelling are not now dissimilar in accuracy to those obtained in experimental trials, but it may be hard to model the real inspection process in detail. It is expected that the use of reliability models will become an increasing trend in the validation of inspection over the near future. This is analogous to the way finite-element stress analysis has complimented or replaced structural testing in many plant applications. Models also have their own distinct uses.
||Figure 2: Schematic illustrating the the basis of a POD model
The approach is similar to that used to predict POD by Thompson in the US aerospace industry. Our models allow analysis of image-based data, produce simulated images and allow correction for human and environmental effects. The estimates of POD combine estimates of signals expected from specific defects and transducers, estimates of background noise and a 'threshold criterion'. The simulation of defects is being constantly improved based on the analysis of 'real defect' data from operating plant.
These convert signal and noise values to POD and PFI using statistical methods. The signal and noise values can be derived from models or experiment, for example measurements on samples with reference defects. The approach in calculating POD is similar to the physical models above. This method avoids the statistical difficulties associated with conventional POD trials and allows predictions to be made for new inspection techniques that may be too complex to physically model. This approach has been used for many years in the USA aerospace industry , through the ã v a approach [Berens 3]. We have used this approach to make POD predictions for magnetic methods including MFE and as the basis for a basic spreadsheet model derived for member organisations of the Harwell Offshore Inspection Service (HOIS), managed by AEA Technology. A similar methodology has been used in Germany for aerospace component inspection. A modular approach can be adopted, with input data to the POD model derived from a physical model or experiment..
Image classification models/ Inspection simulators ('Visual' POD, 'Spot the Ball')
These represent methods for analysis of image-based inspection data such as radiographs, to give information in terms of POD and PFI. Inspection simulators are a special class of computer model that simulates the inspection process by presenting simulated inspection results to the operator. Interpretation of image-based data is more subtle and requires a more complex detection criteria than analysis of signal/noise data above. The detection criterion may be simply exceeding a threshold signal level at a number of locations, over a number of pixells or over a specified area ( as in the rastor scanning model developed by Ogilvy) or more closely configured to actual inspection system operation. Colin Windsor  has recently pioneered a neural-network based approach for detection of defects in image-based techniques, used in the radiographic model NNXPOSE. This uses receptive fields to search for and enhance specific defect types such as cracks porosity or slag inclusions and more closely reproduces the human interpretative skills of the inspector.
||Figure 3: Schematic illustrating the the basis of a POD model and the use of simulated images for POD trials to aid in the validation of model predictions. ('Visual POD', 'Spot-the-ball contest').|
Expert judgement has been used where input on POD is required for fracture mechanics or risk-based assessments and is not available from experiment. Provided the judgement comes from trained inspectors and sensitivity analysis is used this can be an effective method. The National NDT Centre maintains a computer database of POD information (PODDATA) which can be used to aid such judgements.
These use methods for statistical analysis or curve-fitting to experimental data, with the aim of making this data accessible for use in other applications (such as fracture mechanics). These do not model the inspection process as such.
Human reliability models
These take account of the effects of human error in the inspection process, and correct predicted POD values for these effects. An example is the methodology applied by AEA Technology to utilise human error data from the PISC III work  discussed later.
Figure 4: Effect of varying a single parameter: defect tilt.POD model calculations using PODUT
Figure 5: Model predictions of POD,PFI and ROC for voids in steel plate using radiographic POD model XPOSE. The detection threshold is varied
|Figure 6: Example output from AEA Technology radiography POD model XPOSE showing: (a) simulated radiograph and (b) corresponding POD, PFI curves as a function of defect size. The iImage quality indicator (IQI) lines numbered 1-7 are also shown in the radiograph.|
Figure 7: Comparison of techniques: POD model calculations for inspection for mis-orientated cracks in 25mm plate.
The simplest correction uses a constant multiplier (or POD reduction factor), typically 95% or lower for more severe environments. A better correction method uses curves such as the data for human error expressed in terms of POD in Figure 9, derived from PISC III data. This recognises that the effects of human error are greater for small defects close to threshold. In this case the human error POD is used as the multiplier H. An example showing the correction of a model prediction of POD from radiographic model XPOSE for human error is given in Figure 10. Experience to date shows that, where account is taken, through this use of a multiplier, the agreement of model predictions with experimental POD work is improved. This is an important area for future work. At the recent European-American workshop in Berlin  a general equation was suggested in which the theoretical POD was reduced to take account of environmental effects:
|Figure 8: Illustration of the environments that may be experienced in actual plant inspections.|
Figure 9: POD curves for human reliability derived from human reliability data from experiments in PISC III using AEA Technology ultrasonic simuilator PCSIMONE. I/Io is the signal (I) divided by the threshold (Io)
Where g(AP) and h(HF) are factors relating to the application of the technique (environment, surface, couplant, geometry etc.) and human error respectively.
Our POD models are based on physical models for ultrasonic and radiographic inspection which have been proven and validated over many years. We have taken the additional step of moving from a simple signal/noise calculation to a prediction of POD. To validate existing POD models we have used the following approaches:
Figure 10: Example showing correction of model POD curve for human error (HR) and comparison with experimental NORDTEST POD data . Correction using mean and lower bound human reliability curves in Figure 9. Radiography of 25mm steel plate.|
Where comparison with experiment is possible, reasonable agreement has been found between model calculations and experiment and parametric variations observed in experimental data are reproduced by models. One such comparison is given in Figure 10 and for the TOFD model similar comparisons have been given . For precisely defined input data model POD curves are generally sharper than experiment, but more realistic curves are obtained if 'randomising factors' such as defect orientation, background, attenuation or defect visibility are allowed to vary between practical limits found in real components e.g Figure 4.
There are several applications where modelling is the only possible approach: (i) parametric and sensitivity studies, (ii) assessment of historical inspection data (e.g. ageing plant), (iii) extrapolation and interpolation from experimental data, (iv) optimisation of inspection at the design stage, (v) POD trials using simulated inspection data ("Spot-the Ball") and (vi) to aid design of and support experimental POD trials. Currently our POD models have been used in all these applications and the approach has been extremely valuable in support of safety cases and inspection validation.
|1)||A number of computer models are now available for prediction of inspection reliability in terms of probability of detection (POD) and false calls (PFI). These cover an increasing range of inspection techniques and run in real time on a standard PC. The models are being increasingly used and validated. Methods are evolving to correct model predictions for human error.|
|2)||The model approach provides complimentary data to experimental assessments and allows existing experimental data to be more widely used. The models can provide specific data not available from experimental measurements such as parametric studies, assessment of historical data and optimisation at the design stage.|
|3)||POD models are already being used in economic assessments, integrity assessments, to support safety cases and validation of inspection procedures and plans. The values of POD now being obtained by modelling are not dissimilar in accuracy to those obtained in experimental trials.|
|4)||POD models should be seriously considered as an integral part of future POD trials. This could reduce the number of test samples required, help gain acceptance and familiarity for the modelling approach, provide validation and lead to improvements in the model predictions and correction methods used for human and environmental effects.|
|5)||It is anticipated that he use of POD models will increase significantly in future years. There is a need for development of customised models for specific applications and industries, increased validation as use increases and more work on human and environmental effects. Support from industry to meet these objectives is sought.|