·Home ·Table of Contents ·Workshop - Reliability

Basic Ideas of the American-European Workshops 1997 in Berlin and 1999 in Boulder Christina Müller (Nockemann), Berlin, Germany. Unter den Eichen 87, 12205 Berlin, Germany. fax +49 30 8104 1837, phone +49 30 8104. 1833 Matt Golis, Columbus, USA. Tom Taylor, Richland, USA. Contact

Abstract

The main objective of the workshops was to come to a common understanding of NDE reliability and the existing problems and different ways of its determination between Europe and America and different fields of industry.
The main output of the first workshop was the set up of a conceptual model :

R º f(IC) - g(AP) - h(HF)
The conceptual model states that the total Reliability R of an NDE system is simply the sum of functions of: C, the Intrinsic Capability of the system AP, the effect of Application Parameters, and HF, the effect of Human Factors.
The second workshop discussed each one of the functions in the conceptual model seperately and detailed and accomplished an important first step in formalizing and developing an NDE reliability process by developing definitions for the most essential NDE reliability terms within the formula and related.


1^st European-American Workshop
Determination of Reliability and Validation Methods of NDE
June 18-20, 1997, Berlin, Germany

Goal

The principal motivation of the European-American Workshop was to provide a forum for examining the currently available technical and scientific methods for determination of reliability of nondestructive evaluation (NDE) systems and approaches to their validation. Importantly, the Workshop was not intended to address the broad questions of NDE reliability or resolve whether NDE is an art or measurement science. Also, the many legal, cultural, and political aspects of NDE, including certification and training, were bypassed. The main objective of the Workshop was to come to a common understanding of NDE reliability in a scientific-technical sense and different ways of its determination. The following topics were treated or touched during the workshop:
• Understanding reliability, validation and capability in NDE
• Essential quality characteristics for NDE
• NDE reliability and life cycle management
• Modeling of NDE processes
• Approaches for modular validation and inclusion of modeling results
• Statistical test design and analysis
• Human factors

This Workshop was held on June 18-20, 1997 in the Federal Institute for Materials Research and Testing (BAM), Berlin, Germany.

The scientific part of the workshop was jointly organized by BAM in Germany and NIST in the US within their framework of long-term cooperation. The technical organization was carried out by DGZfP. The international aspect of the conference is underlined by the institutions we have won as additional cosponsors: ASNT, BInstNDT, COFREND and ECNDT.

Attendees

The workshop was attended by nearly 100 participants: 20 from US and Canada, 50 from Germany, and 30 from other European countries, including Great Britain, Netherlands, France, Finland, Spain, Norway, Turkey, Czech Republic, Russia and Ukraine, Hungary, and Sweden. There were 28 invited oral presentations and 20 posters. The last day was devoted to consensus workshops that were attended by all participants.

Paper Presentations

The invited papers focused on the following issues:
• The different technical and scientific approaches to the problem of how to guarantee or demonstrate the reliability of NDE:
  • Application of established prescriptive standards
  • Probabilities of Detection (POD) and False Alarm (PFA) from blind trials
  • POD and PFA from signal statistics
  • Modeling
  • "Technical Justification"
• The dissimilar validation/qualification concepts used in different industries in Europe and North America
  • Nuclear Power Generation
  • Aerospace Industry
  • Offshore Industry
  • Service Companies

Focal Points of conclusions

During the Workshop, it became clear that there exists no absolute truth on how to determine the reliability of NDE, especially on how to quantify the human factors. However, there appears to exist a variety of promising approaches and a valuable pool of experiences, such as: Program for Inspection of Steel Components (PISC), Performance Demonstration Initiative (PDI), European Network for Inspection Qualification (ENIQ), Engine Titanium Consortium (ETC) and NORDTEST. Within this scope, there was a consensus agreement on the equal footing of practical trials, technical justification, and quality assurance in providing a reliable NDE. This is an important result of the Workshop.


Also, it was agreed that much of the misunderstanding within the technical community can be traced to the use of different vocabularies or application of the same word to different things. Consequently, the creation of a common dictionary was identified as necessary.

It was concluded by the participants that it is worthwhile to find a common understanding on the mathematical structure, on the one hand, and practical meaning, on the other, of the terms used in NDE reliability. This was accomplished by adopting a mathematical formula for NDE reliability that was put forward independently by Serge Crutzen and Matt Golis. The use of this formula should facilitate the objective handling of the terms capability and reliability and support a rational design of qualification/validation rules. In a more sophisticated form, the spirit of the formula is contained within the Modular Validation approach of Christina Nockemann.

The participants agreed that there is no absolute reliability of NDE. Rather, every NDE system must be qualified and validated on a case-by-case basis to address the special requirements of each client and satisfy the objectives of each test.

In addition to the discussion at the consensus workshops, a number of other issues were identified as being of potential importance:

• Human factors, role of realistic test conditions, role of education and training, possible reduction of human factors by automated inspection methods.
• Need for detailed and clear procedures that must be strictly followed and validated.
• Applicability of neural networks.
• When appropriate - possibility to replace "Prescriptive Standards" that describe how a method has to be applied, by "Performance Standards" that limit their content to the description of the capability that must be demonstrated. The "Performance Standards" have the benefit of easy harmonization, free competition on an open market, better adaptability to new technologies and avoidance of incapable procedures/techniques that might be the output of a bad consensus.
• "Useful" definition and application of calibration in NDE
• Opportunity to reduce the costs of an inspection by application of "Risk Informed Inspection". However, a "Risk Informed Inspection" requires that precise POD data are available.
• Cost reduction by application of computer modeling.
• Data base of specimens, flaws and reliability results

Specific Conclusions of Consensus Workshops

On the last day, there were four consensus workshops

• A: Understanding of Capability and Reliability of NDE
• B: What are the essential quality characteristics of NDE?
• C: Organization of Practical Trials
• D: How to do a modular validation? (How to include modeling results in a realistic
  validation?).

The workshops were moderated and held in sequence and were followed by a summary session. At each workshop conclusions were drawn and recorded.

WORKSHOP A
Understanding of Capability and Reliability in NDE

• from NDE point of view
• from the users of NDE point of view
• common definition

Moderators:

Serge Crutzen, JRC Petten, NL (recorder of A)
Steve Doctor, PNNL Richland, USA
Wulf-B. Klemmt, DASA Bremen, D
Matt Golis, AQC Columbus OH, USA

A "model" (empirical formula) was proposed and agreed to by the participants:

R º f (IC) - g (AP) - h (HF)

The formula states that the Reliability R of an NDE system applied or that the overall performance R of the application of the procedure, is simply the sum of functions of:

IC, the Intrinsic Capability of the system (technique or combination of techniques), generally considered as an upper bound,

AP, the effect of Application Parameters, such as access restrictions, surface state generally reducing the capability of the NDE system, and

HF, the effect of Human Factors, generally reducing the capability or effectiveness .

• The terminology of the model has the advantage of being an acceptable frame for discussion for both US and EU aeronautical, nuclear, and other industries.
• This is a good way of identifying which of the elements is considered during round-robin tests (RRT's), parametric studies, data discussions, and the like.
• The formula helps to understand what the models are addressing: IC or IC + AP or HF ?...
• This formula also helps to identify the range of coverage of performance demonstration (qualification / validation) actions

  and

  explains

  what

  "Open

  Trials"

  address:

  IC + AP and what blind trials address: HF or the whole of R.

WORKSHOP B
What are the essential quality characteristics for NDE?
Methods of their determination
(POD, false calls, uncertainty in measurement, essential parameters )

Moderators:   Gunther Engl, Siemens Erlangen, D
Chris Fortunko, NIST, Boulder, USA (recorder of B)
Ward Rummel, Littleton, CO, USA
L. Horacek, NRI Rez, Czech Republic
• Essential characteristics of NDE
  
  • Signal to noise ratio (Should it be > n as precondition for POD ?)
    
  • How to set threshold ?
  • How to deal with consequences of false calls, i.e. cost of repairs
• Validity of statistical tools for detection, classifying and sizing
• Uncertainty in detection, classification, and sizing <-> "Right" technique ?
• Uncertainty in measurement, calibration, and repeatability
• What is the definition of calibration in NDE?
• Impact of materials, component geometry, instrument/operator performance
• Commensurability (comparability of different NDE system results)
• Performance demonstration
• NDE reliability depends on (Ward Rummel)
  capability - inherent process
  reproducibility - calibration
  repeatability - process control

WORKSHOP C
Practical Workshop I:
Organization of practical trials
(How to do a performance demonstration?)


Moderators:

Frank Ammirato, EPRI Charlotte, USA
Tom Taylor, PNNL Richland, USA (recorder of C)
Patrice Lemaitre, JRC Petten, NL
Olav Førli, DNV Oslo, N
Sharon Vukelich, USAF ASC/ENFP, Wright-Patterson AFB, Ohio, USA
• Limitations
  
  • Practical trials are only part of process of qualification /performance
    demonstration
  • need also quality assurance & technical basis i.e. in terms of the "Technical Justification"
  • Important to establish the precise situations for practical trials
• Characterization criteria
  
  • Detection criteria: can vary; a lot depending on "situation"
  • Component / Defect Parameters
  • Flaw type
  • Orientation
  • Number of defects
  • Sizing
• Basic / common features
  • All countries / industries use samples that mock actual inspection
  • Reality of mock-up varies a lot
  • "Protocol"
• Types
  • Depending on "Technical Need", it is possible to use either or both"Open Trials" and "Blind Trials"

Other conclusions


• If one uses artificial defects - there must be a correlation (from NDTresponse) to a real defect characteristics
  • Empirical
  • Modeling

(1) Scheme of technical justification presented by P. Lemaitre

WORKSHOP D
Practical Workshop II:
How to do a modular validation?
How to include modeling results in a realistic validation?


Moderators:

Bruce Thompson, ISU CNDE, Ames, USA (recorder of D)
Martin Wall, AEA Harwell, GB
Floyd Spencer, SNL Albuquerque, USA
Gerd-Rüdiger Tillack, BAM Berlin, D


Current status of modeling compared to the total reliability of NDE (in reference to the equation proposed by workshop A)

Status
Good engineering starts always with some sort of modeling

• Modeling is a cross-cutting element of a quantitative measurement size with
  
  • Parametric studies
    
  • Calibration sample design
• Can use models to assess POD when experimental studies are impractical
  • Design
  • Aging plants
  • Examine interplay of multiple factors

Use of simulators to provide input to statistical studies regarding human response

Issues


Validation is a central issue
• Many studies have been done
  PISC III
  ETC
  Many research papers
• Results of these need to be integrated
• How to develop POD models for multi-dimensional data
• Problem of pattern recognition for 2D data e.g. X-ray
• How to deal with naturally occurring flaws
  • Learning from library response
  • Modeling the complex morphology

• How to choose appropriate models for a given problem
  • What is good enough
  • Validate models
• How to treat human factors
  • Data from PISC
    
  • Consider within the modular approach (Multiple PODs)
  • Compare predictions of POD from models to POD from humans examining
    model generated synthetic data(measurement of the human factor)
• How to describe the entire inspection process in a complete model
• User friendly formats.

2nd American-European Workshop on NDE Reliability - September 21-24, 1999
The second European-American workshop on NDE Reliability was held at the National Institute for Science and Technology (NIST) in Boulder, Colorado from September 21 to 24, 1999. The workshop was attended by a total of 43 scientists and engineers from 10 countries.


The objective of the second workshop on NDE reliability was to provide a forum for examining the conceptual model developed during the initial workshop and develop definitions for terms that are related to NDE reliability.

The workshop discussed each one of the functions in the conceptual model and accomplished an important first step in formalizing and developing an NDE reliability process by developing definitions for the following NDE reliability terms.

Definitions

NDE System is the procedures, equipment and personnel that are used in performing NDE inspection.

Reliability- NDE reliability is the degree that an NDT system is capable of achieving its purpose regarding detection, characterization and false calls.

Detection is a threshold-driven identification of the existence of a signal/indication to be of interest or worthy of further investigating.

Signal / Data Interpretation is deciding relevance of a signal/indication as being valid for further indication/materials characterization. e.g., geometrical reflections vs. cracks

Indication Characterization - Estimation of size, location, orientation, type, nearest-neighbors.

Indication Evaluation is the comparison characterizations to acceptance criteria for the purpose of making accept/reject decisions.

Ideal Capability is the hypothetical optimal performance of an NDE technique based on the governing physical principles.

Application Capability is the degree, to which an NDE system achieves its intended purpose, excluding human factors. It is defined in the context of the specification of expected application parameters.

Application Parameters are the factors concerning material conditions, discontinuities, procedure and equipment that influence the ability of an NDE system to meet its intended purpose.

Human Factors are the mental and physical make of the individual, the individual's training and experience, and the conditions under which the individual must operate that influence the ability of the NDE system to achieve its intended purpose.

Human factor function describes the affect of the human interface in the inspection process.

The workshop also redefined the relationships for NDE reliability into a more general form as follows.

R = f(AP,HF)

© AIPnD , created by NDT.net

|Home|    |Top|