·Home ·Table of Contents ·Workshop - Reliability

Whole R Considerations for NDE in American Aerospace L. Schaefer, PNDE 364 Renton Center Way SW #56A-443 Renton, WA 98055 USA (206) 390-1661 E-mail: lloyd.schaefer@usa.net Contact

Abstract

NDE has been a business given in the aerospace industry for many years. It is routinely applied via regulatory requirement, design practice, and procurement specification, to the entire engineered component life cycle. Applications include assurance of raw material, screening of newly fabricated components and assemblies for manufacturing defects, as well as in routine maintenance, and design life extension of high value structure and systems. The majority of NDE conducted in these settings is observed to be prescriptive and deterministic in nature, providing primarily qualitative assurance of freedom from defects that would adversely impact design service goals.

Emphasis on advancing understanding of NDE processes in the last 2 decades of the 20^th century has questioned NDE's role and value from several perspectives. High profile failures of inspected structure such as the Space Shuttle, an Aloha Airlines 737 fuselage, and a turbine disk in a United Airlines DC-10 have driven emphasis towards tighter control and higher reliability of the inspection process. In the same period, Design, Quality and Manufacturing initiatives such as; Zero Defects, 6 Sigma, and Robust Design, have sought to reduce/eliminate finished part inspection through improved monitoring and real-time control of manufacturing processes. Lifecycle and risk management activities have also desired to understand and utilize the influence of NDE in the prediction of the maintenance and economic life of infrastructure. All of these perspectives contain a common element; the desire to quantify the effect of the NDE processes. Successful development of this knowledge will facilitate the essential culture change for the prescription of NDE from qualitative and regulated "goodness", to one of precise, quantified value. The pace and intensity of these efforts varies widely within different aerospace market segments, but continues to make essential progress.

This paper will review global Reliability, or whole R, for NDE processes, as applied in the aerospace industry. Status and trends will be summarized in the context of the second American-European NDE Reliability Workshop result, with an eye towards creation of a NDE reliability metrics culture and knowledge/practice infrastructure. Initiatives to treat the NDE processes through the concepts of target, the desired POD curve, and variance from target, will be reviewed for both in progress and completed, implemented examples.

Introduction

A common theme expressed in the second American-European NDE reliability workshop, held in September of 1999, was the emphasis on having accurately measured confidence in present capabilities, over applying resources to reduce the target size/condition to be measured. While not unexpected conclusions, given the workshop topic, it re-emphasized the comprehension and definition of global R for NDE processes that is desired from the aerospace NDT community by its customers. In addition to the emphasis on reliability over detectable flaw size in NDE performance metrics, there are also observed changes the scope and significance of reliability deliverables. Customers in the form of structural analysis and risk assessment functions are valuing NDE reliability in terms of the entire POD function (e.g., the probability of detection versus flaw size), vs. a single number statement of reliability e.g. 29/29, to determine the net distributions of flaws in a structure[1]. Therefore the workshop also found it important to examine POD distributions for linkage to the ideal/theoretical capability (inspection physics models), in order to assure accuracy over the entire range of probabilities stated. The function to describe overall reliability for NDE processes was defined as;

R=f [AC,HF] < IC,


where overall NDE reliability, R derives from variance due to applied capability (AC) limits and human factors (HF) influence, and is found to be less than some Intrinsic or Ideal Capability.

It is recognized that, now more than ever, it is preferred to create and operate structure where the inherent and operating accumulations of defects can be tolerated for the useful life of the component/system. The efforts put forth in the workshop were observed consistent with that economic vision of built in quality, and reality that such a target must be validated through measurement. For example; expensive, complex infrastructure such as an airplane may have been constructed with adequate quality for its design service life of 20 years. However, changing economic realities, designs which tolerate a distribution of imperfections, and an environmentally motivated desire to maximize useful life, has made it increasingly desirable to operate airframe structures and propulsion systems beyond their original design target[2]. The policies and tools that were discussed in the workshop have the potential to develop a more precise knowledge of the useful life of these structures.

The second American-European NDE reliability workshop accomplished an important first step in formalizing and developing an NDE reliability process. This was to create a lexicon of NDE reliability, which could be adopted across the diverse industry groups represented. Within the following discussion of the proposed definitions, introduced in adjoining summaries of this workshop, it will be attempted to further describe the potential effects and import of NDE reliability to the aerospace industry.

Aerospace Methods and Motivations to Define NDE Reliability

As summarized in (3) and recommended articles within the reference, the basic concepts and approaches to describing NDE reliability in terms of POD for aerospace programs are not new. The pace and frequency of the efforts to create a reliability focused culture in NDE have been somewhat uneven and often event (failure) driven. There is also a correlation observed between regulatory/contractual mandate and the degree of rigor that has been applied to defining POD. This historical record and present level of interest in NDE reliability is believed a result of; the high cost of obtaining the knowledge, and the slow but progressive interest in quantifying the value of the NDE process. With legacy reliability programs such as Retirement for Cause, ENSIP, and requirements such as MSFC-STD-1249, and MIL-STD-1823 demonstrating/prescribing the benefits of knowing NDE reliability, design and life extension initiatives to investigate and apply NDE probabilistically are gaining in acceptance. In response to these initiatives, and resulting studies, new models for the reliability function are being developed and considered. An effective reflectivity method for ultrasonic inspection has been introduced[4] in an attempt to make reliability estimates more sensible in terms of the physical model of inspection. A new approach to the treatment of hit/miss data is under investigation[5] which employs a 4 parameter model to manage responses which are perceived by the investigator as independent of crack length in eddy current exams.

NDE Reliability Relationship to the Taguchi Loss Function

When the design and manufacturing processes are on target, the total economic loss, described by the Taguchi loss function[6], for the producer, consumer, and society is minimized. Inherent in those targets are tolerable levels of imperfections, and also limits/variability in attempts to relate the manufacturing process metrics to component defect distributions. Given an imperfect structure, the NDE process adds value through attaining its target of a precise description of the probability of detection of these defects. The variability in the NDE estimate results in economic loss through inability to accurately predict the useful life of the structure. This philosophy helps achieve some consistency between NDE and other manufacturing process initiatives.

Ideal Capability (IC) - The NDE Process Target?

NDE reliability was described in the workshop to be the degree that an NDT system, which is the procedure, equipment & personnel, is capable of achieving its purpose regarding detection and characterization, at an acceptable false call rate. There were discussed two quite different approaches of defining the reliability target. One suggested that the target be described simply as the size of the defect to be detected via a calculated proportion and confidence bound, derived through a combination of open and blind trials. The other, which motivated the change from the 1997, Intrinsic Capability, to the present, Ideal Capability for IC, defined IC as the hypothetical optimal performance of an NDE technique based on the governing physical principles. Concerns were voiced on the one hand over the relative success of an NDE development being measured against what is theoretically possible, and on the other that significant errors are possible if the fundamentals are not considered or fully understood.

Fig 1: Posterior Reliability Assessment of prescriptive ET Inspection, A-hat vs. a Behavior revealing an under-detection of the target flaw by a size factor of 4

Fig 2: A-hat vs. a behavior of probabilistically developed ET Inspection achieving the design target POD distribution

Review of reliability assessments performed over the last 20 years in aerospace suggests that elements of both approaches are required to make an accurate statement of NDE reliability. Figure 1 shows posterior analysis of surface eddy current reliability where the legacy criteria was limited to; detection of a single target size, and pass fail (a.k.a. hit/miss) decision data, coupled with a progressive lowering of the decision threshold to address newly revealed misses. This approach overlooked a poor physical relationship to eddy current physics, and failed to achieve the desired POD, as well as produce a large fraction of false calls. Redefining the inspection based on examination of the a-hat vs. a relationship, and linkage to the applicable eddycurrent physics, resulted in a very different, automated inspection with performance shown in figure 2. Accordingly there was significant improvement in the target flaw detectability, a lowering of false calls and adoption of the a-hat vs. a methodology[1] permitted introduction of the entire POD function into the risk assessment. The goal and outcome of the program was not to re-define the target based on an ideal/theoretical capability, but to define how well the current and improved process met the existing target. A familiar exchange in aerospace between NDE and the design process is; "what do you need to find", vs. "how small can you see". It has been found essential, as in the foregoing example, to know both answers in developing the optimum process with minimal loss. The definition of the next conceptual model component, Application Parameters, provides a bias towards the practical goal as the reliability target.

Application Parameters - The Alternative NDE Process Target

In the revised conceptual model of NDE reliability, the Application Parameters (AP) + Human Factors (HF) were predicted to be less than the Ideal Capability (IC). This is not a surprising conclusion, but in application parameters we see the interconnectedness to the ideal capability in the definition reached. Application parameters were defined as capability limited only by physical principles, but are interpreted in the context of the specification of expected flaws, components, materials, equipment and procedures. This is the more common manner of expressing NDE capability in aerospace, but as discussed above, it has inherent risks from a reduced emphasis on the inspection physics.

There is wide range of practical targets within the aerospace industry, with turbomachinery generally on the small end, and airframe structures on the large end of the flaw size of concern spectrum. The retirement for cause (RFC) approach examines military turbine components for defects on the order of.050" and less, and employs a high degree of rigor and automation to conduct and validate the reliability of the inspections. Commercial airframe structures are designed to be visually damage tolerant, i.e. tolerance of defects closer to 1 foot in length. Directed NDE of these structures, consisting of the conventional non-visual NDE methods for smaller flaws, represents less than 25% of total inspections performed[7], and performance parameters for these exams are of a prescriptive, deterministic nature. So, with such diversity of needs, it would seem appropriate that the reliability target flaw dimension be anchored in the context of the application. While this sounds reasonable, it is still essential to understand and measure the variance of the NDE process for each application, regardless of the size/condition of concern.

Application Parameters was subdivided into two sections for further clarification. Target Application Parameters were defined as the nominal of all parameters in the context of the application being studied. Performance/Full Application Parameters were defined as the total variance about the nominal. Again the main difference in this approach to defining the NDE reliability target, is that there would be a reduced examination of the underlying material and inspection physics and relationships. Despite these intuitive potential shortcomings, this bias towards the empirical result is the manner in which most reliability assessments are conducted, and will likely continue to be the primary approach until model based reliability approaches mature.

NDE Reliability Knowledge Acquisition & Modeling

Reliability demonstrations are complex and expensive, whether for large or small flaws. A common, but it is felt unnecessary cost to acquiring this knowledge is that sufficient details from previous demonstrations are often not retained, or otherwise known, such that there is a fair amount of "reinventing the wheel". While there was palpable opposition to creating a code or standard for the generation of NDE reliability information, the author believes that there is considerable value in some core guidelines and sponsoring of an NDE information infrastructure such as was baselined in the NTIAC database effort. Included in such an entity would be; libraries of cracked samples, common data acquisition and display formats, insitu NDE data on actual cracks, development of geometry and artificial to real flaw transfer functions, and a defined minimum set of assessment records to enable comparison/applicability evaluations. Enablers of these resources need to include liability protection for data on defective structure, and agreements on intellectual property rights. Many of the essential long term goals such as model based NDE reliability determination, and even near term goals of crack/notch transfer functions remain unrealized due in part to the lack of coherency among legacy efforts, communication/cooperation among various agencies and competitors.

The modeling approaches to understanding & predicting NDE behavior and resultant applied reliability continue to have great importance. Initiatives and programs such modular validation[8], and the FAA sponsored Engine Titanium Consortium, have strong modeling perspectives[9], and have materially contributed to lowering the both the costs of reliability knowledge, and the resultant inspections through, for example, reduced requirements for reference standards[2] and development of advanced signal processing tools.

Human Factors - Tolerance

The previously described range of target flaw sizes correlates to the presence/influence of human factors in NDE reliability. Human factors have been difficult to measure regardless of the industry of application. Where the perception is that there is a robust margin between the applied inspection and the target flaw/condition, the unknown variability of human involvement is accepted. The opposite, full automation is seen least in the robust design of fuselage, and most in the fractional inch target world of turbine engines. Human factors were defined as the physical and cognitive elements that impact performance. Blind trials are the most frequently used tool to assess human factors in aerospace NDE. Types of trials range from inserting defective parts into the manufacturing stream, to formal examinations of inspector performance. It has been observed that there is a qualitative relationship in performance demonstrations where the abilities demonstrated are retained over time, good or bad. It is clearly an improvement to accounting for human factors through movement from prescriptive to performance based NDE certification of inspectors.

Whole R Application Benefits for Commercial Aircraft Structural Inspections

It was reported by Thompson et. al.[10] that Boeing commercial aircraft inspection practice had accumulated a peak of 1085 NDE procedures, with an implemented 78x for the present fleet. Well in excess of 90% of these current procedures are implemented deterministically, as general "goodness" inspections, via qualitative lab trials and limited calibration controls vis a vis gas turbine or other hi-rel, & probabilistic NDE applications. As stated in (7), this qualitative approach has yielded a credible, yet imperfect safety record for the fleet. We are thus motivated, by fleet and fatigue test data feedback on structural health, as well as operational economics, to learn more precisely what contribution NDE has made to the reliability.

Fig 3: Comparison of deterministic estimate of performance approach with developed S/N model & blind trial results of 50 inspectors, on a service bulletin eddy current examination procedure for fuselage lap splices. Digital ET image at right depicts reflection differential probe presentation for unflawed lapsplice rivet hole [L], .2" cracked rivet hole [C], and unflawed tear strap hole [R]

Periodically, and typically motivated by a structural failure, an extensive look is made at a particular inspection to determine the implemented R of the inspection. As reported in (11) & depicted in fig 3, blind trial data for an eddy current critical lap splice inspection showed that initial estimates for crack detectability, defined as 63% probability of detection at the mean confidence level, varied considerably from the initial procedural estimate. In depth probabilistic analysis of the inspection process revealed several causes for the variance in estimate of detectability. Calibration practice did not incorporate response tracability to a master reference, introducing the variance contribution shown in fig 4.

Fig 4: Induced variation of calibration reference signal, for 3 exam frequencies (kHz), resulting from procedural tolerance and distribution of artifact (EDM notch) properties within specification limits

Variance in crack orientation, randomized probe performance, and manual orientation of the inspection, combined to produce a POD for the exam shown in fig. 3 which was over twice the detectable flaw size assumed for the most favorable flaw parameters during the initial procedure development. During the probabilistic assessment, a s/n based model estimator was derived which, as shown in fig 3, was sufficiently robust to capture the variances of field implementation, reported in 5, for a single flaw orientation at the desired level of significance. Work continues to capture the actual in service reliability for varying flaw orientation through; transfer function developments, application of automated ET imaging, morphological processing and automated distillation of inspection data. It can be clearly seen from this investigation that there can be a wide gap between perceived reliability resulting from best practice deterministic NDE, and a probabilistic approach that is focused on the implemented whole R function. This reliability determination process is one of continuous improvement advocated by Boeing and includes in depth follow on efforts to quantify and optimize high value NDE processes, as well as reduce or eliminate inspections which are shown not to add value to structural life prediction & control.

Conclusions

The foregoing discussions have attempted to describe the difficult task accomplished in separating the reliability process into discrete functions with clear domains. The resultant definitions are believed to have helped map out the workspace of future efforts in NDE reliability. It is clear that the importance of having reliability metrics for NDE processes is receiving increasing recognition within aerospace. Common approaches to the acquisition and analysis of reliability data will continue to help reduce the total effort required per assessment, and is essential in the long-term development and validation of reliability models. The ability to define and model the underlying inspection physics is also integral to being able to define NDE reliability accurately. Empirical reliability demonstrations continue to be expensive to conduct. Development of reliability guidelines and support for NDE information infrastructure will help reduce the costs of obtaining knowledge, and most importantly minimize the economic loss experienced across all industries resulting from incomplete knowledge of the NDE processes.

References

1. Alan P. Berens, "What Metric Should be Used to Measure Intrinsic Capability?", Proceedings of the 1999 American-European Workshop on Nondestructive Inspection Reliability, Boulder Colorado Tom
2. T. Taylor, Steve Doctor, "What is NDE Reliability Anyway?", Proceedings of the 1999 American European Workshop on Nondestructive Inspection Reliability, Boulder Colorado
3. V. Panhuise, "Introduction - Quantitative Nondestructive Evaluation", Vol. 17, 9^th Edition, Metals Handbook
4. R.H. Burkel, D.J. Sturges, R.S. Gilmore, "Effective Reflectivity: A POD Methodology for Ultrasonic Inspection", ASNT Fall Conference, 1995
5. F.W. Spencer, "Interlayer Crack Detection", Aerospace Engineering, April, 1999.
6. R.E. DeVor, T. Chang, "Statistical Quality Design and Control", Vol. 17, 9^th Edition, Metals Handbook
7. Ulf G. Goranson, "Damage Tolerance Facts and Fiction", Proceedings 17^th Symposium of the International Committee on Aeronautical Fatigue
8. C. Nockemann et. al., "Modular Validation", European-American Workshop, Determination of Reliability and Validation Methods of NDE, Berlin Proceedings, 1997
9. L.W. Schmerr, D.O. Thompson, "The role of modelling in NDE Standards", European-American Workshop, Determination of Reliability and Validation Methods of NDE, Berlin Proceedings, 1997.
10. J. Thompson, J. Kollgaard, B. Pearce, "Inspection Challenges for Aging Commercial Aircraft" ASNT Fall Conference, Phoenix AZ, 1999
11. F. Spencer, "Field NDI Reliability Study Designs to Incorporate Human Factor Issues", SANDIA National Labs Technical Report, Draft

© AIPnD , created by NDT.net

|Home|    |Top|