TABLE OF CONTENTS

Abstract Enhanced operator training and assessment Measurement of Operator Performance Feedback of Operator Performance Control of Training and Testing Development Cost Reduction and Avoidance Overview of the VNDE Principal Components of the VNDE Practice Mode Performance Testing Mode Scenario Development Mode Potential VNDE applications Existing NDE Training Screening of Candidates for NDE Qualification Augmenting of Qualification Testing Preperation and Qualification for Site-Specific Conditions and Tasks References

Abstract

The Virtual Nondestructive Evaluation system (VNDE) provides electronically created substitutes for the real spaces, objects, and functions required for NDE examinations, ie, the computer and mouse substitute for the ultrasonic instrument and transducer, respectively. As a consequence of its operating in a synthetic environment, the VNDE provides capabilities for training and performance testing that may not be possible with real systems in either laboratory or field settings. For example, the capability for enhanced data capture provides the basis for more precise performance measurement, more timely feedback on operator proficiency, and more complete diagnosis of operator strengths and weaknesses. This paper describes the background and development of the VNDE, and introduces the three modes in which the system operates--practice, performance test, and scenario generation.
• Practice Mode provides for operator practice in examining each of a set of virtual pipe-weld specimens and for presenting feedback on operator performance. The term scenario refers to a specific practice specimen and all the material and performance characteristics associated with that specimen. For example, a scenario might consist of a pipe-weld of 12-inch diameter, 1-inch thickness stainless pipe containing a single axial flaw on the upstream side of the weld for which the 45° and 60° shear wave examination techniques are appropriate.
• Performance Test Mode provides for selecting a set of scenarios and assembling them into a performance test. The performance test may then be automatically administered and scored to assess practical operator proficiency.
• Scenario Generation Mode provides data (including a library of waveforms obtained from actual reflectors--flaws, counterbores, weld roots) and tools for use in the development of new ultrasonic examination scenarios. While the waveforms and tools are relatively easy to use, the development of scenarios for use in training and testing requires the knowledge and perspective of experts in the application of ultrasonic NDE techniques.

Enhanced operator training and assessment

The VNDE is designed to complement existing approaches to the training and assessment of NDE operators by providing capabilities that do not now exist and by enhancing those that do exist, as described below. It also provides a way for ultrasonic examiners to economically meet the annual training requirements of ASME Section XI, Appendix VII, "Qualification of NDE Personnel for Ultrasonic Examination". Ultimately operators must demonstrate their proficiency by examining real specimens with real NDE techniques and equipment. However, using a combination of virtual and real systems might both enhance the effectiveness and reduce the costs of operator training and assessment.

Comprehensive Measurement of Operator Performance
Current approaches to the measurement of NDE performance during practice or demonstration testing provide a limited picture of an operator's proficiency. Measures of operator performance in assessing the condition of a component are now typically limited to the percentage of reportable indications correctly reported, and to the percentage of indications falsely reported. These measures are appropriate and useful, but they provide only limited information about the results of the examination and little information about the adequacy of the process employed by the operator in conducting the examination. From current measures we can learn about an operator's overall level of performance, but we can learn very little about what specifically led to this level of performance, or what the operator should do in the future to sustain good performance or to improve poor performance. For this we need detailed information about the differences between the conditions reported by the operator and the true conditions of components examined, and we need specific information about any inadequacies of the processes employed by the operator in conducting the NDE examination.

Previous research has revealed that in conducting NDE examinations the cognitive processes of the operator, in particular, can be critical to the attainment of satisfactory performance (Harris, 1990; Harris & McCloskey, 1990). Cognitive processes involve mental functions like processing information, applying logic, making discriminations, and reaching conclusions and decisions. For example, the presence or absence of certain critical processes were shown to account for the difference between an examination success rate of about 80% (when the processes were present) and a success rate of only about 20% (when these processes were absent). These critical processes are not being measured now because none of the tools or techniques currently used in NDE practice or testing are capable of capturing this type of information. The VNDE, with its enhanced capability for data capture, has the capability for automatically measuring both detailed results and the specific process characteristics of an operator's NDE examination.

Detailed and Timely Feedback of Operator Performance
With the capability described above, the VNDE can enhance the learning process by providing more effective feedback on operator performance. Detailed and timely feedback, one of the essential components in skills learning, can assure that higher levels of performance can be attained faster, and with greater long-term retention, than is possible under current methods alone. Conducting practice sessions in a virtual environment provides opportunities to obtain multiple measures of operator performance, to use these measures to analyze operator strengths and weaknesses, and to provide detailed, timely, meaningful feedback to the operator. Moreover, a diagnostic profile can be developed to serve as the basis for performance-based training and practice, emphasizing what is required to remedy deficiencies while not spending unnecessary effort on those aspects of the examination process that have been mastered. Currently, operators cannot learn why they performed well or performed poorly, because the required measures are not available.

Control of Training and Testing Development
Conducting NDE training exercises and performance testing in a virtual environment provides opportunities for greater control over the training and testing development process. Obtaining real specimens from defective components that have been, or need to be, removed from operating nuclear power plants, or obtaining them from manufacturing procedures in which specified conditions (defects) are artificially inserted into components, can be a lengthy and uncertain process. The NDE signal characteristics of indications obtainable from the resulting specimens cannot be accurately predicted from the process that produced the specimens and, as a consequence, must be determined by applying NDE techniques after the specimens are obtained. The result is that even relatively large specimen sets might not have the desired distribution of physical characteristics, signal characteristics, and levels of difficulty needed for the development of an ideal set of training exercises and performance tests unless the specimens were designed and analyzed adequately.

A virtual system can provide greater control over exercise and test development by providing an extensive catalog of real signals associated with specific NDE techniques (ultrasonic 45° shear wave, ultrasonic 60° shear wave, ultrasonic 30-70-70 mode-conversion, etc.), specific types of reflectors (axial crack, circumferential crack, counterbore, weld root, etc.), documented component conditions (pipe diameter, pipe wall thickness, variations in pipe wall thickness, pipe material, etc.) and tools with which to select and modify signal characteristics (amplitude, signal-to-noise ratio, continuity, etc.). An NDE expert or instructor can have at his or her fingertips a nearly unlimited number of possibilities for the construction of virtual NDE examinations for use in practice or performance testing. For example:

• reflectors of different types can be located, where desired, in specimens of any selected physical characteristics;
• the reflectors can produce indications of any desired signal characteristics with the application of any specified techniques; and
• these combinations can be employed to produce examinations at any desired level of difficulty.

Cost Reduction and Avoidance
Conducting practice and testing with actual specimens such as pipe welds, sections of reactor pressure vessels, and nozzles can be an expensive business. Assembling an adequate sample of specimens from components cut out of nuclear power plants or from specimens manufactured expressly for testing purposes can cost many millions of dollars. Since some of the components can be very large, the storage and handling of specimens require a relatively large facility with special handling equipment. If the specimens are from an operational nuclear plant, they may require a facility with restricted access, special handling requirements and health physics procedures. Examining these specimens and documenting their actual conditions requires special equipment and consumes significant amounts of the time of highly skilled technical personnel. And, of course, the administration of practice and testing sessions, the acquisition of data and the scoring of performance measures, and the storage and safe-keeping of records, also requires the time of skilled personnel.

Methods employing synthetic environments will not eliminate the need for the types of activities described above. Conducting NDE examinations with real components will, necessarily, remain an important part of future NDE operator training and qualification. Furthermore, the use of synthetic environments will also depend, to some degree, on the acquisition of data from real components for which the true conditions must also be determined. However, as proven in aviation training and assessment and elsewhere, an optimal combination of virtual and real environments will probably prove to be the most beneficial. By using a combination of real and virtual examinations for NDE training and qualification, the extent of specimen acquisition, documentation, storage, and handling activities and their associated costs might be reduced significantly. For example, as discussed earlier, cost savings can be realized by facilitating the development of practice and test examinations, and by the automatic administration, scoring, and records management associated with virtual examinations.

Overview of the VNDE

The VNDE, operating with a mouse control on a desktop or laptop computer running Windows 95, provides a virtual environment in which operators can practice ultrasonic examinations and be tested on their examination capabilities. In addition, the system provides tools which can be used by NDE instructors and test specialists to develop practice examinations and performance tests to meet specific instructional and testing objectives and requirements. The initial version of the system provides a virtual environment for conducting ultrasonic examinations to detect and report the presence of intergranular stress-corrosion cracking (IGSCC) in pipe welds. Other components can be included such as vessel shell welds and dissimilar metal welds As noted earlier, the ultrasonic VNDE provides for three program modes: practice, performance testing, and scenario development.

Principal Components of the VNDE
The layout of the ultrasonic VNDE on the computer monitor is shown in Figure 1. The system is shown after a scenario (Practice 1) has been loaded and started, and after the operator has scanned the pipe and reported locations of the weld root, a counterbore (on the downstream side of the weld), and a circumferential flaw (on the upstream side of the weld). The principal components of the system are:

Fig 1: Layout of the ultrasonic VNDE on the computer monitor.

Main Menu: Provides the options of either loading a scenario or exiting from the system.

Scoring Menu: Provides options for obtaining scores based on scenario performance.

Program Mode: Practice or Performance Test mode may be selected from the Program Mode box at the upper right. Scenario Generation mode may be selected when in Performance Test mode.

Scenario: The identification and description of the ultrasonic examination scenario is provided in the scenario description box at the upper left-hand of the screen.

A-Scan Display: The A-scan display for ultrasonic waveform presentation is below the scenario description box.

Position Box: Provides the transducer position in the pipe-weld scan area (in inches X and Y) that is associated with the displayed waveform.

Pipe-Weld Area: The pipe-weld area to be examined is represented by the large area in the right-hand part of the screen. The area represents a 5-inch circumferential segment of the pipe along the weld and a 5-inch axial segment of the pipe along the flow. The shaded area represents the weld crown.

Key: The color coded key can be turned on or off by the operator in the practice mode. This scenario contains crack, weld root and counterbore signals, starting from the upstream side of the weld.

Controls: VNDE controls are located below the A-scan display.

Practice Mode
In Practice Mode an operator can practice conducting ultrasonic examinations under a variety of different conditions. For example, the operator might wish to practice the application of the 45° shear-wave, the 60° shear-wave, and the 30-70-70 techniques in examining 12-inch diameter, 1-inch wall thickness, austenitic stainless steel piping for IGSCC. From a list of examination scenarios meeting these requirements, the operator can select one or more on which to practice. During the examination, the operator can:

• switch among the available techniques, conduct axial or circumferential scanning, oscillate the transducer, change the gain, and/or select scanning area and transducer orientation,
• report and display the location of any flaw, counterbore, and/or weld root indications, and
• overlay a template to enable a comparison of reported indications with the true conditions of the pipe (for example, overlay the actual flaw location so that it can be compared with the reported flaw location).

Performance Testing Mode
The Performance Testing Mode is designed for use in assessing operator proficiency in conducting ultrasonic examinations. To prepare the test, an NDE testing specialist selects an appropriate sample of examination scenarios from the scenario library and compiles them into a performance test of the desired length. If a sufficient sample of scenarios is not available to meet the specific objectives of the performance test, the test specialist may use the tools provided by the system to develop new scenarios. (Scenario development will be described in the next paragraph.) Performance testing consists of:

• presenting the operator with a set of virtual specimens for which to complete ultrasonic examinations and to report the results obtained, within the time allotted,
• measuring operator performance in terms of examination effectiveness, application of procedures, and operator information processing, and
• providing an operator performance profile, detailing operator strengths and weaknesses as reflected by the performance measures.

Scenario Development Mode
The system provides tools for the instructor or test specialist, in the Scenario Development Mode, to design and construct examination scenarios for use in practice or performance testing. The system contains an expandable set of signal waveforms, obtained from actual ultrasonic examinations, that can be employed for this purpose. The waveforms reside in a system library which can be searched to obtain subsets of waveforms that meet specific scenario development requirements. For example, the library might be searched to obtain all available waveforms obtained from application of the 60° technique to flaw indications, or all waveforms obtained by application of the 45° technique to counterbore indications. From the waveforms thus identified, specific ones could then be selected, as appropriate, for use in scenario development. The waveforms obtained from the library might be further modified for practice or testing purposes by changing certain of the characteristics of the signal, such as the amplitude or signal-to-noise ratio. The examination scenario is then developed by applying the selected waveforms to specific locations of the pipe-weld area. That is, the scenario developer can place indications from flaws, counterbores, weld roots, and so on, in any desired locations in the material to be examined. During scanning of the material, later, the transducer will return the appropriate signals at transducer locations that correspond to the orientation of the transducer, the nature of the technique, the geometry of the component, and the physics of sound propagation in the material.

Potential VNDE applications

Several principal applications to operator training and assessment are anticipated for the VNDE in light of current needs of the industry. These are summarized in the paragraphs that follow.

Augmentation of Existing NDE Training
The VNDE can serve as a component of either traditional classroom-laboratory training or the newer computer-based, multi-media training programs--training that emphasizes the development of skills required for the application of NDE techniques. The VNDE can help meet the objectives of these programs by providing opportunities for practice and performance testing. Operators will be able to enhance and reinforce the training received by practicing on virtual NDE examinations and by receiving immediate, detailed feedback on their performance. Operators will also be able to assess their capabilities through performance testing and the diagnosis of performance strengths and weaknesses constructed from testing results.

The availability of this type of feedback will help focus operators on what specifically they need to do to increase their level of proficiency. Moreover, practice sessions and testing can be designed to move from lower to higher levels of difficulty, so that each operator can advance at his or her own pace in developing NDE capabilities and the confidence associated with their application. For example, the earlier practice sessions could emphasize classic (textbook) procedures and signal characteristics. After these are mastered, later examinations could include the complexities and subtleties closer to those found in typical field examinations.

Screening of Candidates for NDE Qualification Programs
Administrators of NDE qualification programs report that the relatively low passing rates experienced by these programs is due, at least in part, to the inadequate preparation of candidates. Many candidates for qualification reportedly show up without the requisite training and experience required for them to pass the qualification tests. As a consequence, added program costs are incurred through the testing and failing of low-potential candidates while, at the same time, opportunities are being denied to candidates with greater potential. The VNDE offers the means for objectively screening candidates for NDE qualification. A virtual performance test could be developed and administered, at the home facility of the candidate, to assess the potential of the candidate for qualification. The standard for passing the virtual performance test could be set sufficiently low so that a person failing the test would have very little chance of passing qualification testing. In this way, the prescreening would very seldom deny the opportunity of demonstrating NDE qualifications to an individual who has them.

Augmentation of Qualification Testing
As discussed earlier, current approaches to the measurement of NDE performance during qualification testing do provide only a limited picture of an operator's capabilities. Detection performance measures, for example, are typically limited to calculating the percentage of reportable indications correctly reported and the percentage of indications falsely reported. These measures provide information about the overall results of the examination but little information about the details of the results or the adequacy of the process employed by the operator to conduct the examinations. A principal reason is that process information is difficult and/or burdensome to obtain during either laboratory of field NDE examinations of actual materials or components. Virtual examinations, on the other hand, provide opportunities to measure how well the operator has applied NDE techniques and procedures, how effective an operator has processed information, and detailed information about the examination results. Consequently, virtual examinations might be employed to augment real examinations so that a complete assessment of an operator's capabilities can be obtained.

Preparation and Qualification for Site-Specific Conditions and Tasks
The VNDE can facilitate the preparation of operators for the specific types of jobs and tasks they are to perform at specific sites. Scenarios representative of the types of conditions they are likely to face and the types of defects they are likely to encounter can be provided for both demonstration and operator practice. Moreover, if required, the VNDE can serve as the means for administering performance tests to operators to assure that they have the capabilities required for the performance of specific types of tasks under the conditions that exist at the site. With the flexibility of the VNDE, on-site task preparation and qualification testing could actually be completed prior to the operators arriving at the site. Scenarios could be generated by experts knowledgeable of site conditions and tasks and sent to vendor locations in advance of the scheduled work.

References

1. Harris, D. H., and McCloskey, B. P. Cognitive Correlates of Ultrasonic Inspection. Technical Report EPRI NP-6675, Electric Power Research Institute, January 1990.
2. Harris, D. H., Effect of Human Information Processing on the Ultrasonic Detection of Intergranular Stress-Corrosion Cracking. Materials Evaluation, Vol. 48, 1990, 475-480.
3. ASME. 1995"Section XI, Rules for Inservice Inspection of Nuclear Power Plant Components", ASME Boiler and Pressure Vessel Code, 1995 Edition, American Society of Mechanical Engineers, New York.