Interlayer Crack Detection Results Using Sliding Probe Eddy Current Procedures

David G. Moore and Floyd W. Spencer
Sandia National Laboratories[Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under Contract DE-ACO4-94-AL85000.], P.O. Box 5800 MS-0615
Albuquerque, New Mexico 87185 U.S.A.
Telephone (505) 844-7095 E-mail dgmoore@sandia.gov.
Contact

Abstract

The Federal Aviation Administration's Airworthiness Assurance NDI Validation Center (AANC) at Sandia National Laboratories in Albuquerque, New Mexico is currently completing a study pertaining to the detection of cracks in multi-layered aluminum sheets. Each specimen models pertinent aspects of the Boeing 737 aircraft lap splice joint, Line Numbers 292 - 2565. This paper discusses specimen characterization, steps taken to make the specimens useful for nondestructive technology assessment and the protocols used to conduct the reliability assessment experiments. Results from fifty-six (56) independent inspections taken at eight (8) different airline facilities using the existing eddy current sliding probe procedures are presented. The inspectors performed this experiment without knowing which fastener sites contained fatigue cracks. The inspection results are shown in the form of 4-parameter probability of detection curves. The influencing factors on inspector reliability are identified and discussed.

Keywords:eddy current, probability of detection, aircraft inspection, statistics, fatigue cracks

Introduction

The Federal Aviation Administration's Airport and Aircraft Safety Research and Development Division (AAR-400), located at the William J Hughes Technical Center in Atlantic City, New Jersey, sponsored this program. Details of the program, including background and experimental protocols, have been presented elsewhere (Moore). Briefly, the inner-layer crack panels were configured to simulate pertinent aspects of an in-service Boeing 737 lap splice. Each test specimen consisted of one mock doubler, four mock tear straps and two aluminum skins riveted together with a 76.2-mm (3-inch) overlap that included the doubler. Programmed fatigue cracks were grown in aluminum sheets and placed in the lower skin requiring inspections through either 1.83-mm (0.072 inch) or 2.03-mm (0.080 inch) of combined upper skin and doubler material.

The spacing of the tear straps was varied to simulate manufacturing tolerances. Since the tear straps create a major source of noise for the eddy current inspection, the number of tear straps per unit length of lap splice was also doubled. Thus, tear straps were present in 40% of the total of 340 inspection sites contained in 17 test panels. This gives an inspector a larger number of tear strap inspection sites with a minimum number of test specimens.

The programmed fatigue cracks were created from starter notches at select locations. The lower skin panels were then cycled until the desired length of fatigue crack was reached. The starter notches were removed by drilling the final holes for fastener installation. Individual eddy current signals were verified to simulate signals from an aged aircraft. Seventeen panels, each containing 20 rivet inspection sites, were presented in a random order to each of the participating inspectors. Inspectors used their own equipment and were expected to follow the Boeing Procedure 53-30-11. One panel, containing no cracks, was presented a second time to each inspector without their knowledge that it was a repeat inspection. Thus, each inspector performed inspections at a total of 360 rivet sites. The experiment monitor recorded calls made, equipment used, setup values, and observations concerning inspection technique.

Test Specimen Characteristics and Equipment Used

Table 1 shows the distribution of rivets to be inspected among the specimen characteristics of thin versus thick skins and tear strap presence or absence. The numbers for flaws is the number of rivet sites containing cracks. The cracks were of varying lengths and some of the rivet locations had cracks emanating from both sides of the rivet hole. Figure 1 shows the distribution of crack lengths among the various conditions of skin thickness and tear strap presence.

	Thin Skins 1.83-mm (0.072-inch)		Thick Skin 2.03-mm (0.080-inch)		Total
	Unflawed	Rivets w/ Flaws	Unflawed	Rivets w/ Flaws
No Tear Strap	74	22	99^*	21	216^#
Tear Strap	37	27	52⁺	28	144^#
Total	111	49	151^#	49	360^#
Table 1: Distribution of rivets among tear straps and skin thickness.

* Includes 12 sites on one panel inspected twice. + Includes 8 sites on one panel inspected twice. # Includes one panel inspected twice.

Fig 1: Crack Length Distribution by Tear Strap (TS) and Skin Thickness.

In accessing the performance of inspections following the Boeing Service Bulletin 53-30-11, it was intended that the breadth of inspector backgrounds and equipment being used be reflected in the data. To this end, eight different facilities were visited and the inspectors at those facilities used inspection equipment available to them. Table 2 lists the names of the facilities visited and the equipment used.

Facilities Visited	Equipment Used	Probe and Reference Standard
Tramco (B.F. Goodrich) Boeing Delta Airlines Aloha Airlines United Airlines American Airlines Northwest Airlines First Air	Instruments (# of times used) Nortec NDT 19 (8) 19e (11) Nortec NDT 19eII (5) NDT 24 (19) Nortec 2000 (4) Hocking P2200 (6) Zetec 21A (3)	SPO 3806 - Several serial numbers Boeing NDT 3004 - Several vendors as well as company made
Table 2: Facilities and Equipment Used in Experimentation.

Outline of the Steps in the Boeing 53-30-11 Procedure:

SET-UP
Adjust the balance point to lower right of the screen. Adjust the phase control so that lift-off moves toward the left. Adjust the probe guide so that it will keep the centerline of the probe aligned with the center of the fastener row. Adjust the vertical and horizontal gains so that the reference notch signal is at 80 percent of the display and the signals from the good fastener holes are approximately 60 percent.

INSPECTION
Align the probe on the lap-joint so that the center line of the probe will follow the centerline +/- 1.27-mm(0.050-inch) of the lower row of fasteners. Note: you can use a probe guide to help keep the centerline of the probe aligned with the center of the fastener. Move the probe slowly along the fasteners and monitor the instrument display at the same time. Make sure the centerline of the probe will move across the center of the fasteners. Make a mark at the locations that cause signals that are at or above the reject level. Mark any locations where the opening in the signal loop is more than 15 percent of the display height.

RESULTS
Signals that are more than 70 percent of the display height or that have openings in the signal that are 15 percent or more of the display height are signs of cracks. Note: An edge of a tear strap that is too close to a fastener can cause a signal with an opening in the loop that looks almost the same as a crack signal. But, the signal will have a display height that is lower than the signals from the areas without a tear strap. To identify if a signal is the result of a tear strap or a crack, do an inspection of a fastener in the top row of the lap joint. The fastener in the top row must be the same size and material, and be aligned vertically with the lower fastener. If the signals are the same, the signal is caused by a tear strap.

Analysis and Inspector Results Summary

Standard presentation of Probability of Detection curves estimated from hit/miss data employ binary regression techniques using either a probit (normal) link function or a logistic regression function (Berens). The early indications from the data gathered for this program indicated an inadequacy of the usual two-parameter models to fit the data. Extensions to a 4 - parameter model were explored and have been reported elsewhere (Spencer, Moore). Briefly we argue that both hits and misses can be, and often are, made for reasons that are independent of crack length. A generalized PoD curve model that captures this behavior is given as follows:

PoD(a) = a + (b - a)•F(a;m,s),

(1)

where: a(³0) is a lower asymptote,b(£1) is an upper asymptote, and the F(•;m,s) function is a distribution function modeled with two parameters. In identifying sources fro reliability analysis of field inspection we equated a to a "false call" rate and b to 1 minus a random "miss rate." All the parameters can be estimated by the maximum likelihood method (Spencer). In the data presented here for the 56 inspections we present, for comparison, both a usual 2- parameter model, as well as a 4- parameter generalization. The following form of equation (1) gives the 4-parameter model:

a+(b-a)F((ln a - m)/s),

where F is the standard normal distribution function, ln is the natural logarithm function, and a is crack length. The 2-parameter model assumes that a= 0 and b=1.

Table 3 summarizes inspection results from the 56 independent inspections. The inspections in table 3 are ordered by increasing estimates of the crack length for a probability of 0.90 detection that would be obtained using the 2-parameter model. The 2-parameter probability of detection (PoD) curves are also shown in figure 2. The table of results, as well as the PoD curves, shows substantial variation from inspector to inspector. It is apparent in comparing the various categories of table 1 that there are some fundamental differences in the inspections. Consider inspection 1, with the lowest value for ā₉₀, the estimate of the 90 % detection crack length. There were relatively few detections (13), with no false calls. However, the detections were consistent with respect to flaw size. The result is an estimated PoD curve that shows a quick transition in size from the non-detected to the detected flaws. On the other hand, consider inspection 44 with 36 detections. The detections are spread across a wider range of flaw sizes and the inspection also resulted in a substantial number (34) of false calls indicating a fundamentally different inspection than that of inspection 1.

	Cracks < 0.1 inch		Cracks > 0.1 inch		Total	False	ā₉₀	Inspt	Cracks < 0.1 inch		Cracks > 0.1 inch		Total	False	ā₉₀
Inspt	Detects	Misses	Detects	Misses	Detects	Calls*	(mm)	Inspt	Detects	Misses	Detects	Misses	Detects	Calls*	(inch)
1	0	63	13	20	13	0	4.39	29	0	63	7	27	7	0	7.72
2	6	58	17	16	23	8	4.72	30	3	61	12	22	15	0	7.80
3	7	56	22	11	29	7	4.90	31	2	62	9	25	11	2	7.80
4	4	59	16	18	20	3	4.95	32	3	60	16	17	19	7	8.10
5	6	56	21	12	27	7	5.00	33	0	64	8	25	8	0	8.20
6	1	63	16	17	17	10	5.23	34	4	60	15	19	19	6	8.46
7	0	64	10	23	10	0	5.23	35	3	61	16	18	19	4	8.99
8	4	60	18	16	22	3	5.23	36	4	60	15	18	19	7	9.30
9	3	60	16	17	19	13	5.26	37	7	56	19	15	26	25	9.37
10	1	62	12	22	13	0	5.26	38	6	57	16	17	22	7	9.50
11	2	61	13	21	15	2	5.31	39	4	59	13	21	17	8	10.39
12	2	60	16	18	18	1	5.38	40	3	60	13	20	16	0	10.46
13	1	63	13	20	14	4	5.41	41	13	50	17	16	30	26	10.80
14	3	61	16	18	19	2	5.46	42	2	62	8	26	10	1	11.38
15	10	54	20	14	30	6	5.54	43	6	57	15	18	21	7	11.51
16	6	57	19	15	25	12	5.59	44	16	47	20	14	36	34	11.76
17	7	57	19	15	26	8	5.59	45	4	59	11	23	15	9	12.37
18	0	63	10	23	10	2	5.64	46	7	57	15	19	22	7	13.06
19	0	63	10	23	10	2	5.87	47	1	63	6	28	7	1	15.04
20	9	54	18	15	27	4	5.97	48	4	60	14	19	18	7	15.47
21	1	63	12	22	13	4	6.15	49	2	62	11	23	13	1	15.52
22	1	62	10	23	11	0	6.50	50	15	47	16	17	31	30	18.19
23	3	59	14	19	17	7	6.60	51	2	62	8	25	10	0	19.02
24	4	60	14	20	18	2	6.68	52	5	58	8	25	13	5	19.69
25	6	57	16	17	22	11	6.93	53	4	60	10	24	14	3	20.45
26	3	61	14	20	17	2	7.16	54	20	42	19	13	39	52	21.26
27	1	63	11	23	12	2	7.26	55	4	60	5	29	9	0	38.79
28	0	64	9	25	9	2	7.47	56	8	56	12	22	20	16	40.08
Table 3: summary of 56 sliding probe inspections

*The number of unflawed rivets inspected varied between 255 to 262

Although the estimated 90 percent detection crack (ā₉₀) exceeds 6.05-mm (0.256-in) for more than half of the inspections, none of the inspectors missed the two largest cracks, which are 6.05-mm (0.256-in) and 10.6-mm (0.584-in) in length. The detections among very small flaws cause variation in the estimated PoD curve that increases the estimate of the 90 percent detection crack (ā₉₀). The 4-parameter model enables the upper end of the curve to be separated from the detections made on arbitrarily small cracks. In Figure 3 the PoD curves estimated from a 4-parameter generalization are shown and in Table 4 the resulting estimates of ā₉₀ are compared to those obtained in the two-parameter fits. In all cases of fitting the 4-parameter model, b=1. This reflects the fact that all inspectors were finding the sites with the largest cracks.

In Table 4 the false call rates for each inspection are shown as a comparison to the estimated a from the 4-parameter model. The estimated a values are, in general, larger than the observed false call rates, but there is a positive correlation. That is, as the false call rate increases the estimated a also increases.

In Table 5 the false call rates calculated from all 56 inspections are given along with the number of calls and the number of inspections. Note that inspectors characterized some of the rivets as not being inspectable. These sites are not reflected in the counts. From Table 5 it is seen that the tear strap locations had a higher rate of false calls in general, than did the non-tear strap locations. Moreover, the difference is quite substantial on the thick skins.

Inspt	2-par	4-par	alpha	False	Inspt	2-par	4-par	alpha	False
Inspt	ā₉₀	ā₉₀	alpha	Calls*	Inspt	ā₉₀	ā₉₀	alpha	Calls*
1	4.39	4.39	0	0.000	29	7.72	7.72	0	0.000
2	4.72	4.72	0	0.031	30	7.80	7.80	0	0.000
3	4.90	4.34	0.026	0.027	31	7.80	5.28	0.026	0.008
4	4.95	4.95	0	0.012	32	8.10	4.67	0.037	0.027
5	5.00	4.29	0.025	0.027	33	8.20	8.20	0	0.000
6	5.23	5.23	0	0.038	34	8.46	4.17	0.093	0.023
7	5.23	5.23	0	0.000	35	8.99	5.59	0.027	0.015
8	5.23	5.23	0	0.012	36	9.30	6.02	0.026	0.027
9	5.26	5.26	0	0.050	37	9.37	3.76	0.13	0.097
10	5.26	5.26	0	0.000	38	9.50	4.72	0.083	0.027
11	5.31	5.31	0	0.008	39	10.39	5.84	0.043	0.031
12	5.38	5.38	0	0.004	40	10.46	5.99	0.025	0.000
13	5.41	5.41	0	0.015	41*	10.80	6.12	0.115	0.101
14	5.46	5.46	0	0.008	42	11.38	11.38	0	0.004
15	5.54	4.45	0.032	0.023	43	11.51	4.24	0.093	0.027
16	5.59	5.59	0	0.046	44	11.76	3.56	0.282	0.132
17	5.59	4.88	0.026	0.031	45	12.37	12.37	0	0.035
18	5.64	5.64	0	0.008	46	13.06	6.93	0.053	0.027
19	5.87	5.87	0	0.008	47	15.04	15.04	0	0.004
20	5.97	5.97	0	0.016	48	15.47	6.12	0.066	0.027
21	6.15	6.15	0	0.015	49	15.52	7.34	0.026	0.004
22	6.50	6.50	0	0.000	50	18.19	3.99	0.231	0.115
23	6.60	6.60	0	0.027	51	19.02	5.28	0.028	0.000
24	6.68	7.01	0	0.008	52	19.69	4.39	0.088	0.019
25	6.93	6.93	0	0.042	53	20.45	8.46	0.039	0.011
26	7.16	7.16	0	0.008	54	21.26	5.89	0.26	0.203
27	7.26	5.82	0.017	0.008	55	38.79	6.32	0.047	0.000
28	7.47	7.47	0	0.008	56	40.08	5.59	0.121	0.061
Table 4: comparison of 2-parameter and 4-parameter fits.

*Two local maximum likelihood values that differ by very little. The value shown has alpha estimated that is consistent with the observed false call rate. Another local maximum occurs with alpha=0.222 and ā₉₀=4.11-mm (0.162-in). The bolded rows are those inspections where, although it was indicated that the added alpha parameter increased likelihood, the gains in the likelihood equation were not significant (0.05 level) with the addition of the added parameter.

	Thin Skin	Thick Skin	Total
No Tear Strap	96/4072 0.0236	129/5538 0.0233	225/9610 0.0234
Tear Strap	51/2029 0.0251	113/2911 0.0388	164/4940 0.0332
Total	147/6050 0.0243	242/8320 0.0291	389/14550 0.0267
Table 5: Rate of calls on unflawed rivets among tear straps and skin thickness.

Fig 2: PROBABILITY OF DETECTION CURVES FIT TO INDIVIDUAL INSPECTIONS - 2 PARAMETER LOGNORMAL.

Fig 3: PROBABILITY OF DETECTION CURVES FIT TO INDIVIDUAL INSPECTIONS - 4 PARAMETER GENERALIZED LOGNORMAL.

Conclusions and Observations

The data gathered from this experiment clearly show that a traditional 2-parameter binary regression model provides an inadequate fit to many of the individual inspection results. In many cases, those models would indicate that a 90 percent detection crack length would be estimated as being larger than a set of the largest cracks in the test specimen, even though these large cracks were all detected. We have introduced the fitting of a 4-parameter model to capture inspection behaviors that result in hits at smaller cracks that are independent of crack length and misses at large cracks that are independent of crack length. Thirty-two percent (18 of the 56) of the individual inspections data were better fit at a statistical significance level of 0.05 by the addition of a parameter allowing for hits independent of crack size. An additional 16 percent (9 of 56) were better fit by the expanded model, but the better fit was not statistically significant.
As is often the case, there was substantial inspector-to-inspector variation in this experiment. This variation indicates that fundamentally different inspections are being carried out from one inspector to another.
For the interlayer inspection, many of the inspectors failed to follow the Boeing procedures. Only a few of the inspectors implemented the checks that were included in the procedures to give a better inspection in the presence of tear straps. Also, many of the inspectors only used an absolute threshold for the rise of the signal, and did not include calls based on signal loop width, as called for in the procedures.
Inspectors, for the most part, did not use a non-conductive straightedge to aid in keeping the probe centered over the rivets. The nature of the experiment did not allow a direct measurement of how much, but being off-center at individual rivets was an oft-observed phenomenon.
The data show that the factor of tear strap presence or absence and the factor of total skin thickness do have an effect on the outcome.

Acknowledgments

This work was supported by the FAA William J. Hughes Technical Center, Atlantic City International Airport, New Jersey, USA. The Airworthiness Assurance NDI Validation Center (AANC) is operated and maintained for the FAA at the Albuquerque International Airport by Sandia National Laboratories under Interagency Agreement DTFA03-95-X-90002. This is a work of the United States Federal Government and is not subject to copyright. The United States Government retains the nonexclusive right to reproduce this work.

References

Berens A P, et. al., (1988) A User's Manual for Probability of Detection Software System (POD/SS), UDR-TR-88-12, UDRI, Dayton, Ohio. USA
Moore D G and Spencer F W, (1998) Detection Reliability Study for Interlayer Cracks, Airframe/Engine Maintenance & Repair Conference Society of Automotive Engineers, Inc.
Spencer F W, (1998) Identifying Sources of Variation for Reliability Analysis of Field Inspections, RTO Workshop - Airframe Inspection Reliability Under Field/Depot Conditions, Brussels, RTO-MP-10, RTO/NATO.
Spencer F W, (1998) Fitting Probability of Detection Curves to Single Inspector Hit/Miss Data, ATA NDT Forum, Albuquerque, New Mexico, USA