Ethics and Best-Case Analysis
In "Erratic Measure," (Republished in the May issue of NDTnet) Ronald Christensen and I point out that the flaw detection methods of today violate the Unit Measure axiom of probability theory. Ethical questions follow from this, for the violations invalidate value-free methods for estimating the probabilities of error, leaving only value-laden methods.
In designing things, engineers often employ the method of analysis which is worst, from the standpoint of cost but best, from the standpoint of safety. This is called "worst-case analysis." Engineers don't usually employ best-case analysis. However, in estimating the probability of detection (POD), the NDT community employs best-case analysis.
The question of whether to use best-case or worst-case analysis arises when a test is positive for a flaw and the positive relates to more than one flaw. That this relation is one-to-many violates Unit Measure and invalidates probability theory. However, Unit Measure can be preserved and probability theory restored by selecting a flaw and counting it as detected while counting all of the other flaws as not detected. A value judgement is made in selecting the flaw. Thus, this method of analysis is value-laden.
Under best-case analysis, the largest of the flaws is selected. Under worst-case analysis, the smallest of them is selected. Best-case analysis provides an upper bound on the POD of large flaws and a lower bound on the POD of small flaws, while worst-case analysis provides the opposite. Thus, best-case analysis makes NDT look better than it is while worst-case analysis makes it look worse.
The use of best-case analysis without warning is ethically striking under any circumstances but this is particularly true when the organization employing it is a safety regulator. Nonetheless, studies published by the U.S. Nuclear Regulator Commission ("Steam Generator Group Project. Task 13 Final Report: Nondestructive Examination Validation.," NUREG/CR-5185, 1988. U.S. Nuclear Regulatory Commission, Washington, DC.) and the U.S. Federal Aviation Administration ("Reliability Assessment at Airline Inspection Facilities, Violume III: Results of an Eddy Current Inspection Reliability Experiment." DOT/FAA/CT-92/12, III, Federal Aviation Administration, Washington, DC) use best case analysis. Neither agency warns the readers of its report that the POD estimates are the result of best case analysis.
For more than 14 years, I've observed that the NDT community displays no, discernable interest, in modifying its practices to preserve Unit Measure and have always wondered why this is. Here is a theory. If the NDT community were following custom in engineering, by using worst-case analysis in estimating the POD, it could make NDT look better by modifying it to preserve Unit Measure. However, in reality, the NDT community is using best-case analysis and failing to warn people of the departure from custom. It follows that the NDT community would make NDT look worse by modifying it to preserve Unit Measure. Is thiswhy I can't find an interest in preserving Unit Measure?