Violations of probability theory in PISC III
The flaw finding methods of NDT violate an axiom of probability theory in relation to claims about their reliability (see http://www.ndt.net/article/v04n05/oldberg/oldberg.htm ). These violations tend to be extremely large, making this kind of NDT nearly opaque to scientific investigation.
Nonetheless, a number of studies have attempted to investigate this kind of NDT scientifically and to make probabilistic claims about the reliability. One of these attempts is reported in "Final Results of the PISC III Round Robin Test on Steam Generator Tube Inspection," (see http://www.ndt.net/article/v04n10/bieth/bieth.htm ).
In order for probability theory to be preserved by a test, it is necessary that 1) the test defines a set of true positives, false positives, true negatives and false negatives (set #1) , and 2) the test defines a set of sampling units (set #2), and 3) set #1 relates one-to-one to set #2. However, procedures for the inspection of steam generator tubes define neither set #1 nor set #2.
Theauthors of "Final Results of the PISC III Round Robin..." must have proceeded by attempting to fill in the information that was missing from the testing procedures. However, their report identifies neither set #1 nor set #2. This feature of the report hinders the reader's ability to determine how the study's violations of probability theory are manifest.
One gathers that set #2 is supposed to be the set of flaws, that a true positive is generated if a flaw is proximate to an indication and that a false negative is generated if a flaw is not proximate to an indication. One also gathers that set #1 is supposed to be the set of true positives and false negatives. This cannot be true, however, for the true negatives and false positives are missing from set #1 while the associated sampling units are missing from set #2. That these items are missing is evidently one of the ways in which the study manifests violations of probaility theory.
Does the set of true positives and false negatives relate one-to-one to the set of flaws? If so, the study's "Flaw Detection Probability" is, a probability. Otherwise, it is not.
The authors do not the address the question of whether the relationship is one-to-one explicitly. However, the nature of materials is not conducive to a one-to-one relationship, for the density of flaws is great and thus, if an indication is proximate to one flaw it is apt to be proximate to a great many flaws.
If the relationship is one-to-many, it can be made on-to-one through the worst case assumption that it is the smallest of the proximate flaws that has been detected or the best case assumption that it is the largest of the proximate flaws but as either assumption is value-laden, to impose it on a theory of the reliability would be to place this theory outside the bounds of science. That the report's probability of detection rises to 1 for large flaws implies that, if they have made one or the other of the assumptions, the authors have made the best case assumption.