where expertise comes together - since 1996

Web's Largest Portal of Nondestructive Testing (NDT)
Open Access Database (Conference Proceedings, Articles, News), Exhibition, Forum, Network

All Forum Boards
Technical Discussions >
Ethics and Verifiability
Career Discussions
Job Offers
Job Seeks
Classified Ads
About NDT.net
Articles & News

IMechE Engineering Training Solutions
World leading provider of Non-Destructive Testing and Corrosion Prevention and Coatings training, examinations and Level 3 Consultancy Services

3069 views
09:33 May-13-1999

Terry Oldberg

Engineering, Mechanical Electrical Nuclear Software
Consultant,
USA,
Joined Oct 1999
42
Ethics and Verifiability

In a thread entitled "Conflicting Claims," I pointed out that papers published in the May 1999 issue of NDTnet make conflicting claims. In particular, the paper entitled "The Selection of an NDT Method..." (http://www.ndt.net/article/v04n05/rihar/rihar.htm) claims that "The selection of an NDT method is usually the concern of NDT laboratories, which know the NDT methods and their feasibility in detecting defects" but the paper entitled "Erratic Measure" (http://www.ndt.net/article/v04n05/oldberg/oldberg.htm) claims that all of the various methods for detecting defects violate an axiom of probability theory. The violations of probability theory restrict the ability of NDT laboratories to know the feasibility and may eliminate this ability altogether.

In a continuation of the thread, Rolf Diederichs asks, in effect, what should be done about this situation. Answers to this question are value-laden, hence the question cannot be answered scientifically. I am initiating this thread to foster discussion of the value laden aspects of the Conflicting Claims issue. In the following paragraphs, I provide my own views and invite the views of others.

If a professional has restricted knowledge of the reliability of a diagnostic technique or no knowledge at all, I think it is ethically incumbent on him or her to make this plain to the client. Were this to be practiced in NDT, clients would be warned that the body of research underlying claims about the reliability of NDT is statistically flawed. I don't believe this is happening.

What we have instead, I think, is an example of the fallacy of argumentum ad verecundiam (argument from false authority). The ASNT's rules for the certification of personnel provide a prime example.

If a person's probabilities of false positive and false negative error cannot be estimated, neither the ASNT nor anyone else can know whether he or she is competent. "Erratic Measure" reveals that there are significant problems with these probabilities. Nonetheless, personnel are, in fact, certified under the ASNT's sanction.

In my view, the ethical way of dealing with this situation is for sanctioning organizations, like ASNT, to withdraw their sanctions while the problems are addressed and eliminated. What do other members of the NDT community think should be done about this situation?



 
03:44 May-13-1999
Norm Woodward
Re: Ethics and Verifiability Mr Oldberg has raised charges against probability of detection (POD), a basic tenet of NDE. If the concept is bogus, as he claims, then we have tremendous safety issues to consider. He discusses in depth the reliance of the nuclear industry on this "pseudoprobability,", but as I am personally well aware, the flightworthiness of our commercial and military fleets are linked inescapably with this concept. Even our newest aircraft, built to damage tolerance designs, rely one POD data to determine inspection intervals that will prevent cracks and other defects to grow undetected to potentially fatal extents. And for regulatory agencies, such as the FAA or the NRC, to ignore, or be ignorant of, these charges would be neglegence of the highest order.

Fortunately, these agencies have been made aware of these charges by the author even before the 1995 ASME paper was published. They consulted with the several experts in this field, including authors of some of the papers refered to in the Mr Oldberg's paper, and came to an unanimous conclusion that the arguments had no merit.

It may be noted that much of Mr Oldberg's work on this subject was done while in the employ of EPRI. Those of you in the power industry know that EPRI still seems to hold great stock in the POD concept.

: In a thread entitled "Conflicting Claims," I pointed out that papers published in the May 1999 issue of NDTnet make conflicting claims. In particular, the paper entitled "The Selection of an NDT Method..." (http://www.ndt.net/article/v04n05/rihar/rihar.htm) claims that "The selection of an NDT method is usually the concern of NDT laboratories, which know the NDT methods and their feasibility in detecting defects" but the paper entitled "Erratic Measure" (http://www.ndt.net/article/v04n05/oldberg/oldberg.htm) claims that all of the various methods for detecting defects violate an axiom of probability theory. The violations of probability theory restrict the ability of NDT laboratories to know the feasibility and may eliminate this ability altogether.

: In a continuation of the thread, Rolf Diederichs asks, in effect, what should be done about this situation. Answers to this question are value-laden, hence the question cannot be answered scientifically. I am initiating this thread to foster discussion of the value laden aspects of the Conflicting Claims issue. In the following paragraphs, I provide my own views and invite the views of others.

: If a professional has restricted knowledge of the reliability of a diagnostic technique or no knowledge at all, I think it is ethically incumbent on him or her to make this plain to the client. Were this to be practiced in NDT, clients would be warned that the body of research underlying claims about the reliability of NDT is statistically flawed. I don't believe this is happening.

: What we have instead, I think, is an example of the fallacy of argumentum ad verecundiam (argument from false authority). The ASNT's rules for the certification of personnel provide a prime example.

: If a person's probabilities of false positive and false negative error cannot be estimated, neither the ASNT nor anyone else can know whether he or she is competent. "Erratic Measure" reveals that there are significant problems with these probabilities. Nonetheless, personnel are, in fact, certified under the ASNT's sanction.

: In my view, the ethical way of dealing with this situation is for sanctioning organizations, like ASNT, to withdraw their sanctions while the problems are addressed and eliminated. What do other members of the NDT community think should be done about this situation?




 
01:50 May-14-1999

Terry Oldberg

Engineering, Mechanical Electrical Nuclear Software
Consultant,
USA,
Joined Oct 1999
42
Re: Ethics and Verifiability I had hoped that discussers of "Erratic Measure" would help organize the discussion by placing discussion of the scientific and ethical issues in separate threads, with this thread reserved for the ethical issues. To partition discussions in this way has a demonstrated ability to help resolve issues and is a feature of the so-called "scientific method." However, Mr. Woodward's remarks mix the scientific and ethical aspects of the issue. I'll address both types of remark here, while requesting that this thread be reserved for the ethical aspects in the future.

A portion of Mr. Woodward's remarks about the POD have a scientific flavor but seem a bit wide of the target. To help improve the aim of subsequent discussers, I'll point out that at least 2 probabilities are required in order to characterize the reliability of a test; by itself, the POD doesn't do the job. This second probability (often taken to be the "probability of false call" in the literature) is missing from the NRC study which is reviewed in "Erratic Measure" and some other studies. That it is missing is symptomatic of the violations of the Unit Measure axiom of probability theory which are claimed by "Erratic Measure." I'll also point out that the "probabilities" of detection cited in the literature are not always probabilities, according to "Erratic Measure." For example, the "probability" of detection cited by the authors of the NRC study which is reviewed in "Erratic Measure" is a probability only under the condition that its value is 0. When its value is greater than 0, the study's POD violates the Unit Measure axiom of probability theory.


The non-scientific portion of the posting raises questions about the process by which the claims made by "Erratic Measure" have been adjudicated. In this light, I will point out that the paper is published within the peer-review system. So far as I am aware, the peer-reviewed literature contains no refutation of the paper. Thus, the paper's conclusion stands until it is refuted within the peer review system under the rules of scientific due process.

Mr. Woodward states that the NRC and FAA were aware of the paper's arguments before the paper was printed and "...came to the unanimous conclusion that the arguments had no merit." I don't know how to address this statement, other than to point out that it seems to attempt to replace the peer review system by the fallacy of argumentum ad verecundiam (argument from false authority). If Mr. Woodward were to provide citations to publications providing the two organizations' arguments or to repeat these arguments, then it would be possible for the scientific community and participants in this Forum to address them. However, Mr. Woodward does not provide the arguments and I am not aware of any such publications.

: Mr Oldberg has raised charges against probability of detection (POD), a basic tenet of NDE. If the concept is bogus, as he claims, then we have tremendous safety issues to consider. He discusses in depth the reliance of the nuclear industry on this "pseudoprobability,", but as I am personally well aware, the flightworthiness of our commercial and military fleets are linked inescapably with this concept. Even our newest aircraft, built to damage tolerance designs, rely one POD data to determine inspection intervals that will prevent cracks and other defects to grow undetected to potentially fatal extents. And for regulatory agencies, such as the FAA or the NRC, to ignore, or be ignorant of, these charges would be neglegence of the highest order.

: Fortunately, these agencies have been made aware of these charges by the author even before the 1995 ASME paper was published. They consulted with the several experts in this field, including authors of some of the papers refered to in the Mr Oldberg's paper, and came to an unanimous conclusion that the arguments had no merit.

: It may be noted that much of Mr Oldberg's work on this subject was done while in the employ of EPRI. Those of you in the power industry know that EPRI still seems to hold great stock in the POD concept.

: : In a thread entitled "Conflicting Claims," I pointed out that papers published in the May 1999 issue of NDTnet make conflicting claims. In particular, the paper entitled "The Selection of an NDT Method..." (http://www.ndt.net/article/v04n05/rihar/rihar.htm) claims that "The selection of an NDT method is usually the concern of NDT laboratories, which know the NDT methods and their feasibility in detecting defects" but the paper entitled "Erratic Measure" (http://www.ndt.net/article/v04n05/oldberg/oldberg.htm) claims that all of the various methods for detecting defects violate an axiom of probability theory. The violations of probability theory restrict the ability of NDT laboratories to know the feasibility and may eliminate this ability altogether.

: : In a continuation of the thread, Rolf Diederichs asks, in effect, what should be done about this situation. Answers to this question are value-laden, hence the question cannot be answered scientifically. I am initiating this thread to foster discussion of the value laden aspects of the Conflicting Claims issue. In the following paragraphs, I provide my own views and invite the views of others.

: : If a professional has restricted knowledge of the reliability of a diagnostic technique or no knowledge at all, I think it is ethically incumbent on him or her to make this plain to the client. Were this to be practiced in NDT, clients would be warned that the body of research underlying claims about the reliability of NDT is statistically flawed. I don't believe this is happening.

: : What we have instead, I think, is an example of the fallacy of argumentum ad verecundiam (argument from false authority). The ASNT's rules for the certification of personnel provide a prime example.

: : If a person's probabilities of false positive and false negative error cannot be estimated, neither the ASNT nor anyone else can know whether he or she is competent. "Erratic Measure" reveals that there are significant problems with these probabilities. Nonetheless, personnel are, in fact, certified under the ASNT's sanction.

: : In my view, the ethical way of dealing with this situation is for sanctioning organizations, like ASNT, to withdraw their sanctions while the problems are addressed and eliminated. What do other members of the NDT community think should be done about this situation?




 
06:07 May-16-1999

Ed Ginzel

R & D, -
Materials Research Institute,
Canada,
Joined Nov 1998
1197
Re: Ethics and Verifiability I doubt my academic achievements are on a par with the
participants in this discussion so far; however,
I have had over a quarter century of experience in NDT.
(mostly hands on).
The conclusion that seems to to have been drawn by
Terry Oldberg would indicate that performing NDT has
little use if we cannot assure a customer,
without any doubt, that our results are infalable.
The impression I get from this augument is that
Statistically we are be not much better at finding
defects by NDT than a toss of a coin.
I have worked in many phases of industry and in
several industries (including nuclear and petrochemical).
In all cases I have had the opportunity to actually
see some of the flaws I called. This was accomplished
by several different means, excavation by repair welders,
macros taken after sectioning and radiographic verifications.
In most cases flaws were evident, i.e. the test was effective.

From the tone of Terry's comments I get the impression
that he is of the opinion that we are just wasting the
customer's money by performing NDT. However, having exposed
flaws via NDT I fail to see that we are equally at risk
if we perform or do not perform NDT on test pieces.

The role of certification is quite a different matter.
ASNT does not "sanction" anything. They set in place
quidelines for certification. Even in other countries
where we have central certification (e.g. Canada & UK)
the individual's ethics are not tested, merely their
knowledge to a base level.

Terry seems to be rining the bells of doom but I do not
understand why such an extreme position is taken in the
face of day-to-day experience in the field. There are
good technicians and there are poor technicians.
There are effective techniques and there are not so effective
techniques. There are comfortable test environments
and there are miserable test environments. These
variables and others will play a partin the test but
to eliminate the test because of a mathematical
probability seems to be "putting the cart before the horse".

Ed

: In a thread entitled "Conflicting Claims," I pointed out that papers published in the May 1999 issue of NDTnet make conflicting claims. In particular, the paper entitled "The Selection of an NDT Method..." (http://www.ndt.net/article/v04n05/rihar/rihar.htm) claims that "The selection of an NDT method is usually the concern of NDT laboratories, which know the NDT methods and their feasibility in detecting defects" but the paper entitled "Erratic Measure" (http://www.ndt.net/article/v04n05/oldberg/oldberg.htm) claims that all of the various methods for detecting defects violate an axiom of probability theory. The violations of probability theory restrict the ability of NDT laboratories to know the feasibility and may eliminate this ability altogether.

: In a continuation of the thread, Rolf Diederichs asks, in effect, what should be done about this situation. Answers to this question are value-laden, hence the question cannot be answered scientifically. I am initiating this thread to foster discussion of the value laden aspects of the Conflicting Claims issue. In the following paragraphs, I provide my own views and invite the views of others.

: If a professional has restricted knowledge of the reliability of a diagnostic technique or no knowledge at all, I think it is ethically incumbent on him or her to make this plain to the client. Were this to be practiced in NDT, clients would be warned that the body of research underlying claims about the reliability of NDT is statistically flawed. I don't believe this is happening.

: What we have instead, I think, is an example of the fallacy of argumentum ad verecundiam (argument from false authority). The ASNT's rules for the certification of personnel provide a prime example.

: If a person's probabilities of false positive and false negative error cannot be estimated, neither the ASNT nor anyone else can know whether he or she is competent. "Erratic Measure" reveals that there are significant problems with these probabilities. Nonetheless, personnel are, in fact, certified under the ASNT's sanction.

: In my view, the ethical way of dealing with this situation is for sanctioning organizations, like ASNT, to withdraw their sanctions while the problems are addressed and eliminated. What do other members of the NDT community think should be done about this situation?




 
08:41 May-16-1999

Ed Ginzel

R & D, -
Materials Research Institute,
Canada,
Joined Nov 1998
1197
Re: Ethics and Verifiability
In my comments earlier I noticed I incorrectly worded
a common metaphor when I said "...mathematical
probability seems to be "putting the horse before
the cart". "
This should have been "putting the cart before
the horse".

Sorry

Ed




 
08:28 May-29-1999
Jim Johnson
Re: Ethics and Verifiability Have you had an instance where an NDT vendor mislead you
as to their capabilities? Have you increased your risks
by using manual NDT where a computer controled version would
increase POD?

In my experience NDT Labs are hired to "scan this and report"
or "inspect this to code" which they do. They were contracted
to perform a specific act by a cognizant authority who, in every case
knows exactly the inspection problems at hand...not!!!

Most of us inspector types do not think of error probablity.
A few of us who have a better unstanding of human nature do.
We all do our best to perform to contract and process procedure and we
do miss things and report false positives. As long as we rely
on our eyeballs and finger tips we will error. Variability is
normal, especially for humans.

I have never heard of a NDT contract stating that one
must find such and such a discontinuity in such and such an
area with a .999997 probablility of success. Chances are good
that the cognizant engineer does not even have the remotest
idea what the probability of a specific flaw existing is a finite matrix
actually is much less the NDT contractor. So, does the vendor
inform his client that their engineers cannot properly
define a scope of work or does the vendor perform to contract.

Anyway, there are two sides to the ethics issue and regardless
the contract rules.

Jim Johnson


 
07:56 Jun-06-1999

Terry Oldberg

Engineering, Mechanical Electrical Nuclear Software
Consultant,
USA,
Joined Oct 1999
42
Re: Ethics and Verifiability t be established.

o The probability of detection for large flaws, reported by the NRC as 1, is the result of best case analysis. If worst case analysis is used, per standard practice in engineering, the probability of detection is 0.

In the wake of the NRC's research, one does not know the probability of false call at all and knows only that the probability of detection for large flaws is between 0 and 1. Should the NRC continue to require that utilities inspect their nuclear steam generator tubes with this kind of information? Should the ASME continue to sanction the methods of NDT which are used? Should the nuclear plants be kept in operation? The basis for answering these kinds of questions is not really present.

: I doubt my academic achievements are on a par with the
: participants in this discussion so far; however,
: I have had over a quarter century of experience in NDT.
: (mostly hands on).
: The conclusion that seems to to have been drawn by
: Terry Oldberg would indicate that performing NDT has
: little use if we cannot assure a customer,
: without any doubt, that our results are infalable.
: The impression I get from this augument is that
: Statistically we are be not much better at finding
: defects by NDT than a toss of a coin.
: I have worked in many phases of industry and in
: several industries (including nuclear and petrochemical).
: In all cases I have had the opportunity to actually
: see some of the flaws I called. This was accomplished
: by several different means, excavation by repair welders,
: macros taken after sectioning and radiographic verifications.
: In most cases flaws were evident, i.e. the test was effective.

: From the tone of Terry's comments I get the impression
: that he is of the opinion that we are just wasting the
: customer's money by performing NDT. However, having exposed
: flaws via NDT I fail to see that we are equally at risk
: if we perform or do not perform NDTon test pieces.

: The role of certification is quite a different matter.
: ASNT does not "sanction" anything. They set in place
: quidelines for certification. Even in other countries
: where we have central certification (e.g. Canada & UK)
: the individual's ethics are not tested, merely their
: knowledge to a base level.

: Terry seems to be rining the bells of doom but I do not
: understand why such an extreme position is taken in the
: face of day-to-day experience in the field. There are
: good technicians and there are poor technicians.
: There are effective techniques and there are not so effective
: techniques. There are comfortable test environments
: and there are miserable test environments. These
: variables and others will play a part in the test but
: to eliminate the test because of a mathematical
: probability seems to be "putting the cart before the horse ".

: Ed


: : In a thread entitled "Conflicting Claims," I pointed out that papers published in the May 1999 issue of NDTnet make conflicting claims. In particular, the paper entitled "The Selection of an NDT Method..." (http://www.ndt.net/article/v04n05/rihar/rihar.htm) claims that "The selection of an NDT method is usually the concern of NDT laboratories, which know the NDT methods and their feasibility in detecting defects" but the paper entitled "Erratic Measure" (http://www.ndt.net/article/v04n05/oldberg/oldberg.htm) claims that all of the various methods for detecting defects violate an axiom of probability theory. The violations of probability theory restrict the ability of NDT laboratories to know the feasibility and may eliminate this ability altogether.

: : In a continuation of the thread, Rolf Diederichs asks, in effect, what should be done about this situation. Answers to this question are value-laden, hence the question cannot be answered scientifically. I am initiating this thread to foster discussion of the value laden aspects of the Conflicting Claims issue. In the following paragraphs, I provide my own views and invite the views of others.

: : If a professional has restricted knowledge of the reliability of a diagnostic technique or no knowledge at all, I think it is ethically incumbent on him or her to make this plain to the client. Were this to be practiced in NDT, clients would be warned that the body of research underlying claims about the reliability of NDT is statistically flawed. I don't believe this is happening.

: : What we have instead, I think, is an example of the fallacy of argumentum ad verecundiam (argument from false authority). The ASNT's rules for the certification of personnel provide a prime example.

: : If a person's probabilities of false positive and false negative error cannot be estimated, neither the ASNT nor anyone else can know whether he or she is competent. "Erratic Measure" reveals that there are significant problems with these probabilities. Nonetheless, personnel are, in fact, certified under the ASNT's sanction.

: : In my view, the ethical way of dealing with this situation is for sanctioning organizations, like ASNT, to withdraw their sanctions while the problems are addressed and eliminated. What do other members of the NDT community think should be done about this situation?




 
08:24 Jun-06-1999

Terry Oldberg

Engineering, Mechanical Electrical Nuclear Software
Consultant,
USA,
Joined Oct 1999
42
Re: Ethics and Verifiability The ethical lapse arises when a procedure is specified and mandated in the presence of little if any information about its reliability. For example, a certain procedure for the inspection of nuclear steam generator tubes is sanctioned by the American Society of Mechanical Engineers (ASME) and mandated for periodic use by the U.S. Nuclear Regulatory Commission (NRC). However, due to violations of probability theory which are inherent in the ASME's procedure, the NRC's research:
o could not measure the probability of false call
o reported a probabability of detection of 1 for large flaws which was the result of best case analysis of the research data. With worst case analysis, the probability of detection is zero.

The ASME's sanction of this procedure and the NRC's mandate of its use are unethical, in my opinion, for a scientific basis does not exist for their decisions. Are the inspectors who execute this procedure also ethical? If they know about the shoddy engineering and do nothing to correctit, they are unethical, in my opinion.

: Have you had an instance where an NDT vendor mislead you
: as to their capabilities? Have you increased your risks
: by using manual NDT where a computer controled version would
: increase POD?

: In my experience NDT Labs are hired to "scan this and report"
: or "inspect this to code" which they do. They were contracted
: to perform a specific act by a cognizant authority who, in every case
: knows exactly the inspection problems at hand...not!!!

: Most of us inspector types do not think of error probablity.
: A few of us who have a better unstanding of human nature do.
: We all do our best to perform to contract and process procedure and we
: do miss things and report false positives. As long as we rely
: on our eyeballs and finger tips we will error. Variability is
: normal, especially for humans.

: I have never heard of a NDT contract stating that one
: must find such and such a discontinuity in such and such an
: area with a .999997 probablility of success. Chances are good
: that the cognizant engineer does not even have the remotest
: idea what the probability of a specific flaw existing is a finite matrix
: actually is much less the NDT contractor. So, does the vendor
: inform his client that their engineers cannot properly
: define a scope of work or does the vendor perform to contract.

: Anyway, there are two sides to the ethics issue and regardless
: the contract rules.

: Jim Johnson




 


© NDT.net - The Web's Largest Portal of Nondestructive Testing (NDT) ISSN 1435-4934

Open Access Database, |Conference Proceedings| |Articles| |News| |Exhibition| |Forum| |Professional Network|