Mechanised and automated ultrasonic equipment is now gradually being taken into use as an alternative to radiography for examination of pipeline girth welds during pipelaying. When this is done, one meets with the requirement to formulate ultrasonic acceptance criteria to replace the radiographic workmanship level criteria used so far. To address this topic, and later also fitness-for-purpose acceptance criteria, on a general basis, a project has now been launched under the auspices of the International Pipe Line & Offshore Contractors Association. The objectives of the project are to develop ultrasonic acceptance criteria equivalent to the presently used radiographic ones, and to demonstrate their adequacy. The project therefore includes an experimental reliability data acquisition and verification programme, which is outlined. Results to be achieved, and evaluations to be done, are exemplified using available non-destructive examination reliability results from other investigations.