Contaminant detection: a new score to assess the performance of testing laboratories
Scientists at ANSES have created a new score for statistically assessing the detection capabilities of testing laboratories. This score can be used when it is necessary to verify the ability of laboratories to detect a contaminant such as a pathogen or chemical in a sample.
Testing laboratories are regularly required to take part in inter-laboratory proficiency tests (ILPTs) that assess their performance. During these tests, the laboratories are asked to analyse samples whose contents are known only to the organising laboratory. Various statistical scores are used to objectively determine the performance of testing laboratories. However, few scores are available for qualitative tests such as presence/absence tests. “Most qualitative ILPTs do not include samples close to the limit of detection” explains Christian Baudry, a statistician at the ANSES Ploufragan-Plouzané-Niort Laboratory. “A laboratory either manages or fails to analyse a sample correctly. However, analytical difficulty is not taken into account when assessing the participating laboratory”. Scientists from ANSES's Ploufragan-Plouzané-Niort and Fougères Laboratories therefore joined forces to create a new score that is more accurate and easier to interpret and takes account of detection difficulty. Indeed, under their multiple French and European reference mandates, ANSES's laboratories are regularly called upon to organise ILPTs for the field laboratories approved by the French Ministry of Agriculture (or for National Reference Laboratories) to carry out official analyses aimed at monitoring the health of animals, plants and foods circulating in France.
A score format inspired by a pre-existing test
This score, called S-score for “Summary score for binary data”, is divided into two parts. The integer part of the number, from 1 to 3, corresponds to the results of the proficiency test: 1 means the results are satisfactory, 2 means they are questionable, and 3 means they are unsatisfactory. “The integer part should be interpreted like the z-score, which is the score most commonly used for quantitative tests and the one to which testing laboratories are accustomed” explains Michel Laurentie, head of the methodological platform for statistical support for reference activities at ANSES. The decimal part of the score indicates the percentage of incorrect results obtained. For example, a laboratory receiving a score of 1.25 will have obtained 25% incorrect results during the test, but the results will still have been deemed acceptable overall, especially if the analytical method was difficult to implement.
Methods for calculating and using this score were published in the journal Accreditation and Quality Assurance this past February. ANSES's reference laboratories plan to start using the S-score as part of the inter-laboratory assessments they will be organising from June 2024. This score will be proposed for inclusion in the international statistical standard applied to laboratory proficiency testing (ISO 13528).