Developing and evaluating new veterinary laboratory tests

The Animal Health Committee (AHC) laboratory task group, with administrative support from the Department of Agriculture and Water Resources, seeks to sustain and improve the quality of livestock and livestock products and to assure market access through the application of best practice to veterinary laboratory services.

Guidelines have been developed to assist laboratory scientists in the development and approval of new diagnostic tests through to their inclusion in the Australian and New Zealand standard diagnostic procedures (ANZSDPs).

The first step in this process is the development of the test to the point where it is eligible for submission to the laboratory task group for evaluation. If approved, the test is expected to be included in the relevant ANZSDP at its next revision unless relevant information and development in the transition period suggest otherwise.

The purpose of this procedure is to detail the process and responsibilities for managing new test development and evaluation.

This procedure applies to the development, evaluation and approval of new laboratory tests for diseases of production, feral, captive and companion animals.

4.1 New test development records

4.1.1 The laboratory task group will consult annually for the purpose of updating records. Each member will receive a list of tests under development and will be required to provide an update to the laboratory task group.

4.1.2 The laboratory task group, with administrative support from the Department of Agriculture and Water Resources, will maintain and update a spreadsheet with lists of the following:

• new tests being developed
• tests in routine use in diagnostic laboratories (often validated in-house, to at least World Organisation for Animal Health (OIE) Stage II, but not submitted to the laboratory task group for inclusion in ANZSDPs).

4.1.3 The laboratory task group, with administrative support from the Department of Agriculture and Water Resources, will develop and maintain validation templates for submissions of new tests for evaluation.

4.2 New test evaluation

4.2.1 Laboratory task group members

Evaluation of new tests will be undertaken by the laboratory task group under the AHC. The laboratory task group consists of members with relevant expertise in test types and/or validation. Members should have adequate knowledge of the policy and procedures for new test development and evaluation. All assessors are requested to ensure confidentiality and should declare any conflicts of interest to the laboratory task group before receiving dossiers and when conflict of interest arises during the evaluation process.

4.2.2 New test approval process

4.2.2.1 Submitters forward dossiers to the laboratory task group in electronic (PDF) format.

4.2.2.2 The laboratory ​task group reviews the submitted dossier and other relevant information to determine if it falls under the scope of the laboratory task group to review and is in the correct format with sufficient validation data. If not, it will be returned to the submitter with feedback on reason(s) for rejection or corrective action(s) required.

4.2.2.3 The laboratory task group will assemble a review panel and nominate a coordinator of the review panel. This panel should consist of at least 2 independent technical assessors who are experts in a relevant field and/or assay validation and may be members of the laboratory task group. The coordinator is responsible for ensuring the assessors complete the review within the given timeframe and, if required, identify additional assessor(s) or replacement assessor(s).

4.2.2.4 Written reports from the assessors will be submitted to, and considered by, the laboratory task group. Where reviewers have differing opinions the laboratory task group will make the final decision to recommend, or not, the new test for submission to the AHC for endorsement.

4.2.2.5 The coordinator of the review panel will complete the New Test Evaluation (NTE) Recommendation Template for consideration by laboratory task group members. The document will be forwarded to members out of session (OOS) with the following time frames for approval: 10 business days to provide comments or approval and, if required, another 10 business days to confirm approval for the updated recommendations (with members’ comments addressed). There will be a period for the review panel and the submitter to address laboratory task group members’ comments between these 2 consultation rounds subject to the nature and level of issues raised by members.

4.2.2.6 The new test will be approved by the laboratory task group through consensus. Approvals may be granted with specific conditions.

4.2.2.7 The laboratory task group will communicate with the submitter about the progress and outcome of the assessment. This includes the provision of a decision and specific conditions if appropriate. If not approved, feedback or recommendations for remedial action will be provided. The assessors will remain anonymous to the submitter in all communications unless prior permission is obtained.

4.2.2.8 Submitters may choose to resubmit the test after inclusion of further information. Where this is done within 6 months, the same review panel should be used for consistency.

4.2.2.9 Approved new tests will be submitted to the AHC for endorsement within 10 business days of approval.

4.2.2.10 Should the AHC provide any variations to an approved test and/or decide not to endorse the test (partly or fully), the laboratory task group will seek reasons and/or clarification from the AHC. The laboratory task group will circulate AHC feedback to all members, and communicate the outcome of the AHC process to the submitter, subject to directives from the AHC and the laboratory task group members.

4.2.2.11 Once a test is approved by the laboratory task group and endorsed by the AHC it will be listed on the Department of Agriculture and Water Resources website—see Approved tests.

4.2.2.12 The test protocol will be made available on the department’s website for access by the public.

4.2.3 New test evaluation records

The laboratory task group, with administrative support from the Department of Agriculture and Water Resources, will maintain a summary list of tests under consideration, approved tests and relevant dates, correspondence and decisions. It will present this information as a paper at each laboratory task group face-to-face meeting.

4.2.4 Normal target timeline for the assessment process

• Review by the laboratory task group and decision to proceed—10 business days.
• Formation of a review panel, including confirming independent assessors—15 business days.
• Independent assessment process—60 business days.
• Analysis of assessment reports and development of recommendations to the laboratory task group—20 business days.
• Laboratory task group’s first review of the recommendations (approval may be granted at this stage)—10 business days.
• If required, revision of the recommendations by the review panel—10 business days or more (subject to issues raised).
• If required, the laboratory task group’s second review of revised recommendations—10 business days.
• Submission to the AHC for endorsement—10 business days.

4.2.5 Costs

Charges are due at the time of submission to the laboratory task group and will be invoiced once the submission is sent for independent review by an assessment panel. Payment for submissions will be used to cover costs of the assessors. The laboratory task group, with administrative support from the Department of Agriculture and Water Resources, is responsible for invoicing the submitter to recoup these costs.

The costs may be reviewed by the laboratory task group from time to time and are $3,000 for new submissions. Extensions of existing assays might be considered where the minor changes would not impact significantly on the fitness for purpose of the assay involved. The cost for review of an extension to an existing assay will be $2,000. Assays submitted as extensions which are not accepted by the laboratory task group as extensions, may be resubmitted as a new test with payment of an additional $1,000. The laboratory task group’s decision on the evaluation of submitted dossiers will be final.

Submitters may review and resubmit within 6 months, and will incur a cost of $500.

These documents supporting this procedure can be found on the department’s website:

• Validation template for serological assays
• Validation template for nucleic acid detection
• Validation template for extension of existing assays
• Policy to develop and evaluate new veterinary laboratory tests

This policy establishes agreed mechanisms and guidelines for the development, evaluation and communication of new tests developed in Australia and New Zealand for infectious animal diseases, for use in veterinary laboratories. It is consistent with international (OIE) guidelines and is intended to ensure a uniform understanding of the way in which new tests are developed, evaluated and communicated.

This policy applies to new tests developed in Australia and New Zealand for use in production animals as well as feral, captive and companion animals.

The Animal Health Committee laboratory task group, with administrative support from the Department of Agriculture and Water Resources, seeks to sustain and improve the quality of livestock and livestock products and to assure market access through the application of best practice to veterinary laboratory services.

The laboratory task group has a mandate to facilitate the development, evaluation and adoption of new diagnostic methods presented for consideration to Animal Health Committee. New test development is facilitated by:

• maintaining a register of new test methods under development to reduce duplication
• promoting the establishment of collaborative links between institutions.

New test evaluation is facilitated by:

• developing and communicating information on validation requirements for new tests that will be included in the Australian and New Zealand standard diagnostic procedures (ANZSDPs), including validation templates and guidelines for authors
• coordinating independent evaluation of new test methodologies and validation data for tests that are to be included in the ANZSDPs
• providing feedback to submitters following the evaluation process.

Adoption of new diagnostic methods is facilitated by:

• monitoring and communicating new test developments
• providing a transparent and auditable process for evaluating new diagnostic tests developed for the livestock industries and assisting their inclusion, as appropriate, into standard test procedures (e.g. ANZSDP)
• promoting the uptake of new tests by Australian and New Zealand laboratories.

This policy is designed to assist laboratory scientists in the development and evaluation of new diagnostic tests through to their inclusion (where appropriate) in ANZSDPs.

5.1. Examination of fitness for purpose (the reasons why a test is performed)

When developing a new test, consideration of the purpose for which the test will be used must be considered. This particularly applies to test evaluation/validation because it is important to ensure that the test is evaluated using samples from the same population as those that the test will be used on.

The 6 levels described by OIE for ‘Fitness for Purpose’ and which are accepted in Australia and New Zealand are to:

1. demonstrate population 'freedom' from infection (prevalence apparently zero) in the following categories:

• 'free' with or without vaccination
• historical 'freedom'
• re-establishment of 'freedom' following outbreaks

2. demonstrate freedom from infection or agent in individual animals or products for trade purposes
3. demonstrate efficiency of eradication policies
4. confirm diagnosis of clinical cases:
   • in groups of animals, or
   • in individuals
5. estimate prevalence of infection to facilitate risk analysis (surveys, classification of herd health status, implementation of disease control measures)
6. to determine immune status in individual animals or populations (post-vaccination).

5.2. Priorities for test approval process

Consistent with best current practice all tests used in Australian/New Zealand animal health laboratories should be adequately validated. Test developers should seek laboratory task group approval for new tests applicable to:

• national disease control/eradication and surveillance programs
• health certification testing (for livestock destined for international export)
• tests developed for exotic disease and required to be transferred to other laboratories. The above tests (when approved) will be included in ANZSDPs (as a matter of course) and will carry the statement ‘Approved by AHC laboratory task group for the stated purpose’—the stated purpose being any one or a combination of 1–6 above.

5.3. Requirements for test approval

It is recognised that there are 5 stages in the validation process as outlined by the OIE. These include:

1. feasibility
2. development and standardisation
3. assay performance characterisation
4. monitoring performance (including transferability—tested in another lab)
5. maintenance/extended validation criteria.

Submitters are expected to complete the relevant validation template for serological or nucleic acid detection methods, which detail the continuum of the developmental approach in a standard format. Tests may be submitted as extensions to an existing assay where the minor changes would not be expected to impact significantly on the fitness for purpose of the assay involved. (Tests on different animal species could reasonably be expected to have quite different specificity.) In the case of an extension, submitters must complete the validation template for extension of existing assays. Completed templates should be provided to the laboratory task group.

New tests submitted for approval must have reached at least stage 4 before submission to the laboratory task group. Assuming the submission contains all of the elements of the validation template in sufficient detail and quality, the test may be approved for the stated purpose.

The process for test evaluation and the cost of the assessment by the independent reviewers are outlined within the procedure to develop and evaluate new veterinary laboratory tests. The laboratory task group may approve inclusion of a new test in the appropriate ANZSDP.

New tests that are developed for use in individual laboratories that do not fit with the priorities outlined above, do not require laboratory task group approval, but should nevertheless be validated.

Laboratory managers responsible for the introduction of such new tests should develop a system of independent review prior to implementation, as a routine practice.

In cases where individual reagents are changed in a standard test, managers should ensure that the laboratory verifies that any new critical reagent performs adequately against standard reagents before the new reagent is used in routine diagnostic testing.

It is desirable but not essential for a new test to be published in a peer-reviewed scientific journal prior to submission to the laboratory task group for test evaluation.

Neither the laboratory task group, nor its members, are legally liable for any repercussions for conferring ‘Approval’ for any given test or for not granting ‘Approval’. By submitting a dossier for laboratory task group evaluation the submitter agrees to current procedures and policies.

The evaluation process applies to all tests as nominated in section 5.3. Tests already in common use will be assessed for approval on a priority basis.
Although the laboratory task group has no formal authority to force laboratories to use an ‘Approved’ test, it recommends through its members and associated networks that approved tests are used where possible. NATA Supplementary Requirement for Veterinary Testing states that laboratories ‘should’ use ANZSDPs ‘where appropriate’.

Laboratories are encouraged to collect and store samples that may be useful for validation purposes, especially for uncommon diseases, so that when validation is needed, sufficient samples are available, even if it means aggregation of samples from various laboratory sources.

In the context of non-standard methods, in-house methods and standard methods that have been modified or are being used outside their intended application:

Validation is the demonstration of technical soundness, method performance and limitations of the method and an indication of the suitability of the method for its intended use. Validation studies should include evaluations of uncertainty of measurement (where relevant), detection limit (or sensitivity), selectivity of the method (or specificity), linearity, limit of repeatability and/or reproducibility and robustness against external influences.

In the context of ‘standard’ methods (i.e. kit methods validated by the manufacturer, methods in the ANZSDPs):

Verification is the demonstration of the ability to achieve the performance specifications of the manufacturer/standard method under the testing laboratory’s own conditions.