Verification and Validation

Verify or establish

CLIA draws the line clearly ¹:

• Unmodified FDA-cleared or approved test: the lab must verify it can match the manufacturer’s performance for accuracy, precision, reportable range, and that the reference intervals fit its patients.
• Modified test or LDT: the lab must establish accuracy, precision, analytical sensitivity, analytical specificity including interfering substances, reportable range, reference intervals, and any other characteristic the test needs.

In both cases this is done before reporting patient results, and the results determine the lab’s control procedures and calibration schedule ¹.

What CLIA requires

What each characteristic means for a molecular test:

For a qualitative test, accuracy is reported as positive and negative percent agreement with the comparator. For a quantitative test such as viral load, it’s a regression and bias analysis across the range.

Limit of detection

CLSI EP17 defines two limits ²:

• Limit of blank (LoB): the highest apparent result expected from samples with no analyte. For PCR, a well-designed assay’s LoB is effectively zero.
• Limit of detection (LoD): the lowest concentration detected about 95% of the time.

For molecular assays, LoD is found from hit rates ²:

1. Prepare a dilution series of a characterized target in the specimen matrix, bracketing the expected LoD.
2. Test many replicates at each level (20 is common), across days and operators.
3. Record the fraction detected at each level.
4. Fit a probit curve, or take the lowest level with at least 19 of 20 detected, as the LoD.
5. Confirm with additional replicates at the claimed LoD.

The two methods can disagree. In the chart above, 200 copies/mL had 19 of 20 detected, but the probit fit through all the data puts 95% detection nearer 270. Probit uses every level and is usually the more conservative and defensible claim.

LoD should be expressed in units that mean something clinically (copies/mL of specimen, or international units), and it’s measured for the whole process, extraction included.

Analytical specificity

Two parts:

• Cross-reactivity: test near-neighbor organisms and homologous sequences. A C. difficile toxin assay is tested against other clostridia; a gene assay against pseudogenes.
• Interference: spike target into samples containing potential interferents - blood, mucin, lipids, bilirubin, common drugs, transport-medium components, heparin - and confirm results aren’t affected.

A verification in practice

A lab brings in an FDA-cleared, unmodified respiratory panel. A practical verification plan might include:

• Accuracy: 20 or more previously characterized positive samples (covering the main targets) and 20 or more negatives, compared with the lab’s existing method or a reference lab.
• Precision: a low positive and a negative run in replicate over several days by different operators.
• Reportable range: not applicable for a qualitative panel; for a quantitative test, a dilution series.
• LoD: confirm the manufacturer’s claim with replicates at that concentration.

The medical director reviews and signs the summary before patient testing starts. Sample numbers come from the lab’s policy, accreditor guidance, and statistics, not from CLIA, which doesn’t specify them.

What to take from this

Unmodified FDA-cleared tests are verified for accuracy, precision, reportable range, and reference intervals; LDTs and modified tests must also establish analytical sensitivity and specificity, including interferences. LoD is the concentration detected about 95% of the time, found from replicate hit rates and probit analysis across the whole process. Specificity studies cover cross-reacting organisms and sequences and interfering substances. All of it is done and signed off before patient results are reported.