Verification, FAT, SAT & Commissioning

Verification and validation

Verification proves each requirement is met: did the gate confirm in under 2 seconds? Validation proves the system solves the problem in the brief: are there fewer clogs and forgotten starts than in the baseline log? Build the tests from the requirement IDs, not from whatever is easiest to demonstrate. NIST’s robotics program frames performance the same way: a capability is only real if there’s a test method and a metric someone else can repeat ¹.

Bench test: the home FAT

In industry, a factory acceptance test (FAT) checks the integrated machine at the builder’s shop before it ships. The home version is a bench test, before anything is mounted to duct:

• CT accuracy: run each tool (or a heater on a test circuit) and compare the controller’s current reading with a clamp meter. Accept within 10 percent.
• State machine: with lamps standing in for the collector relay and bleed gate, drive every transition: start, second tool, one tool stops, last tool stops, spin-down, fault, reset. Check each against the state diagram.
• Gate cycles: 50 open-close cycles per gate on a dusty slide. Every one confirms open within 2 s.
• Jammed gate: hold a slide shut. The bleed gate opens and the buzzer sounds within 2 s.
• Unplugged CT: disconnect a sensor. It reads zero and nothing starts.

A simulation of the logic is useful before the bench, but it’s evidence about the model, not the hardware; NIST’s work on digital twins stresses verifying and validating models before trusting them ². The bench test is where the model meets real parts.

Installed test: the home SAT

A site acceptance test (SAT) repeats what matters with the real installation, because utilities, wiring, and the environment change behavior. Here:

• Air speed (AIR-01): drill a small hole in each 4-inch drop and measure with an anemometer, only that drop’s gate open. Accept at least 3800 ft/min, Pentz’s minimum for vertical runs ³. Multiply by area to get CFM; at 0.087 ft², 3800 ft/min is about 330 CFM. Record the readings; they’re the baseline for later.
• Response (RSP-02): start each tool 10 times, timing from switch-on to collector running with the gate confirmed. Accept at most 3 s every time.
• Two tools (GATE-03): run the saw and the router table together. Both gates open, the others stay closed.
• Power blip (SAFE-04): kill the controller’s power for a moment while running. It restarts in IDLE with the bleed gate open and the collector off.
• Bypass (MAN-05): unplug the controller. The manual switch still runs the collector.
• Inspection (ELEC-06): every line-voltage part listed and rated, every box closed, low-voltage wiring separate.

One complete test case

Write each test so someone else could run it. For SAFE-04:

• ID and requirement: SAFE-04, the collector never starts with all gates closed.
• Setup: system installed, collector plugged in, table saw running, controller in RUNNING.
• Steps: unplug the controller’s power supply for 2 s; plug it back in; watch for 30 s.
• Expected: on restart the bleed gate opens, all tool gates close, the collector stays off until a tool current is seen again, and the saw’s gate then reopens through OPENING.
• Pass/fail: any moment with the collector running and no gate confirmed open is a fail.
• Restore: stop the saw; return to IDLE.
• Record: date, code version, result, and anything odd.

Fault campaign

Run each fault deliberately and safely: a gate jammed shut, a limit switch unplugged, a CT unplugged, two tools started within half a second of each other, a tool started during spin-down, the bin nearly full, the controller reset in every state. For each, write what happened and whether it matched the design. A fault that does something unplanned is a finding to fix, not a failed project.

A month of use

Then validate. Keep the clipboard log for a month, the same way as the baseline:

• Clogs: target zero (baseline: 2 in two weeks).
• Forgotten starts and wrong gates: target zero (baseline: 15 in 64 starts).
• False starts: count and explain every one (a CT picking up another load? a threshold too low?).
• Full bin: re-measure air speed in one drop with the bin nearly full to see how much it falls, and set the emptying schedule from that.

Punch list and release

Keep a punch list of everything found, with a severity: blocks use (a safety function failed), fix soon (a nuisance false start), or accepted (a known limitation, written down). Release the system for everyday use only when every safety test passes, every blocking item is closed, the code version in the controller matches the notebook, and the unplug rule is tagged on the collector.

Gate 5

Pass when every requirement has recorded evidence, the month’s log beats the baseline, every fault behaved as designed or was fixed, and the punch list has no blocking items.