Sample by risk and source type, not convenience
Sample by risk and source type, not convenience. Review every high-consequence field and use randomized checks to detect systematic errors in the remaining collection.
The finished QA plan combines complete review of high-consequence fields with randomized checks across source classes, predefined acceptance rules, correction records, and a response when systematic errors appear.
Build a review sample with OCR Quality Checker. Use the next control in How to Measure OCR Accuracy with CER and WER and Preservation Master, Access Copy and OCR Derivative.
Sample-based QA plan
- Define collection strata
- Pilot the hardest material
- Use random and risk-based samples
- Set rework thresholds in advance
- Record batch-level corrections and resampling
Combine high-consequence review with randomized checks
Define strata before sampling
Separate languages, layouts, source conditions, capture devices, time periods, and document types. Random sampling from the whole collection can miss a small but consistently failing class.
Create a defect taxonomy
Track character errors, missing regions, wrong reading order, page omission, duplicate pages, bad rotation, field association, and export corruption separately. Each class needs a different corrective action.
Use risk-based escalation
Automatically flag invalid dates, impossible totals, low-confidence regions, unexpected scripts, empty pages, and unusually low word counts. Review all consequential fields even when they do not trigger a flag.
Close the feedback loop
When a sample fails, identify the affected batch and cause, reprocess it, and draw a new acceptance sample. Record decisions so the final quality statement describes what was actually checked.
Combine representative sampling with mandatory risk review
Divide the collection into groups that can fail differently, such as source type, date range, language, layout, scan operator, or damage level. Draw ordinary samples within each group and deliberately include known hard cases during the pilot. Define acceptance and rework rules before production begins.
Random sampling estimates ordinary quality, while targeted review protects fields whose errors have serious consequences. If a batch fails, investigate the process, correct it, and draw a fresh sample. Rechecking only the pages that already failed cannot show whether the rest of the batch is now reliable.
- Stratify by failure behavior
- Set thresholds before review
- Resample after corrective action
QA layer
| Layer | Purpose | Example |
|---|---|---|
| Preflight | Prevent invalid input | Missing page or unsupported file |
| Automated rule | Find suspicious output | Empty page or invalid total |
| Random sample | Estimate ordinary defect rate | Pages selected within every stratum |
| Targeted review | Protect consequential data | Names, dates, identifiers, totals |
A sample supports a bounded claim
Passing a sample does not prove that every page or field is correct. The result applies only to the represented source classes, review rules, and period covered by the plan.
How the evidence informs sampling
Quality guidance supports risk-based and randomized checks. Collection owners must define strata, acceptance thresholds, escalation, and fresh sampling after a systematic correction.