Dictionary plausibility is weak evidence for identity-bearing text
Review identity-bearing text character by character. Dictionary plausibility is weak evidence for a name, code, citation, or unfamiliar word.
The review should preserve identity-bearing spelling and visible marks exactly enough that a dictionary correction cannot quietly change a person, place, identifier, citation, or unfamiliar term.
Put the procedure into practice with Image to Text. Continue with How to Choose an OCR Language Model and Numeric OCR Validation for Dates, Totals and Identifiers.
For contact details where one accent or digit changes the record, use the reviewed business-card to vCard workflow.
Names and punctuation review
- List protected names and tokens
- Check every visible diacritic
- Document Unicode normalization
- Resolve line-end hyphens individually
- Disable automatic correction for identifiers
Inspect visible marks and code points character by character
Create a protected-token list
Before batch review, list names, places, organizations, product codes, citation styles, and expected punctuation patterns. Flag variants rather than automatically normalizing them.
Inspect combining marks
What looks like one accented character can be represented as a precomposed code point or a base letter plus combining mark. Choose a normalization policy for searching while preserving a source-faithful transcription where required.
Resolve line-end hyphens
Distinguish a real compound or identifier hyphen from a word broken across printed lines. Keep page and line coordinates until the decision is made; do not delete all end-of-line hyphens globally.
Separate typography from identity
Straight and curly quotes may be safely normalized for some text uses, while exact diplomatic transcription may require them. Names, passwords, paths, and identifiers require stricter preservation than prose punctuation.
Review identity-bearing text at character level
Select names containing accents, apostrophes, hyphens, initials, or characters uncommon in the dominant document language. Compare visible marks one by one and check the resulting Unicode characters when data will enter a database or identity workflow.
Do not let spell-check normalize a name into a common word. Verify punctuation that changes identifiers, email addresses, citations, measurements, or legal names. Keep the source crop with the reviewed value when downstream users may need to resolve font ambiguity or combining-mark differences.
- Disable silent normalization
- Inspect code points when identity matters
- Retain the source crop for ambiguity
Token review priorities
| Token type | Common OCR change | Safe control |
|---|---|---|
| Personal name | Changed toward common word | Source-by-source review |
| Accented word | Mark removed or attached incorrectly | Code-point and visual comparison |
| Identifier | O and 0 or I and 1 swap | Format rule plus source check |
| Line-break hyphen | Always kept or always removed | Use word and page context |
Spell-check can make identity text worse
Dictionary normalization may replace a rare but correct name, remove a diacritic, straighten meaningful punctuation, or alter an identifier. Visual fidelity outranks linguistic plausibility for these tokens.