Causal caution
Observed deviations have multiple competing explanations
A missing field can originate in source conversion, section or chunk construction, candidate generation, thresholding, duplicate resolution, schema mapping, normalization, validation, or evaluation. A supported final value can similarly originate from direct detection or later backfill. Diagnosis therefore requires stage-level lineage, not inspection of final JSON alone.
HNLP deviations
Unrecognized headings, missing localized aliases, dictionary gaps, misspelling, OCR corruption, column flattening, overbroad ignored sections, and dominance thresholds.
BERT deviations
Model/tokenizer mismatch, language shift, chunk boundaries, label competition, low score, sanitizer rejection, overlap replacement, schema constraints, grouping, or backfill.
LLM deviations
Prompt interpretation, context dilution, quantization, decoding variability, reasoning mode, unsupported normalization, malformed serialization, and model-version drift.
Deployment decision
Speed, precision, implementation effort, and distributional stability
| Criterion | HNLP | BERT / GLiNER | Generative LLM |
|---|---|---|---|
| Latency mechanism (not measured) | Rules and dictionaries | Chunked encoder inference | Autoregressive generation and model serving |
| Repeatability controls (not ranked) | Fixed code and resources | Fixed model, tokenizer, runtime, and mapper | Fixed decoding, server, prompt, and model version |
| Implementation effort | Low initially in-domain; continuous rule and dictionary maintenance | Model packaging, labels, calibration, mapper, and runtime | Prompting is quick; production controls remain substantial |
| Cross-dataset stability (not measured) | Test sensitivity to headings, language, layout, and vocabulary | Test model/domain, target span form, and label transfer | Test semantic adaptation against prompt/model drift |
| Auditability | Strong rule-level traceability | Span and score traceability plus opaque representations | Lower internal transparency; source-span evidence is essential |
Controlled next study
Factorial benchmark required for causal claims
Use one PDF conversion, one schema, one blinded human-adjudicated atomic gold set, one matcher, and one missing-output policy. Randomize model and reasoning conditions; repeat stochastic runs; hash every artifact; retain HNLP line scores and state transitions; retain BERT raw logits, chunks, candidates, mapping, and backfill; and report latency, memory, energy, and cost on identical hardware.
Conclusion
A governed field-routed ensemble is the defensible engineering outcome
Use HNLP for high-throughput fields with stable headings and vocabulary; BERT/GLiNER for calibrated compact entities; an LLM for unusual structure and implicit relations; and human review for low-confidence, conflicting, or high-impact fields. Validate every accepted fact against the PDF. This routing policy makes the speed–precision–coverage trade-off explicit instead of hiding it behind a pooled score.