ive been thinking about this after running evals on our compliance screening setup. the accuracy gap between purpose built compliance infrastructure and general LLM plus RAG is bigger than most people expect and the reasons r worth understanding.
the obvious answer is corpus quality which is real but its not the whole story.
the less obvious one is that compliance reasoning is scenario specific. a general model with reg context will reason about whether something seems compliant. a purpose built system reasons about whether this specific content violates this specific rubric under this specific regulatory framework. the latter requires the model to be scoped in ways generic prompting doesnt enforce.
the other one is citation validation. getting an LLM to produce a citation is easy. getting it to produce a citation that points to an actual current section of the regulation that actually supports the flag it raised is hard. a bad citation looks like this: violates 12 CFR 1026.17 with no subsection, or worse cites a section that governs a different product type entirely. a reviewer who checks that citation loses trust in the entire output immediately. a good citation points to the exact subsection, matches the applicable standard, and can be verified in under 30 seconds. generic RAG pipelines produce hallucinated or stale citations at a rate that makes reviewer trust collapse fast and post-LLM validation that checks the cited section actually exists and says what the model claims it says is a separate engineering problem most teams dont build.
we hit that and moving to midlyr ai for the screening layer, reviewer trust collapsed every time citations were off and no amount of prompt tuning fixed it reliably. the result is that general purpose approaches work fine in demos where someone is checking the output manually. in production where a reviewer is making decisions based on the flag and the citation the accuracy gap becomes a trust problem fast.
purpose built infrastructure isnt a magic. its just doing the scoping and validation work that generic approaches leave to the model.