Can partial formal verification work without full formalization of language semantics?
This explores whether you can capture the rigor of formal verification — catching errors, enforcing completeness — without translating language all the way into symbolic logic, and where that partial approach beats both pure prose and full formalization.
This explores whether you can get the discipline of formal verification without paying the full cost of formalizing language semantics — and the corpus has a surprisingly direct answer: not only can partial formalization work, it often works *better* than going all the way. The core finding is that full formalization throws away information. When QuaSAR and Logic-of-Thought enrich natural language with selective symbolic structure rather than replacing it, they gain 4-8% accuracy, because pure language lacks structure while pure logic loses semantic nuance — the middle keeps both Why does partial formalization outperform full symbolic logic?. The interesting move is that augmentation, not substitution, is the win condition.
A second line shows how this works in practice for code reasoning. Instead of symbolic rigor, you can use natural-language templates that *enforce the discipline* of formal methods — forcing a model to consider every case, support every claim, resist confirmation bias — without ever formalizing what the language means Can structured templates replace formal verification for code reasoning?. This 'semi-formal' scaffolding crosses real reliability thresholds: execution-free verification of code patch equivalence hits 93% accuracy, good enough to serve as a reinforcement-learning reward signal where you'd normally need to actually run the code Can structured reasoning replace code execution for RL rewards?. The discipline, it turns out, lives in forced completeness more than in symbols.
There's also a complementary inversion worth knowing: you don't always have to choose where the formalization goes. The interwhen work auto-synthesizes *provably correct* formal verifiers (in Lean and z3) straight from prose policy documents — letting the LLM do the messy translation while a small, rigorous checker polices the output Can we automatically generate formal verifiers from policy text?. And those verifiers can run asynchronously alongside generation, catching violations with near-zero latency on correct runs Can verifiers monitor reasoning without slowing generation down?. So partial verification isn't just a quality compromise — it's an architecture where rigor and fluency each do the part they're good at.
Why does any of this matter beyond a few accuracy points? Because the corpus also argues that *some* external verification is not optional. Hallucination is formally inevitable for any computable LLM — internal self-correction can't eliminate it, so external safeguards are mathematically necessary Can any computable LLM truly avoid hallucinating?. Self-improvement runs into the same wall: models are bounded by a generation-verification gap and can't validate their own fixes from the inside What stops large language models from improving themselves?. Partial formal verification is attractive precisely because it supplies that external check cheaply, without demanding you formalize everything first.
The thing you might not have known you wanted to know: a lot of what looks like a 'reasoning' failure is really a failure to *enumerate* — models stumble not from missing knowledge but from never bringing the relevant unstated preconditions forward, and simply forcing explicit enumeration lifts accuracy from 30% to 85% Do language models fail at identifying unstated preconditions?. That's the same mechanism the semi-formal templates exploit. So partial verification may work not because symbols are magic, but because the real value of formalism was always the forced completeness — and you can get that with structured prose.
Sources 8 notes
QuaSAR and Logic-of-Thought both achieve 4-8% accuracy gains by enriching natural language with selective symbolic elements rather than replacing it. Full formalization loses semantic information; pure language lacks structure. Augmentation preserves both.
Semi-formal reasoning using natural-language templates enforces the discipline of formal methods without formalizing language semantics. Templates prevent case-skipping, unsupported claims, and confirmation bias—capturing the verification benefits of formalism through forced completeness scaffolding rather than symbolic rigor.
Semi-formal reasoning templates enable execution-free patch equivalence verification at 93% accuracy on real agent code, crossing the reliability threshold needed for RL reward signals. This makes execution-free verification viable for certain task classes like fault localization and code reasoning.
interwhen automatically generates code-based verifiers—including provably correct Lean and z3 checkers—from prose policy documents. This inverts the usual neuro-symbolic division: the LLM both translates policy to formal logic and extracts verifier inputs from reasoning traces.
Decoupling verification from generation lets verifiers run alongside a single trace, forking to extract verifiable state and intervening only on violations. On correct runs the latency penalty is near-zero; interwhen matches or beats CoT across benchmarks at similar token budgets.
Three formal theorems prove that any computable LLM must hallucinate on infinitely many inputs, and internal mechanisms like self-correction cannot eliminate this mathematical constraint. External safeguards are therefore necessary, not optional.
Self-improvement in LLMs is formally bounded by the generation-verification gap, meaning every reliable fix requires something external to validate and enforce it. Models cannot escape this constraint through metacognition alone.
LLMs struggle not from lacking world knowledge but from failing to bring background conditions forward as relevant constraints. Prompting that forces explicit enumeration of preconditions raises accuracy from 30% to 85%, revealing the frame problem persists in statistical systems.