Should LLMs handle abstraction only in optimization?
What if LLMs worked exclusively on translating problems to formal constraints, while deterministic solvers handled the numeric work? Explores whether this division of labor could overcome LLM failures in iterative computation.
If LLMs cannot reliably execute iterative numerical procedures, the design question becomes what they can reliably do in optimization workflows. The constraint-optimization paper's answer: keep them at the abstraction layer. Have the LLM read the structured input (grid topology, financial statement, threat model), understand the physics or domain, deduce the correct constraints, and write the mathematical formulation into solver code. Then hand off to the solver for the actual numeric matrix arithmetic.
The division of labor exploits the comparative advantage on both sides. LLMs are strong at natural-language-to-formal-structure translation, at picking the right variables and constraint forms from context, and at producing code that calls established libraries. They are weak at the iterative arithmetic that follows. Deterministic solvers (interior-point methods, primal-dual, branch-and-bound) are strong at the iterative arithmetic and weak at parsing the messy real-world problem statement. The architecture is the bridge.
This pattern recurs across domains. The same logic argues for neuro-symbolic approaches in general: LLM as translator from human problem to formal representation, symbolic engine as solver of the formal representation. The paper situates the constraint-optimization case within this family but argues the LLM should be restricted more aggressively than typical neuro-symbolic systems — not just translating but explicitly forbidden from attempting the numeric work.
For builders, the operational rule is: any task where the bottleneck is iterative computation should route through abstraction-only LLM use plus a solver, not through more capable LLMs or longer reasoning. Wrapping the LLM in a thinking budget is the wrong knob; wrapping it in a solver call is the right one.
Related concepts in this collection
-
Do large language models actually perform iterative optimization?
Explores whether LLMs execute genuine numerical procedures like Newton-Raphson or instead pattern-match to memorized solution templates when solving constrained optimization problems.
same paper, the failure mode this paradigm avoids
-
Do larger language models solve constrained optimization better?
Explores whether scaling LLMs—through more parameters, better training, or reasoning extensions—improves their ability to satisfy constraints in real optimization problems like power grids and portfolios.
same paper, the empirical reason for the paradigm
-
Can large language models translate natural language to logic faithfully?
This explores whether LLMs can convert natural language statements into formal logical representations without losing meaning. It matters because faithful translation is essential for any AI system that reasons formally or verifies specifications.
partial counterpoint: even the abstraction step has known failure modes
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
Original note title
the right paradigm for LLM optimization is abstraction-only — read input deduce constraints emit solver code