Large language models (LLMs) have made extraordinary progress in understanding natural language, generating fluent responses, and powering entirely new customer and employee experiences. However, because LLMs are inherently stochastic, using them for complex numerical or logical reasoning offers limited guarantees that their outputs will be compliant or optimal. For example:
A manager tells an AI planning assistant, “Ship as much as possible next week while keeping costs low,” and the system delivers an aggressive plan that looks efficient but silently exceeds warehouse capacity and breaks delivery guarantees.
A coordinator tells an AI assistant, “Reassign crews to reduce overnight stays and minimize disruption,” and the model produces a lower-cost schedule that looks optimal but violates mandatory duty-time or rest constraints.
A financial operations AI assistant must approve transactions under strict regional and risk constraints, but it clears a suspicious transaction by inventing a rationale that sounds compliant while violating internal policy.
In environments with strict operational requirements, combining LLMs with formal solvers helps ensure that natural-language-driven decisions stay within defined boundaries and avoid infeasible, suboptimal, or noncompliant outcomes.
Our R&D team is investigating a hybrid approach that combines the creativity and flexibility of LLMs with the rigor, guarantees, and transparency of formal mathematical solvers such as Z3, Pyomo, and OR-Tools. We’re also building a reusable formal AI engine to make this capability a standard part of the enterprise technology stack. The platform will enable organizations to ingest business rules directly from their existing systems, continuously verify decisions against those rules, and deploy AI agents safely within clearly defined boundaries.
Our vision is to reduce operational risk while accelerating decision‑making at scale. Leaders gain faster, policy-compliant decisions from natural-language inputs, while organizations can automate ad hoc changes in scheduling, supply-chain resource allocation, financial operations, policy verification, and even video or 3D design. Built-in safeguards prevent infeasible, noncompliant, or unsafe outcomes from reaching production.
In controlled experiments across logistics-style optimization problems and three public benchmarks, our hybrid approach consistently demonstrated:
Greater accuracy
Greater interpretability and auditability
Our approach flips the usual “LLM does everything” paradigm and instead follows this design:
This approach clearly separates responsibilities: LLMs extract rules and constraints from natural language, while deterministic solvers such as OR-Tools, Z3, and Pyomo handle optimization and verification. The result combines the strengths of both approaches:
LLMs | Solvers | Hybrid (LLM + Solver) | |
|---|---|---|---|
Understand human intent across different domains | ✅ Excellent | ❌ None | ✅ Excellent |
Deterministic behavior | ❌ No | ✅ Guaranteed | ✅ Yes |
Provably correct in mathematical reasoning and optimization across multiple constraints | ⚠️ Fragile | ✅ Guaranteed | ✅ Yes |
Auditability and interpretability | ⚠️ Fragile | ✅ Clear | ✅ Clear |
Resistance to prompt noise and injection | ❌ Vulnerable | ✅ Immune | ✅ Strong |
We started with a challenge that some of our clients face:
Turning natural-language requests into optimal, real-time resource decisions while strictly enforcing operational, policy, and cost constraints.
This challenge sits at the heart of modern logistics and supply chains, including workforce scheduling, asset allocation, routing, fulfillment planning, and capacity management. It is also where LLMs and formal solvers need to work together to produce trustworthy systems. For our evaluation, we constructed controlled test scenarios, data sources, and constraints, along with 240 synthetic queries. Examples include:
Please revise the existing schedule following a cancellation. The target facility must be fully supplied by 24 December 2025. When possible, source inventory from a nearby warehouse and route it through a specific consolidation point. The earliest allowable start date is 14 December 2025.
Last-minute request: a VIP will arrive at location A in three hours. We need the required staff on site within two hours. Please adjust staff allocations while minimizing changes to the existing schedule.
We can see from the queries that the system must reliably extract and enforce hard and soft constraints directly from natural-language requests:
Hard constraints are nonnegotiable (e.g., earliest start dates, contractual terms, and capacity ceilings). Violating any one of them invalidates the solution.
Soft constraints encode preferences, such as minimizing delays, reducing costs, and limiting changes. The objective is to optimize without crossing hard boundaries.
Some aspects of these optimization problems cannot be fully predefined. They must be assembled dynamically by drawing not only on predefined constraints and objectives from structured business logic, internal documents, and operational data but also on the user’s request.
To understand how different techniques perform in this setting, we implemented and compared three approaches.
Pure LLM: The simplest approach passes all relevant data and the user’s natural-language request to a single LLM prompt, which is asked to produce an optimal plan or allocation. While this can work for small or loosely constrained problems, it breaks down as complexity increases. The model may ignore constraints, prioritize the wrong objective, or produce plans that sound reasonable but are infeasible, with no reliable way to detect or prevent failures.
LLM + Code Interpreter: In this approach, the LLM interprets the request and uses tools to access data sources and generate executable optimization code. This adds flexibility and observability, but reliability remains an issue. The LLM must still translate constraints into correct code, and small reasoning or coding errors can produce invalid or suboptimal outcomes, especially as constraints scale.
Hybrid: LLM → structured constraints → deterministic solver: The third approach separates responsibilities. The LLM never “decides” the outcome; it helps formalize the variables, constraints, and objectives. A proven optimization solver enforces constraints, guarantees feasibility, and produces results that can be verified and audited. The LLM’s role is limited to translating natural-language requests into explicit, structured constraints using predefined schemas. Those constraints are automatically compiled into solver code, such as OR-Tools, which deterministically computes a feasible, optimal solution.
We tested the methods using several LLMs, including GPT-5, GPT-5.1, and GPT-5.2. As expected, the hybrid approach outperformed the alternatives:
Method | Assignment accuracy | Average latency | Tokens used per query |
|---|---|---|---|
Pure LLM | 70–78% | 62–190 s | ~175,000 |
LLM + Code Interpreter | 82–84% | 62–140 s | ~9,000 |
LLM → Solver (Hybrid) | 95–97% | 6–25 s | ~2,000 |
Our hybrid method demonstrates a substantial increase in accuracy, more than 4× greater token efficiency, and an order-of-magnitude reduction in latency.
Our next step is to build a generalizable, reusable interface that converts natural language into structured semantic representations, which our backend can translate into solver-ready code. For this experiment, we focused on linear optimization problems and evaluated the approach against three public datasets (NLP4LP, NL4OPT, and IndustryOR).
Standalone LLM
Hybrid method (LLM → structured rules → solver → verified output)
Use an LLM to extract variables, constraints, and objectives from the input and formulate a structured optimization problem.
Pass the structured problem and the original request to an LLM for self-verification.
Translate the structured problem into OR-Tools code to compute the optimal assignment.
We evaluated this approach using proprietary frontier models, including GPT-5.1, GPT-5 mini, GPT-5.1-Codex-Max, and GPT-5.2, alongside open-source models such as Kimi K2, GPT-OSS models, and MiniMax M2. The box plots summarize the results for each method.
The hybrid approach consistently produces more accurate, stable, and verifiable results than a standalone LLM baseline across almost all the underlying language models evaluated. Although absolute performance varies across models, the relative gains from the hybrid approach remain consistent. This suggests that the improvements come from separating natural-language understanding from formal optimization rather than relying on any single model’s reasoning ability.
Accuracy
On NLP4LP and NL4OPT, which primarily consist of linear programming problems, the hybrid method achieves near-ceiling accuracy and outperforms standalone LLM prompting. Rather than producing “approximately correct” reasoning, the hybrid system more consistently generates valid, well-formed mathematical formulations. On the more challenging IndustryOR dataset, accuracy decreases for both methods, but for different reasons. Many IndustryOR problems involve combinatorial structures, such as vehicle routing, task sequencing, and workforce assignment, that exceed the linear optimization capabilities currently supported by our solver backend.
Analysis of the failure modes shows that standalone LLMs frequently violate required constraints, as illustrated below:
Example 1:
Plain Text
The standalone LLM produces a solution with lower total fat, but it violates the requirement that turkey dinners account for no more than 40% of the meals. The hybrid approach correctly enforces this hard constraint and returns a valid answer.
Example 2:
Plain Text
The standalone LLM again produces a solution with lower discharge, but it exceeds the maximum permitted pain-medication limit. The hybrid approach correctly enforces that hard constraint and returns a valid answer.
Latency and token usage
The latency and token-usage data reveal an important distinction. The hybrid approach has higher average latency and token usage than a single-shot LLM prompt, but this reflects architectural choices rather than inefficiency.
The hybrid pipeline includes:
One or more LLM calls to extract structured variables, constraints, and objectives.
A self-verification step to catch internal inconsistencies.
Although these steps add overhead compared with a single prompt, latency remains bounded and predictable, and solver execution is typically fast once the problem is well formed. The additional work creates explicit, reusable, and auditable intermediate representations. By contrast, standalone LLM approaches compress reasoning into one opaque generation, shifting costs to retries, manual checks, and downstream failures. Future iterations can reduce this overhead through:
Caching extracted schemas.
Updating constraints incrementally.
Improving prompt and call orchestration.
Transparency and auditability
Finally, even in cases where both methods fail, the failure mode itself is fundamentally different.
With a standalone LLM, failures are often silent: the model may return a result that is subtly incorrect.
With the hybrid approach, the explicit problem formulation makes failures transparent and helps teams identify which part of the formulation caused an error.
Future versions could expose these formulations in an interface, allowing users to audit or verify them before the solver runs. This transparency improves measured accuracy and makes the system easier to debug and refine, which is essential for real-world deployment.
Key takeaway
Across all benchmarks and most of the underlying models tested, the results reinforce a central conclusion of our work:
LLMs are powerful at understanding and translating intent, but deterministic solvers are essential for enforcing correctness.
The hybrid approach transforms natural language from a source of ambiguity into a reliable interface for mathematically sound decision-making, bringing enterprise AI one step closer to systems that are not only intelligent but also trustworthy.
Enterprise constraints rarely exist in neat schemas or perfectly phrased prompts. They are scattered across databases, spreadsheets, internal policies, and contracts. User requests may be incomplete, ambiguous, or inconsistent with business rules. To make this approach work at scale, we’re building a reusable backend engine that turns this complexity into a reliable enterprise capability.
At its core, this engine acts as a formal backbone for AI-driven decision systems, providing:
A symbolic knowledge base with adapters that ingest business rules, constraints, variables, and objectives.
A translation layer that compiles schemas into solver code.
Interfaces that allow teams to inspect, audit, and modify constraints.
Although these experiments currently focus on optimization, the same method can extend to logical verification. Potential business applications include:
Perform ad hoc dynamic scheduling, routing, and resource allocation while enforcing every operational constraint.
Generate answers and recommendations that consistently respect business rules.
Design and validate complex 3D objects, videos, and architectures while catching infeasible designs before production.
AI today is powerful, but enterprises need more than power: they need correctness, consistency, and control. Our hybrid LLM-and-solver system is a step toward a world where:
Agents do not hallucinate rules or constraints.
Logical deduction and optimization are mathematically sound.
Natural language serves as the universal interface to deterministic systems.