There is a failure mode in scientific software that large language models make dramatically easier to reach: producing a number that is wrong in a way nobody notices. A simulation that crashes is a bug report. A simulation that returns a plausible value derived from a parameter the model quietly invented is a paper.

We build an in-silico research platform where investigators describe experiments in plain language and the system runs them. The design constraint that has held up across every iteration is a boundary rule: the model may translate language, and it may not compute. Everything numerical happens in code that was written, reviewed, and versioned by a human.

Enforcing that rule turns out to require three separate gates, because the model finds three different ways around it.

Gate one: intent to parameters, with a closed vocabulary

The model’s job at the front door is to turn a sentence into a structured request. Not to plan the experiment — to fill in a form.

The form is a strict schema: named parameters, declared types, explicit units, enumerated options where options exist. The model emits an instance of that schema and nothing else. Free-text fields are limited to things that never reach the solver, like a run label.

Two rules make this gate hold:

Reject rather than coerce. If the model produces a value outside the allowed range, or an option that is not in the enumeration, the request fails and comes back to the user as a question. It does not get clamped to the nearest legal value. Silent coercion is how a request for something the system cannot do becomes a result for something adjacent that nobody asked about.

No derived values. The model may not compute a parameter from another parameter, even when the arithmetic is trivial. If a user gives a concentration and the solver wants a rate, the conversion lives in code with the constant checked in. A model doing unit conversion inline is a model doing science, and it will get it right often enough that you stop checking.

Gate two: validation against the physical model, not just the schema

Schema-valid is not the same as meaningful. A parameter set can satisfy every type and range constraint and still describe a system that does not exist — mutually exclusive conditions, a timescale mismatched to the process being modeled, an initial state the solver’s assumptions do not cover.

The second gate is domain validation, and it is deliberately not the model’s job. It is a rules layer that encodes what the underlying scientific model is actually valid for: the combinations that are contradictory, the regimes where the numerics stop being trustworthy, the preconditions the solver documents but does not check.

This is the least glamorous component in the system and the one that has caught the most real problems. It is also the one that must be written by whoever understands the science, not whoever understands the framework. Asking a model to generate its own domain constraints produces a plausible-looking list with the important entries missing.

Rejections from this gate are returned with the specific rule that fired, in the user’s language. “That combination is outside what this model covers, because X” is useful. “Validation failed” sends the user back to the prompt to try rephrasing until something gets through, which is exactly the behavior you do not want to reward.

Gate three: results back to language, with no room to editorialize

The third gate is where the temptation is strongest and the discipline matters most. The solver produces numbers. Someone has to explain them.

The rule: the model may describe outputs it was given. It may not produce, adjust, round, extrapolate, or interpolate a number, and it may not characterize significance.

In practice that means:

• Every figure in the narrative is a template substitution from the result object, not text the model typed. If a number appears in the output, it can be traced to a field.
• Statistical characterization — significance, confidence, effect size — is computed upstream and passed in. The model reports what it was handed.
• Comparisons against prior runs come from a diff computed in code, not from the model reasoning about two result sets in its context.
• The narrative always ships with the parameters that produced it, so any reader can regenerate the run.

Restricting the model this severely costs some fluency. Output reads a little more like a report and a little less like a colleague explaining their findings. That is the correct trade. Fluency is what makes a wrong number persuasive.

What the boundary buys

Reproducibility. Every result traces to a parameter set that traces to a validated request. Rerunning is deterministic because nothing in the numerical path depends on a sampled token.

Auditability. When a result is questioned, the question has an answer that does not involve interpreting a model’s reasoning. The parameters are in the record and the code is in version control.

A model you can swap. Because the model is confined to translation at both ends, upgrading it, changing providers, or running locally is an evaluation exercise on translation quality rather than a revalidation of the science. Our inference runs on local hardware for exactly this reason — the boundary makes that a cost decision instead of a correctness decision.

Honest failure. The system’s failure mode is refusing a request it cannot handle. That is annoying and it is survivable. The alternative failure mode — quietly answering a slightly different question — is neither.

The general form

The specifics here are a research platform, but the pattern is not domain-specific. Anywhere a language model sits in front of a system that produces consequential numbers — financial modeling, engineering analysis, clinical decision support — the same decomposition applies:

1. Constrain input to a schema the downstream system already validates, and reject rather than coerce.
2. Validate semantically against what the system is actually good for, in rules written by domain owners.
3. Let the model narrate outputs it was handed, and never let it produce or characterize a figure.

The model is very good at language. The reason to keep it there is not distrust — it is that language is where its errors are visible and cheap, and arithmetic is where they are invisible and expensive.