The Utility of Adaptive Domain Models

Yi Ma’s discussion of inherited structure and individual learning provides a useful starting point for our Adaptive Domain Model design. An inherited structure can establish the frame within which later experience changes a model. Our engineering question is what that frame can guarantee and how an update can preserve those guarantees while learning from new evidence.

An ADM is intended to carry applicable domain knowledge through training, compilation, and deployment. Physical dimensions constrain legal quantity relationships. Permitted component support constrains which parameters exist. Equivariance and conservation require their own stated laws and preservation arguments. Together, these can define an admissible model space. A Bayesian version additionally specifies a prior within that space and a likelihood relating parameters to observations.

The constellation vision connects these domain models to a language component and to one another. Existing neural architectures already incorporate structure; our proposed contribution is to make specified domain contracts available throughout the model’s lifecycle. The Resonant Recurrent Model explores how a language component could consult and integrate their results. The quality and cost of that integration remain questions for implementation and evaluation.

Foundational Points

Our ADM pre-print motivates four parts of the design developed here.

Structure has an explicit representation. Dimensional relationships, parity, and permitted blade support convey different facts. A representation can omit forbidden components and expose the operations allowed on the remaining ones. Grade describes the algebraic decomposition, while the compiler needs the particular support and operations that justify a restriction. Properties such as equivariance require a specified group action and a law for the layer; they do not follow from a grade label alone.

Training must preserve the declared constraints. An update formed from tangents in a permitted linear parameter subspace can stay in that subspace. Nonlinear constraints, such as unit-rotor normalization, require an appropriate parameterization or a separately justified update. Forward-mode differentiation supplies directional derivative information. Quire accumulation can make covered sums of represented products exact within its capacity; the surrounding arithmetic, final rounding, and statistical approximation retain their own obligations.

Structural zeros can remove work. A block-diagonal generator has a block-diagonal exponential. If the representation and its operations preserve that block decomposition, off-block parameters need no storage or arithmetic. The block structure must be established for the chosen operator; it is not a consequence of every bivector or grade restriction. A sound support analysis may also retain terms that happen to cancel for particular values.

We expect a representation with an explicitly declared block decomposition to take this shape:

// Dense: every interaction has a parameter slot.
type DenseGenerator = float<1>[,]               // n*n entries

// Illustrative API: BlockLayout declares the decomposition.
// Construction must check that each block satisfies the generator contract.
type BlockGenerator =
    { layout : BlockLayout
      blocks : GeneratorBlock[] }

let exponential (g: BlockGenerator) : BlockTransform =
    let blocks = g.blocks |> Array.map expBlock
    BlockTransform.assemble g.layout blocks    // off-block entries have no slots

Verification records the scope of each guarantee. Supported decidable properties can be discharged by their decision procedures. Other properties need additional witnesses, proofs, or explicitly retained obligations. A candidate’s structural certificate records the properties established under those premises. Prediction quality and statistical calibration require evidence beyond structural verification.

Why this would produce better inference

The proposed gains come from removing unnecessary parameters and computation while preserving the relationships the task requires. Their value depends on the domain, the chosen representation, and the cost of the complete system.

Reliable structural properties. A checked dimensional contract can exclude an invalid quantity combination. A suitable constrained representation and update rule can preserve declared support or a proved symmetry. These guarantees can remain useful across training updates. A structurally valid estimate can still be inaccurate, so the statistical model and its uncertainty estimates need their own evaluation.

Less storage and arithmetic. Omitting interactions that the domain excludes reduces the work the representation needs to express them. That can make a specialized model practical on smaller hardware. The comparison must include training state, posterior state, numerical precision, and checking costs, as well as inference parameters. Compactness alone does not establish better accuracy or lower latency on a particular target.

A division of labor. A language component can route a structured question to a domain model and integrate its result. This may reduce the specialized work assigned to the language component. Routing quality, communication, synchronization, and the model sizes actually required determine whether the complete application becomes faster or cheaper. The constellation needs measurements of those costs alongside the domain models’ predictive performance.

Reporting uncertainty

A probabilistic ADM would return an estimate together with a defined account of uncertainty. A Bayesian credible interval summarizes uncertainty about a parameter or derived quantity under the specified model and data. A posterior predictive interval concerns a future observation and includes the observation model’s variability. These have different meanings from a frequentist confidence interval. Stan’s posterior-interval and predictive-interval documentation makes the distinction concrete.

Producing those summaries requires a prior, likelihood, posterior representation, and inference algorithm. A Gaussian approximation is one possible choice where its assumptions and error are justified. A constrained domain need not have a Gaussian posterior, and an approximation must respect the relevant support. Multimodal or nonlinear problems may require a different family or a richer summary.

The caller needs to know which quantity the interval describes, its units, its probability level and construction, and the model and evidence on which it depends. Calibration must be assessed against the intended operating conditions, including distribution shift. A structural certificate establishes its named invariants; it does not establish that a reported interval has reliable empirical coverage.

The connection to Ma’s representation-learning program is a research comparison: both motivate making useful structure available to inference. A typed admissible space and a learned data manifold need an explicit correspondence before their operations can be identified. Our design must supply the probability model and computation on its chosen space.

Gaussian elimination has a separate role in our verification architecture: it can solve the supported linear equations among dimensional exponents. It does not construct a Gaussian posterior or certify an arbitrary geometric manifold. Support, equivariance, and statistical inference use the procedures appropriate to their respective obligations.

Updating as evidence arrives

Continued Bayesian updating is established methodology; Streaming Variational Bayes, for example, develops streaming and distributed updates to an approximate posterior. Our proposed integration adds typed domain contracts, evidence provenance, candidate validation, and a coherent transition between deployed model versions.

New observations can trigger a candidate update under the specified observation model. A discrepancy or distribution-shift signal does not establish that the candidate will improve predictions. Changing regimes may also require a dynamic parameter model or a revised adaptation policy, rather than accumulating all history under a stationary assumption. Candidate acceptance must consider predictive performance and uncertainty quality as well as the structural and resource contracts.

Warm rotation supplies the intended operational boundary: requests observe one coherent model version while a validated candidate is installed. This supports learning beyond the initial training run. Avoiding forgetting, maintaining calibration, and improving under changing conditions remain properties to evaluate for the chosen update procedure.

Where the Language Model Fits

The language component interprets requests and routes them to domain models with explicit interfaces. Natural-language intent can be underspecified, so producing a well-typed query does not establish that it captures the user’s intended question. The routing and interpretation procedures need their own checks and evaluation.

A language model can carry syntax, interface, and resource constraints. Those contracts do not amount to a complete specification of the correctness of its open-ended answers. The constellation combines those useful boundaries with the stronger domain-specific obligations that individual specialists can support. Whether this permits a smaller language component at the required quality is part of the architecture’s empirical case.

The rest of this section

With ADM and its utility argument in place, the rest of the section examines the language node that completes the constellation, from several independent angles that can be read in any order. A Scaffold for Constrained Models names the three commitments that carry a language component when the ADM type scaffold cannot apply, and carries the argument from the domain models to the language node. Building a Constrained Language Model treats its tuning and the deterministic layer that bounds its output. Architecture and Arithmetic and Forward-Mode and Low-Rank Adaptation treat the substrate that makes it precise and cheap. The Constellation returns to this article’s central claim and shows the porous node and the domain models composed into one system. Reversible Cores and Inference-Time Recall reframes the framework’s negative and fractional types as a design theory of reversible computation inside a model. And Adapting Inference on a Gradient is the adoption side, how an organization runs the constellation today and adapts toward a built model along a gradient. Last, The Ultimate Constraint reads the section against its limit case, fully homomorphic encryption, where the constraints developed here become the price of admission. Each is speculative and marks its own open questions.