Retrieving Fidelity

Ask a question about a system whose information base is constantly growing and the research quickly fans out. A design document explains an architectural choice. The implementation establishes what currently happens, while a test records the conditions it was designed to check. A second repository may contain the generator responsible for the code under review. We need enough of those connections to answer a deep question, with room left to consider follow-on review and instructions.

Knowledge compression takes several forms along that route. A language model encodes learned patterns in its weights. An embedding gives us a compact representation for finding similar material, and a summary condenses an argument. A knowledge graph retains selected entities and their relationships, with explicit paths back to the evidence. Language models have “swallowed all of the air in the room” for a few years, but now other time-tested knowledge compression patterns are showing their value as we encounter the practical limits of transformer-based architectures.

In our work we have a need to follow questions across the Clef language and the Fidelity Framework as both continue to develop. A question about Composer can lead through a language specification into a native binding and its generation profile. We want local models to help with that research, including models running with modest memory and a limited context window. Their usefulness depends heavily on the information they can use within those constrained environments.

Graph databases give us a practical form of neurosymbolic reach: a model interprets the question, a graph query follows recorded relationships, and the next model works from the resulting evidence. Much of the “intelligence” is in how we organize the garden-variety work of building the graph and conditioning how it’s accessed by transformer models. Our discussion of construction and convergence considers this role for structure around a learner, and is based on a considerable body of work from previous decades of research.

The service we have built now has a name: Fanion. Its current F# implementation uses explicit context passing with Oxpecker handlers, Ranvier for pull-based readiness and corpus scope, and Fidelity.FSharp.Incremental for queued work and shared demand. A separate AI agent sends a short text question to the provided endpoint and after a short turn waiting receives cited JSON. Fanion contains all of the steps in between.

The resource now runs a paired source build of DuckDB 2.0 and DuckPGQ across a 24-repository corpus. The deployed engine identifies itself as v2.0.0-dev1, a prerelease development build; we pin both source revisions and the resulting native library. DuckPGQ is built into that library, and each connection activates its grammar before graph queries run. The engine notes record the build, indexing measurements and acceptance evidence.

A Finite Window on a Growing Corpus

As anyone in their burgeoning field has seen, the expectation of an unbounded context window is an expensive answer to a selection problem. As the corpus grows, so does the work of distinguishing an old proposal from a current implementation. Even a reader with every document open needs to establish which revision and relationship support the answer.

We can write the immediate constraint as a token budget

Here $S$ is our selected evidence and $C$ is the available context. Every source excerpt consumes part of the same budget as the instructions and eventual answer. In one of our local parser evaluations, the runtime had an 8,192-token context allocation. That is plenty of room for a focused graph task when the gateway supplies a compact schema and relevant candidates.

Larger windows give us more capacity, but at a cost. The Lost in the Middle experiments found that the tested models often retrieved information more successfully near the beginning or end of their context than in its middle. We take that result as a reason to measure retrieval behavior alongside advertised window size. Our workers receive enough evidence for a defined task, and can request another bounded slice when the task requires it. Liu et al., 2024

The compression we want is question-dependent. A source file may contribute three relevant lines to one investigation and several complete functions to another. Keeping the citation attached lets us return to the original whenever the shorter representation needs inspection. Our earlier cognitive architecture sketch considers loading only the domain knowledge a query requires.

Knowledge Representation, Again

There was a time when an explicit network of concepts and relationships was comfortably described as AI. We still think it belongs there. In their 1993 account of knowledge representation, Randall Davis, Howard Shrobe, and Peter Szolovits discussed semantic networks alongside other established AI representations. They also asked a particularly apt question for our work: which properties of the original does a representation preserve? Their discussion uses the word fidelity. What Is a Knowledge Representation?

Tomaž Bratanič’s Graph Algorithms for Data Science also informed this work, particularly its treatment of building graphs from structured and unstructured data. Project declarations, source code and prose from repository documentation each contribute different kinds of relationships, and we need to keep those differences visible when we query them.

Our AI Refinery research also considers structured knowledge consultation during inference, with neuromorphic execution as a prospective target. Here the target is much more mundane and the solution pattern well-traveled.

Our retrieval graph starts with a deliberately concrete vocabulary. Repositories contain files. Files have source chunks, and documents link to other documents. Project declarations establish cross-repository references. Additional extraction rules identify likely relationships, retaining the rule and supporting source with each result.

For a particular source snapshot $s$, a compact model is:

The vertices in $V_s$ represent resources. Each edge $(u,\ell,v)$ records a directed relationship with label $\ell$. Properties on those resources identify their repository and revision, down to the source path and line range. Relationship properties record how the connection was established.

  flowchart LR
    R["Repository"] -->|contains| F["Source file"]
    C["Source chunk"] -->|chunks_of| F
    F -->|links_to| D["Related document"]
    F -->|generated_by: source declaration| G["Generator repository"]
    F -.->|generated_from: inferred match| P["Generation profile"]

The arrow from a chunk points to its parent file.

That small detail becomes consequential when we ask for all chunks belonging to a file: the query must traverse the relationship in the incoming direction. The dashed generation-profile edge represents an inferred research lead, with its extraction basis available for inspection. A source declaration and an inferred match carry different evidential weight.

The snapshot behind the original post contained 2,249 files across twelve curated repositories, represented by 8,050 nodes and 11,576 edges. That included 42 capability nodes and 2,031 derived relationships. By October 17, the corpus had grown to 4,230 accepted files across twenty-four repositories. The graph gives us a manageable way to navigate that material while retaining the source-level detail needed to assess a response to a free-text but scoped query.

A question about WrenHello’s native host on Linux, for example, can lead through project references to WebKit and GTK bindings, then to a binding’s declared generator and likely generation profile. Our graph retains the source evidence for each step.

We store it in DuckDB and use DuckPGQ for graph pattern matching. DuckPGQ supplies SQL/PGQ operations over relational data, so graph traversal and ordinary database handling can share a fast, compact execution engine. Our retrieval API exposes a restricted query language over that graph.

Smart Search, on Both Sides of the Tool Call

You can try one part of this arrangement here. Open the site’s search, or follow this search for Composer graph coeffects. Inspect the returned passages, then use Summarize with AI to see a synthesis from the selected material. The Share button preserves the query for another reader.

Our Smart Search combines BM25 keyword retrieval with vector similarity. BM25 is useful when the question contains a specific identifier or phrase. Embeddings help when the question and the relevant passage use different wording. The service runs both searches and combines their rankings with reciprocal rank fusion:

Each $r$ is one ranked result list, and our base fusion uses $k=60$, followed by the site’s recency adjustments. A passage can receive support from either search route or both. RRF combines rank positions, so vector and BM25 scores can retain their own scales. Cormack, Clarke, and Buettcher, 2009

The browser and our agents’ public retrieval tool call the same hybrid search endpoint. An agent receives bounded snippets and source URLs, then can use those results within its assigned task. The browser adds its own optional AI summary.

Using search on this site gives you direct access to the retrieval infrastructure we also provide to our AI workers.

The committed-source graph offers a complementary route. Its initial discovery operation ranks literal matches, giving exact paths and names priority. From a discovered file, a worker can follow a declared relationship and retrieve revision-specific source lines. Public hybrid search finds relevant published discussion, while the local graph supports closer investigation of the committed sources. We expose both tools so the primary model or a worker can select the appropriate route.

A Small Model with a Specific Job

We give the PGQ worker a narrow responsibility: translate a question into a supported graph query using resources that the gateway has actually discovered. The gateway owns the application instructions and graph access. The local inference server receives a complete request, including the output constraints selected for that call.

When a question supplies a qualified file path, the gateway discovers that resource directly. Broader questions use a planning call: the worker proposes up to three literal lookups using the current repository and layer names. The gateway performs those lookups, normally returning four candidates per lookup. An exact-path check keeps a document that mentions a filename from being substituted for the requested file itself.

The query-generation call receives those real candidate identities, together with brief source excerpts. It also receives the relationship directions and worked query examples. Its response contains a query fragment and parameters. The gateway validates that response and executes the accepted query against the same graph snapshot.

  flowchart TD
    Q["Question and current graph schema"] --> P{"Qualified path?"}
    P -->|yes| F["Gateway: discover actual resources"]
    P -->|no| A["Local model: propose seed lookups"]
    A --> F
    F --> B["Local model: produce bounded PGQ"]
    B --> V["Gateway: validate syntax and parameters"]
    V --> G["DuckPGQ: execute on pinned snapshot"]
    G --> E["Fetch cited source excerpts"]
    E --> R["Freshness check and evidence response"]

Suppose we have discovered a file and want its source chunks. This is the supported query shape, formatted for readability:

MATCH (a:resource WHERE a.id = $seed)
      <-[e:relation WHERE e.kind = $edge_kind]-
      (b:resource)
COLUMNS (b.id, b.path, b.line_start, b.line_end, e.kind)

The worker selects the discovered file’s evidence ID as seed, with chunks_of as edge_kind. Our gateway checks that selection before execution. The incoming arrow follows the chunk-to-file relationship shown above. Requiring b.id gives the gateway an identity it can use to fetch the underlying source.

We also bound the evidence returned from a successful query. The English endpoint permits at most twenty rows and hydrates up to four distinct target or supporting-evidence resources within a shared excerpt budget. Its default source budget is 2,400 characters. The response includes the selected query and parameters, with snapshot identity and source citations available to the caller.

The primary model can inspect the query as well as the passages selected through it.

Grammar, Examples, and Meaning

Our GBNF work began with the actual query language we were prepared to execute. GBNF, the grammar notation supported by llama.cpp, lets us constrain the generated output through formal production rules. We built grammars for the response object and for the PGQ text inside its query field.

The setup follows the two model stages. For seed discovery, we generate a grammar using the current schema’s repository and layer values. Within the query grammar, aliases and labels have fixed forms. Projections use advertised properties, and placeholders must agree with the parameter object. One-hop traversal supports either direction. Bounded shortest-path queries use the supported outgoing form, with a maximum of four hops. The grammar permits the query forms supported by our gateway.

With GBNF enabled, we pass the generated grammar alongside the messages on the individual inference request. This lets us support multiple retrieval sources as they become available. The generic model runtime can receive another application’s instructions and output constraints on its next request. Our gateway returns prompt versions and request hashes. Callers can also request the complete inputs that produced a query, for further research and possible inline correction.

That grammar also keeps reasoning off in our Magnitude deployment with Gemma E4B. We use reasoning_effort: "none" together with the generated grammar in a nested response format:

{
  "reasoning_effort": "none",
  "response_format": {
    "type": "grammar",
    "grammar": "<the gateway-generated GBNF>"
  }
}

The grammar applies from the first generated token. Its root permits bounded whitespace and the permitted query JSON, so a separate reasoning preamble cannot be generated. The effort setting alone was insufficient in our tests. The combined configuration returned twelve correct queries with no observed reasoning, including concurrent requests.

Our system prompt supplies the task semantics that an output constraint alone leaves to the model. It establishes the two stages and identifies candidate records as reference data. We use two complete seed-planning examples and six complete query examples. They cover directionality and chunk membership, then extend to repository ownership and bounded reachability. Another example distinguishes identically named paths in different repositories.

One early prompt mixed the discovery stage with already-discovered resources, and a model copied a placeholder answer. We separated the stages and supplied complete worked responses. In another case, literal search placed a document mentioning the requested file ahead of the file itself. We added exact-path checks and bounded lookup recovery, then validated the requested paths explicitly.

A model can produce valid JSON for “what does this file link to?” when asked “what links to this file?” Our examples now place those two questions beside their respective outgoing and incoming queries.

Where the Time Went

Our original retrieval cycle time took about nine seconds. The current deployed path returned these complete HTTPS responses on October 20:

These are ordinary runtime requests with the model resident and startup priming complete. They include query generation, graph execution and evidence hydration. All eighteen responses in the acceptance run, including concurrent callers and repeats, matched snapshot-pinned graph results, pagination and source provenance. Identical repeats took 8–10 milliseconds. Four callers sharing a new question received complete responses in 1.04–1.27 seconds. This may seem like an arbitrary benchmark, but when two agents are given the same query, and one is asked to do a standard code review an the other is asked to do an adversarial audit of what’s returned, the need for parallel execution means that these grouped requests have real merit for continuing the work at pace.

The hardware is an RTX3050 with 8GB of VRAM, in a Linux VM with four vCPUs and about 8GB of system RAM. Gemma E4B QAT runs as a 4-bit GGUF through Magnitude with automatic MTP acceleration and the query grammar described above. The model and draft weights reside on shared NFS storage.

The reduction from nine seconds has several contributors: discovery planning, model selection, prompt reuse, generation constraints and the inference runtime. Magnitude was the final successful step. Broader questions that need a planning call have their own cost and still need relevance checks. These measurements establish the named-file query category.

The Returned Info Matters

Consider this question:

List outgoing links from docs/PSG_Nanopass_Architecture.md in FidelityFramework/Composer. Return the linked resources.

A separate timestamped request, measured on the earlier accepted engine, returned five graph relationships in 1.14 seconds, checked against the graph itself. Those relationships identify four linked files across two repositories and a repository resource. The complete JSON response occupied about 8 KiB, including citations, freshness and 3,000 characters of source excerpts within the requested 4,000-character budget.

For a concrete context comparison, we read that source document and all four linked files at the same published commits:

That is about 36 times less material to carry into the agent’s context for this question. The answer includes every returned relationship, but its excerpt budget omits one target’s source text. A closer investigation can request that passage explicitly. This is selective evidence with a route back to the source.

The keyword comparison has a useful wrinkle. None of those four target files contains the source filename. Searching the five selected files for “nanopass” finds 45 lines; searching for the source filename finds a grand total of 0.

Following the recorded links answers a relationship question that matching the question’s words does not establish.

Raw file reading is fast: the local Git reads took about ten milliseconds. The useful saving is in what the agent must carry forward and interpret. One bounded response supplies the recorded connections across repositories, with specific leads for closer reading. That matters when an implementation worker and an adversarial auditor ask the same question with different remits: each can spend its context on its own task while sharing the same source identities.

This is the neurosymbolic arrangement in practical terms. A language model interprets the question, symbolic execution follows a validated graph query, and a language model makes use of the cited result. The small retrieval model does its work with reasoning disabled. The graph carries the relationships, and the output constraints keep query generation focused. Its coverage still depends on our extraction rules and source policies; missing relationships remain possible and testable.

Focused Workers, Concentrated Evidence

Our larger model can use retrieval directly, or delegate a question to a smaller worker with a specific assignment. We have exercised native retrieval tools through three workers, each returning fresh, commit-linked evidence. That gives us a basis for more ambitious research fan-out as our confidence grows in the reliability of the pattern.

For a cross-repository investigation, we would divide the work by purpose. A research worker would locate the relevant declaration and follow its supporting relationships. An adversarial auditor would look for a counterexample or a mismatch between documentation and current code. A summarization worker would condense the selected passages, retaining citations and identifying unresolved questions. This division of work fits the specialist-model orchestration we have explored for decentralized AI.

  flowchart TD
    P["Primary model: question and assignments"] --> R["Focused research"]
    P --> A["Adversarial audit"]
    P --> S["Targeted summarization"]
    R --> T["Worker-selected retrieval tools"]
    A --> T
    S --> T
    T --> H["Public hybrid search"]
    T --> G["Committed-source graph"]
    H --> E["Bounded evidence with source links"]
    G --> E
    E --> W["Worker findings and open questions"]
    W --> C["Primary model: focused synthesis"]

Each worker’s context can stay close to its assignment. The primary receives compact findings and can request full source when a conclusion needs closer examination. An auditor’s disagreement is useful output: it identifies where the primary should spend more of its attention budget.

For a pinned snapshot $s$, an accepted query $q$, parameters $p$, and a fixed retrieval configuration $\theta$, the selected evidence is:

The configuration includes ordering and evidence limits, along with the extractor and query-engine versions. Recording those inputs makes a concrete retrieval inspectable and repeatable. Natural-language query selection and the final summary remain model-dependent. We aim for near-deterministic behavior in the bounded retrieval operation, taking particular care with choices such as tied shortest paths.

A Memory Updated by the Work

All of this would age quickly if the graph remained fixed. In this case, the “learning loop” is external to the model’s context, and updates to the corpus can improve the retrieval system’s results. Our retrieval service receives signed push webhooks for tracked branches. A relevant push schedules re-indexing, so committed changes feed into subsequent retrieval.

The webhook supplies a change hint. The indexer resolves the configured branch itself, rebuilds the affected repository components, and reuses the unaffected components. It then recomputes cross-repository relationships and validates a complete new graph generation. A final check of the branch heads precedes full publication.

  flowchart TD
    C["Code push to a tracked branch"] --> H["Signed webhook and durable delivery record"]
    H --> R["Resolve authoritative branch heads"]
    R --> B["Rebuild affected graph components"]
    B --> V["Validate complete generation and recheck heads"]
    V --> P["Publish immutable snapshot atomically"]
    P --> Q["Subsequent queries use current evidence"]
    Q -.->|research informs the next edit| C
    T["Periodic reconciliation"] --> R

We deduplicate deliveries and reconcile the tracked branches at regular intervals to recover missed changes. Requests wait while indexing and cross-linking complete, then bind to a complete fresh generation. Identical questions share computation while each caller retains its own cancellation. Freshness is checked around graph queries and model calls, with an explicit age bound on source observations. Corrections and deletions therefore have a defined route into the service’s memory alongside additions.

The continuously learning part is this ’external’ memory. Model weights can remain fixed while the available evidence changes after a commit. An improved explanation becomes retrievable, and a corrected relationship changes the paths available to the next investigation. Our source revision and extraction rules remain attached to that knowledge.

For anyone waiting nervously for the next announcement about recursive self-improvement, our contribution this week includes a webhook. Please contain your gasps.

Schmidhuber’s 1987 work already explored learning how to learn, including programs that modify program-modifying procedures. He investigated changes to the improvement process itself. In our operational loop, people develop and correct the system, and the retrieval service makes each accepted revision available to subsequent work. Schmidhuber’s thesis and overview

His 2015 On Learning to Think considers a controller that learns to query a recurrent neural world model for planning and decision making. Active consultation of a separate representation is the useful conceptual connection here. Fanion’s present mechanism keeps model weights fixed and updates an explicit source graph through commits. Its gains come from the representation, constrained access and tested evidence selection.

Native hosting also leaves room for the adaptive domain models we are developing: domain computations whose constraints, uncertainty and numerical evidence remain attached to their results. Fanion can provide a route to that evidence as the framework grows. The compiler and numerical system establish the narrowing route, and an accepted model update must preserve its binding through warm rotation. Today’s source graph gives us ideas toward a working foundation while keeping its current remit concrete.

Specialized models will still encounter information outside their own domains. A service that narrows that material, preserves its provenance and makes it available for closer examination remains useful alongside them. Fanion’s role can grow with the knowledge we organize through it. Today the mechanism is a bit stilted and archaic. Webhooks attached to repositories is a blunt instrument for an expansive problem space. But we are interested in taking the known steps first before we venture into the unknown.

With a bit of systems knowledge, we can connect free tools such as DuckDB and DuckPGQ to a local inference runtime and build useful adaptive systems. The engineering is available to anyone prepared to define the graph and test the retrieval behavior. No one needs to wait on a ‘frontier lab’ or other misnamed entity for this capability. Anyone can build it today.

Our current arrangement considers both on-premises services and tools deployed in Cloudflare. Public hybrid search runs alongside the site through the Fickl-to-JavaScript pathway, while committed-source graph retrieval and local inference run within our own infrastructure. Our deployment choices follow the workload. The result we care about is reliable, near-deterministic knowledge compression for intelligent systems design: a focused evidence packet whose origin and selection we can inspect. We can use it for further inference or more concrete redirection.

The information we’re building with is constantly growing. The graph changes and grows with it. The next question starts from there, and our ‘frontier’ expands with data sovereignty and efficiency as first-class considerations.