Clef Programming Language – Clef Programming Language

Counting the Cost of Coordination

Fri, 19 Jun 2026 00:00:00 -0400

Concurrency has been the through-line of our work from the start. We describe Clef as a concurrent programming language that happens to use ML-family semantics inside our Native Type Universe, and the ordering in that sentence is deliberate. The functional core, the immutability, the parametric types, the computation expressions, is the material we build with. What we are building is a language whose first concern is how programs run alongside one another safely and efficiently.

That concern has two faces, and we have written about both. The first is correctness: whether a concurrent program stays safe and makes progress. We took that up in Fearless Concurrency Gets Real, prompted by an interview where a Rust maintainer described that language’s borrow checker fighting the developer, and in that entry we argued that our compiler should carry that burden, not the person at the keyboard.

This post addresses the second face: performance.

Concurrency is a wider domain than correctness alone, and the cost of coordinating concurrent work is its own structural problem. We have addressed it the same way, by building the answer into how a program is organized, so that the cost is not left for a developer to tune away by hand.

A recent talk states that performance problem as well as anyone, so we will start there. Jon Gjengset gave it this spring, “The Cost of Concurrency Coordination,” and it walks the cost of multi-threaded coordination all the way down to the CPU cache line. The slide framing is “Are Mutexes Slow?”, and the answer he builds toward is that the question was never really about mutexes. It is worth watching in full, because the reasoning is careful and the conclusion lands somewhere more interesting than the title suggests.

The talk is not about Rust, and Gjengset says so at the top: the material maps to whatever language you work in. That is the frame we take as well. The thing being measured is hardware, not a language, and the structural response we take to it is a property of how a program is organized, not of any one syntax.

Where the cost actually lives

Gjengset’s benchmark is deliberately plain: a shared counter that many threads only ever read, never write. Reading shared data should be cheap, and with one thread it is, around 250 million reads per second. Add a second thread and throughput drops to a tenth of that. Add more and it flattens out near the floor. A reader-writer lock, the tool usually recommended for read-heavy work, does no better, and as readers pile up it gets worse than a plain mutex.

The cause is not the lock. It is the cache coherence protocol underneath it. A CPU keeps recently used memory in caches close to each core, tracked in 64-byte chunks called cache lines, and a protocol named MESI negotiates which core may read or write each line. A write to shared data requires coordination across cores. When one core wants to modify a line that others hold, it has to tell every other core to drop its copy first. That cross-core conversation costs roughly thirty nanoseconds per transfer, against about one nanosecond for an access already in L1.

A reader-writer lock loses here for a reason that surprises people. Taking the read lock increments a counter inside the lock, and incrementing is a write.

So every reader writes to the same shared field, and that field bounces from core to core, exclusive then shared then exclusive, all the way around the ring.

The “read” lock is sequentializing on a hidden write. Gjengset’s summary is blunt: “it was never about locks. Coordination is expensive because of cache coherence.” The lock is just the interface the cost happens to arrive through. His other warning is the one worth pinning up: “lock-free does not mean contention-free.” A lock-free structure can still drag if two threads keep writing to the same cache line, even when they touch different fields in it. That last case has a name, “false sharing”, and it is the part of the talk that our entry here turns on.

The fix, and who pays for it

Gjengset shows a structure called left-right that scales linearly with readers, and the way it gets there is instructive: it gives every reader its own cache line. No reader writes to memory any other reader touches, so no reader has to coordinate. The readers run fully in parallel because the cores running them never have to talk.

Getting there is expert work. Left-right keeps two full copies of the data, tolerates stale reads, allows only a single writer, and requires a deterministic operation log to keep the copies in sync. Gjengset is careful that it is not a drop-in anything. And the false-sharing bug he hit while building it was a ten-times slowdown traced to two counters landing on the same 64-byte line, fixed by hand with a 64-byte alignment. His closing advice is explicit: you have to “pick the algorithm that best suits the data transfer patterns your application has,” exploiting every oddity of your workload to shave the coordination cost.

That is true, and it is the crux. The cost is real, it is structural, and in the world the talk describes it is paid by an expert who knows about MESI, reaches for the right algorithm, and remembers to align the counter. The knowledge sits with the person, and the fix is applied by hand, per data structure, per workload, and re-checked with a profiler after the fact. And this weighs on maintaining the application as well. This is not a one-time cost this problem space faces, and those experienced in this area know this all too well.

Making isolation a property instead of a discipline

The question we have been asked in our early designs is whether that cost has to be carried the same way. The thing false sharing requires is two independently-used pieces of data sharing a cache line. The thing that prevents it is keeping unrelated mutable state in separate regions of memory.

In a model where unrelated state is already separated by construction, false sharing across that boundary cannot arise, because there is no shared line to contend over.

Our actor model is built to start from that separation. In its design each actor owns an arena, its own region of memory, and nothing outside the actor can address it. Messages transfer ownership between actors through well-structured channels; they do not share it. Two actors that do not share memory cannot share a cache line, so the cross-core coordination Gjengset measures has nothing to coordinate in our frame. When state belongs to an actor instead of living behind a shared reference, the isolation that left-right achieves by hand, one cache line per reader, is simply the default arrangement.

We’re excited about this work, and are going about it diligently and ensuring the theory and practic align. Placing arenas to fit cache tiers, sizing them to working sets, verifying isolation with hardware counters like the HITM events Gjengset’s perf c2c analysis surfaces, is design work in progress in our compiler, Composer, targeting both CPU and GPU (and perhaps other hardware targets that could take similar advantage of our scaffold). The cache-conscious memory work lays that out for the CPU, and a companion piece does the same for the GPU. The structural account, why ownership-by-actor-boundary makes cache isolation a consequence and not a chore, is developed in our Program Hypergraph pre-print.

The honest scope matters here, for the same reason it matters in the talk. Left-right is fast because its author understood the hardware and paid for the performance in design effort. Our arena model does not abandon that effort. It relocates it: the alignment, the isolation, the per-region placement become decisions Composer reasons about over a known memory layout, where today they are annotations a developer remembers to write and a profiler later confirms. Where the structure can carry the decision, the developer does not have to. This matters both for “point of contact” software design and the long-form costs that accumulate through the service life of the application.

Two ends of the same problem

This is the performance companion to an argument we made about correctness in Fearless Concurrency Gets Real. There the question was liveness, whether a concurrent program makes progress, and the move was to let our compiler reason about the whole program so the developer does not thread the analysis by hand. Here the question is coordination cost, and the move is the same shape: the structure of the program places mutable state so that unrelated work does not contend, and the aligning and padding and measuring stop being a role that an intelligent toolchain doesn’t foist onto the developer and team that has to maintain the code.

Gjengset’s talk is the careful, hardware-level statement of why this matters, and it reaches its conclusion from the systems side, with a profiler and a benchmark and a deep read of the cache. We are coming at the same cost from the compiler side, asking what a language can guarantee about memory placement before the program is built. The cost he counts is real and it is not going away. The question we keep working is how much of it the structure can carry, so that fewer programs have to be hand-tuned to escape a tax that the shape of the program could have avoided from the beginning.

Fearless Concurrency Gets Real

Thu, 18 Jun 2026 12:00:00 -0400

A recent Pragmatic Engineer episode with Alice Ryhl talks through how the Rust team at Google and the Tokio side of their work. She’s about as deep in production Rust as anyone. And the practical advice she keeps circling back to, when someone is stuck fighting the borrow checker, is to change the data structure. She says it more than once. When that doesn’t get you out, the escape hatch she names is Rc, the reference-counted pointer you clone when the analyzer can’t see far enough to keep a plain reference alive.

I’ve written about Rust before (see the prior post), and I want to be careful here, because none of this is Rust doing something silly or wrong. It’s Rust being honest about a real limit. Ryhl puts the limit plainly (26:06): “Rust kind of assumes that it can check the scope of that reference by just looking at a single function. But if you have your struct and you’re passing it over functions, it might not be possible to make that analysis and so you just get a compiler error.” The check is function-local. Once a value’s lifetime crosses a function boundary in a way the local view can’t follow, you either restructure your data so the analyzer can see it, or you reach for a counted pointer and pay per clone.

A language that sells you zero-cost memory safety hands you reference counting by hand at exactly the points where its static analysis is needed most, and asks you to reshape your program until the analyzer is happy. The burden sits on the designer. Which raises the question:

Does a concurrent language have to make you contort your code to fit the checker, or can the toolchain be structured to fit the shape of the problem space?

Rust coined the catch-phrase “fearless concurrency,” and it is partially earned for their memory story. The goal here with the Clef language is to take the phrase the rest of the way, into liveness, and show how the Fidelity Framework builds on its ML-family inheritance to actually make good on fearless concurrency, from a more principled starting point than Rust. Our goal is to provide stronger guarantees carried with lighter developer burden.

A challenging landscape

The usual hazard in concurrent code is two pieces of work touching the same data, one writing while the other reads. If a value can’t change after it’s bound, two parts of a program that don’t share a mutable cell have nothing to fight over, so running them at the same time is safe. The terms of art for those two facts are referential transparency and parametricity, and they are what let our compiler, Composer, treat the pure parts of a program as safe to run in parallel for free.

Our compilation pipeline leans on this directly. Computation expressions decompose into delimited continuations for sequential effects and interaction nets for the pure parallel part, and it’s the immutable, parametric core that lets the interaction-net side run symmetric and independent (the DCont/INet duality doc walks the decomposition).

The second piece is one surface for all of it. Async, actors, queries: in our language they’re computation expressions that desugar through the same Bind, which is continuation capture under a different name (see delimited continuations for how the desugaring lands). You don’t get a separate concurrency model bolted on beside the language. You get the language’s own sequencing construct, reused.

And the actors are not an invention we reached for to make concurrency tractable after the fact. They descend straight from F#’s MailboxProcessor, the message-passing actor model Don Syme added to F# as a nod to Erlang. Our Olivier actors keep that shape: each actor owns an arena, messages move between actors, and nothing is shared behind a lock. Immutability gives us memory regions that don’t interfere. Parametricity is what lets the compiler run the pure parts in parallel without a safety annotation. The actor model gives us a concurrency design where the dangerous coupling, shared mutable state under contention, was never on the table to begin with. Both halves of fearless concurrency, the memory safety and the liveness, are reachable from that substrate.

Why “the fight” happens

“The fight” Ryhl describes is worth understanding. Her words point at it: the borrow checker checks a reference’s scope “by just looking at a single function.” It reasons locally by default, function by function, and where a value’s lifetime crosses a boundary you bridge the gap by hand, with lifetime annotations on the signatures the value passes through. Rust gives you the tools to thread that lifetime across functions. Threading it is the work.

For a value that flows through three functions, you annotate three signatures so each local check has what it needs. Most of the time the annotating is mechanical and you stop noticing. The fight starts where the annotations stop reaching: a struct that holds a reference and gets passed around, a sharing pattern the borrow checker cannot express in the lifetime vocabulary available, a relationship decided at runtime rather than written in a type. At that point the static analysis has gone as far as it can, and you reach for something that takes over from there.

Every one of those reaches takes the same shape. Pick up Rc and the lifetime question moves from the compiler to a runtime counter. Pick up unsafe and it moves to your own audit. Pick up a lock-ordering convention in a wiki and a liveness question moves to human discipline. Each is what you do when the static analysis runs out. The burden does not disappear. It shifts to the developer and the team that carries it collectively for the maintenance life of the application.

The direction we are taking is to push where the analysis can see, rather than to ask the developer to annotate the gap. A value’s lifetime, its sharing, and its wait-for relationships are facts about how the whole program fits together, so we represent them where that fit is visible: as edges in our Program Hypergraph that span functions and actors rather than living inside one frame. An analysis that reads from that graph can follow a value across the boundaries a local check can’t follow, and derive the relationship a developer might otherwise have to spell out.

No `unsafe` in Clef

Rust’s safety guarantees come with a documented off-ramp. Ryhl’s own description of the unsafe keyword (27:09) is honest: “unsafe is the escape hatch essentially,” a block where the compiler stops vouching for you. Every serious Rust codebase reaches one. FFI calls into C, raw pointer arithmetic, a write to a memory-mapped hardware register, none of it fits inside Rust’s borrow checker, so the language gives you a labeled room where the rules relax and asks you to be careful. The memory-safety bugs that survive Rust’s static checking cluster in and around unsafe, because that is the only place they can.

We took a different position on the same boundary. Take a plain case: a counter that a closure increments, the kind of shared mutable state that shows up everywhere. In Rust, sharing one mutable value across closures runs into ownership, so the idiomatic answer is to wrap it:

use std::rc::Rc;
use std::cell::RefCell;

let count = Rc::new(RefCell::new(0));
let bump = {
 let count = Rc::clone(&count);
 move || *count.borrow_mut() += 1
};

The Rc<RefCell<_>> is the safety machinery worn on the surface: a reference count kept at runtime, and a borrow check that moved from compile time into a borrow_mut() that panics if the aliasing is ever wrong.

The same thing in Clef is the code with none of that on it:

let mutable count = 0
let bump () = count <- count + 1

You write the obvious thing. The design is that escape analysis sees count captured by bump and places it where its lifetime holds, with no wrapper type, no runtime count, and no unsafe in the source. This is work in progress in Clef: mutable bindings that stay in scope compile and run today, and the closure-capture case in this example is the escape integration we are building now, which managed mutability lays out in detail. The hardware and FFI cases are meant to follow the same rule: a peripheral register would carry its read or write direction in its type, and the boundary to foreign code goes through BAREWire as a structured contract both sides read by construction. The escape hatch Rust labels is, in our design, an ordinary construct the compiler checks rather than a room where it stops checking.

This is where Ryhl’s “change the data structure” advice ends up when restructuring isn’t enough: you reach for Rc. It manages memory at runtime, a count incremented on every clone and decremented on every drop, with the object freed when the count hits zero. That is reference counting, the same automatic memory management a garbage-collected language does, minus the part that reclaims cycles. Two references that point at each other never reach zero, so they leak. unsafe and Rc are two versions of the same move: where the static analysis runs out, the language hands you a runtime mechanism and trusts you to use it correctly.

No skirting deadlock

Rust’s borrow checker proves things about memory. No two live references alias a value mutably, no reference outlives what it points at, no data race compiles. None of that constrains whether the program makes progress. A Rust program can pass every memory-safety check and still freeze solid at runtime: two threads, each holding one lock and waiting on the other, neither able to move, the process reporting green while doing nothing. Rust teams handle deadlock the way the rest of the industry does, with a lock-ordering convention written down in a wiki, code review, and the occasional production incident that teaches everyone to respect the convention.

This isn’t a Rust-specific problem. Deadlock freedom in the general case is undecidable from a proof theoretic point of view, and Rust’s borrow checker is scoped to a fragment it can actually decide. Rust drew its boundary at memory and declined the liveness problem on purpose. We took a different slice of the same space: the part where the structure is visible enough to prove, with design-time support for situations that require direct developer intervention.

A PostAndReply suspends the caller’s continuation until the callee answers, and that suspension is a wait-for edge from caller to callee. A fire-and-forget Tell posts to a mailbox and returns, adding no edge, so the asynchronous fraction of a program cannot deadlock through this at all. The plan is to collect the synchronous edges into a wait-for relation on our Program Hypergraph, on the joint-constraint axis that carries region and lifetime hyperedges. For the fragment where every callee is a statically resolvable actor reference, deadlock freedom is acyclicity of that relation, and acyclicity is a rank function: an integer per actor behavior such that every edge u -> v has r(u) < r(v), which exists exactly when the relation has no cycle. That constraint is QF_LIA, the same kind of Tier-2 obligation our interval and bound checks already use, which is what makes it a tractable thing to build rather than a research problem.

Where the callee is chosen by live data (content-based routing, an actor handle passed in a message), the relation is not statically resolvable, and acyclicity over it is undecidable. That call downgrades visibly to supervised execution with a timeout, and our compiler design identifies the call that crossed the boundary. The visible fragment carries a proof, the dynamic one carries a guard and a label. Neither path goes quiet and there are no runtime surprises.

A handler doing a synchronous request looks like ordinary code:

let inventory = spawn InventoryActor

let handleOrder (order: Order) = actor {
 // suspends until inventory replies: one wait-for edge, order -> inventory
 let! stock = inventory.PostAndReply (CheckStock order.Sku)
 return reserve stock order
}

A cycle is two of those pointing back at each other. Order asks Inventory whether it can reserve; Inventory, mid-reply, asks Order to confirm a pending hold. Each parks on a reply only the other can send:

let handleFoo order = actor {
 let! ok = inventory.PostAndReply (Query order) // order -> inventory
 return commit ok
}
// inside InventoryActor, handling Query:
let handleBar q = actor {
 let! held = order.PostAndReply (ConfirmHold q) // inventory -> order
 return held
}

Our compiler would report the path the way escape analysis reports an escape, the actual chain rather than a generic warning:

CCS8031: synchronous wait cycle
Order.handleFoo waits on Inventory.query,
which waits on Order.handleBar.
break the cycle: supply a priority, convert one leg to Tell, or opt into supervised timeout.

The lowering underneath makes the diagnostic and the proof the same object. This is illustrative dialect, the real attribute names are still settling as Composer is built. Each PostAndReply lowers to a blocking op carrying only its own wait edge as local fact, because no single op can see the whole set:

%r = dcont.suspend_on_reply %callee : !actor.ref<"inventory">
 { rpc.wait_edge = #wait<from = "order", to = "inventory"> }

The acyclicity obligation rides on the enclosing scope, the smallest region closed under “can send a synchronous reply to,” which instructs the seam to gather every edge in the region and prove a rank exists:

module @order_system attributes { verif.obligation = #tier2.acyclic_wait } {
 // actor behaviors and their suspend_on_reply ops
}

Lowering would emit the verification condition into the SMT dialect, and the solver would discharge it like any interval check:

%edges = collect rpc.wait_edge in @order_system
smt.assert (forall (u v) (=> (wait %u %v) (lt (rank %u) (rank %v))))
smt.check // sat: a rank exists, acyclic. unsat: the core is the cycle.

When no such ranking exists, the relation has a cycle, and the developer would see a CCS8031 error that spells it out: the chain of calls that wait on each other, A.handleFoo waits on B.query waits on A.handleBar. Because this analysis rides the program graph, it would surface while you write, in the editor, the same place type errors already show up, not only at the end in a batch compile. And the error is the proof itself failing, not a separate heuristic warning bolted on beside the real check, so there is nothing to second-guess. The full treatment, including the orderable-cyclic case where Composer infers the priority a developer would otherwise hand-write, is in deadlock freedom as an obligation, and the design-time surfacing is what we are building out in our LSP and Lattice toolchain. Liveness deserves the same visible, steerable machinery we gave memory, and that is the direction our actor runtime keeps growing toward as the work continues.

An analyzer and the structure it rides

Come back to the function-local limit Ryhl named at the top. A struct passed across functions can defeat Rust’s borrow checker, whose single-function view loses the reference, and her remedy is to change the data structure: reach for an Rc, or redesign so the lifetime fits inside one function’s view again. The analyzer has a shape it can reason about, and the developer’s job is to deform the program until it matches.

Our escape analysis works the other way. The lifetime question lives in our program graph, which spans functions and actors, so a value passed across a call boundary is the same value the analyzer was already tracking. You write the data structure the problem wants. The analyzer does the cross-boundary reasoning. let mutable x = 0 is ordinary syntax, and Composer classifies the escape (StackScoped, ClosureCapture, ReturnEscape, ByRefEscape) and places x on the stack or in an arena from that classification. There are no 'a-tick lifetime annotations to thread, because the lifetime is a fact the compiler derives rather than a constraint you declare. We spent some time defining managed mutability for how the classification lowers.

Consider a value that escapes from one actor up to an predecessor. The default keeps the value in its owning actor’s arena and guards the escaped reference with a sentinel: one O(1) validity check at the boundary, with deterministic release when a counted obligation set empties, and no leak when references form a cycle. You add no annotation for any of that. From there our analyzer suggests hoisting the value into the ancestor’s arena, which sheds the guard entirely. Hoisting is accept-to-optimize: the safe-but-guarded form is the pit-of-success default, the faster form is a visible suggestion you can take or override. The mechanics are in the arena hoisting tooling doc and further articulated in our coeffect algebra documentation.

Both languages keep a runtime backstop for the cases static analysis can’t settle. Rust’s is the Rc from earlier, reached for by hand and unbounded in how much of the graph it covers. Ours is one sentinel, an O(1) check at a single boundary, placed by Composer and released deterministically with no cycle leak. Clef is not free of runtime checks either. The difference is that ours are bounded, automatic, and would be reached for less often.

The work ahead

The pieces we put together hold because they were chosen together. Computation expressions give one surface where async, actors, and queries all desugar through the same continuation capture, and the immutable, parametric core makes the parallel parts safe to run without anyone marking them. Don Syme brought MailboxProcessor into F# himself as a hat-tip to Erlang, a library at the periphery of the language rather than at its core. We are taking that add-on and making it central to the semantics, the move laid out in our unified actor architecture, so concurrency is Clef’s first concern, and the safety properties follow from that structure in a way that supports a cohesive and concise design-time experience.

Our language design doese not carry an unsafe keyword, because the boundary it would mark is a checked construct in our system. Deadlock freedom is designed to be proven where the wait-for graph is visible and would be supervised where live data picks the callee. For readers who want the formal treatment, the obligation framing and the rank-function discharge are worked out in the fixed-point scaffolding pre-print, where the same structural integrity reaches down into the reals through our negative and fractional types. The fragment we will be able to prove grows as the program graph carries more of the lifetime and wait-for structure into the open, and that is the direction we keep building toward as the work continues.

Small choices loom large

Rust started inside this same family. Its first compiler was written in OCaml, and early Rust carried more of the ML character along with it, a garbage collector, green threads, a typestate system, the runtime-assisted shape of an ML-family language. Mozilla’s push to make it a viable C and C++ replacement is what traded those away for ownership and the borrow checker, and that turn is what made Rust the language it is, defined in reflection of C rather than descent from ML. It is a good language because of that choice. We chose a distinct path: keep the ML substrate and build on it something no language in this family has by default, concurrency that is safe and live by construction.

Ryhl’s advice is good advice for the language she works in: when the borrow checker fights a developer, the fix is to change the data structure. The compiler doesn’t follow the design across the function boundary, so the design is coerced toward the tool. That coercion is a cost the compiler error never shows. It is the hours of reshaping a correct program until a local analysis will sign off on it, and a long arc of support that follows. The fight is a tax on the people who write and maintain the code, paid in the currency of the day, attention and time, not only in the safety of the result.

The design we are building takes a unique direction, the compiler tracking to the required safety features of the code. A program describes the structure the solution calls for, the analysis follows each element as it flows, and Composer carries the burden of proving the result safe and live. Our aim is to deliver a language where correctness and ergonomics are not traded against each other. That is what fearless concurrency means to us, and it is the direction Clef is being built to reach.

Where Native Goes, Mobile Follows

Thu, 14 May 2026 00:00:00 +0000

Most developers approaching mobile deployment take the cross-platform framework ‘shortcut’. React Native, Flutter, Kotlin Multiplatform, .NET MAUI, Capacitor, and Cordova abstract the platform-native binding surfaces behind a uniform runtime, and the developer’s main language stays at the door in exchange for targeting handheld and wearable devices. Native compilation modes (Flutter’s AOT, React Native’s Hermes, Kotlin/Native, .NET MAUI’s NativeAOT) arrived later in each of those solutions as ersatz optimizations on top of the existing runtime compromises, and the contract is what shapes the developer’s day-to-day experience.

That contract made sense when the binding surfaces underneath felt inaccessible to anyone outside the platform vendor. They are no longer hidden. Android exposes the NDK with bionic libc, ELF shared objects, and direct JNI access to the kernel beneath the Java framework. iOS apps are Mach-O binaries linked against libSystem and Apple’s frameworks; Swift and Objective-C are the conventional languages, and the binary format and ABI are open. Samsung’s Tizen applications may be .tpk-packaged native ELF with full Wayland compositor access. Linux mobile distributions like postmarketOS expose a standard Linux toolchain and a Wayland session on the handset itself. The native binding surface is, on every modern mobile target, the substrate the platform vendor’s own applications run on, and the surface remains available to a compiler that has the intelligence and tooling to target it directly.

flowchart LR
Clef["Clef source"]
Composer["Composer<br>(MLIR pipeline)"]
Clef --> Composer
Composer --> Desktop["Linux desktop<br>ELF / glibc"]
Composer --> Phosh["postmarketOS, Phosh<br>ELF / musl (APK)"]
Composer --> Tizen["Samsung TV (Tizen)<br>ELF / glibc (.tpk, .wgt)"]
Composer --> Android["Android<br>ELF.so / bionic (APK + JNI)"]
Composer --> iOS["iOS<br>Mach-O / libSystem (.ipa)"]
Composer --> MCU["STM32, RA6M5<br>bare ELF / picolibc"]

Our approach to mobile and embedded programming was built from the other direction. We compile directly to native artifacts across heterogeneous platforms. The compiler architecture and binding tooling were chosen because they support producing native artifacts for several targets without a runtime substrate in between. Mobile, in this view, is one of the targets the native compilation path reaches without a boutique runtime or other intermediary. What follows walks through our early design decisions and the cross-platform native capability they produce. LLVM platform tuples serve as the low-level back end, with two adjacent UI patterns our framework supports today. We describe per-platform packaging, the language mechanism that lets one design satisfy multiple platform targets, and the costs and benefits the approach carries.

The wrapper around each artifact is what differs across the platform set; the binary inside is what Composer produces from substantially shared Clef source code and per-target handling of the compilation back end.

The Conventional Path and Its Costs

Common cross-platform mobile frameworks each ship a runtime as the foundation of a deployed application. React Native applications carry approximately 25 MB of runtime before any user code, comprising the JavaScript engine (Hermes or JavaScriptCore), the bridge, and the React Native libraries. Flutter ships its Skia rendering engine and the Dart VM, typically 6 to 8 MB compressed in a release build. Kotlin Multiplatform applications retain kotlin.native.runtime even on the K/N path. The numbers compound across a device’s installed application set; a user with thirty React Native applications installed carries roughly 750 MB of duplicated runtime resident in storage, since each application bundles its own copy. And that’s before you get into version pinning and security concerns of those standing dependencies.

Working-set memory follows the same shape. A typical React Native application maintains a 150 to 200 MB working set on iOS; Flutter is somewhat better at 80 to 120 MB. The user experience consequence is direct: cold-started applications feel sluggish even when the application’s own compute is small, because the launch cost the user feels is dominated by the runtime’s bring-up.

The security surface is the third axis. A JIT-compiled or interpreted runtime presents a substantially larger attack surface than a native binary that contains no runtime. JIT-spray attacks, garbage collector observability side-channels, the general property that any code path can be re-executed under attacker-controlled conditions, and the difficulty of formally auditing a runtime that may itself be patched after deployment, are concerns intrinsic to the runtime model. For applications with cryptographic obligations or constant-time requirements, this is the gating consideration. Any constant-time claim a developer might want is contingent on the runtime’s behavior, and the runtime is free to reschedule, recompile, or interpret at moments outside the developer’s control.

These costs are the price of the runtime model. The runtime model accepts them in exchange for a uniform deployment story, the promise of a “single language” across platforms, and an iteration loop that includes hot reload. The native path has to recover the conveniences the runtime model bundles together, and it opens properties that exist only on a binary our verifier reasoned about end-to-end. The remainder of this post covers both halves of that exchange.

The Day-One Design Decision

MLIR as the intermediate representation

The choice of MLIR as our intermediate representation was made because native compilation across heterogeneous targets requires an IR that can express both high-level language semantics and low-level platform specifics across the lowering pathway. MLIR’s dialect system supports progressive lowering to multiple distinct backends without forcing a uniform runtime contract on the upper layers. The Composer pipeline lowers Clef source through nanopass transformations that successively rewrite the program from language-level dialects to platform-specific elaboration, and the final emission is whatever LLVM (or a non-LLVM backend such as JSIR) can target. The choice of MLIR over Java bytecode, WASM, or a .NET CLR-style IL was made specifically to keep the platform space open. (MLIR’s Hello World, lowered to native walks the lowering in detail; the rationale for MLIR over the alternatives is in Why Clef Fits MLIR.)

Nanopass lowering

Our nanopass discipline (many small passes, each preserving a stated invariant) makes per-platform divergence tractable. Different targets share the upper passes and diverge in the lower passes; the divergence is localized to the passes that need it. The tree-shaking pipeline is the most visible payoff. HelloWayland, our 19-line Clef program that draws an SVG-rendered native window on Wayland, generates approximately 19,500 PSG nodes from its declared transitive dependencies and reduces to approximately 1,450 reachable nodes after tree-shaking. The ratio is roughly 13:1, and the reduction is the direct cause of the resulting binary’s compact size and low startup cost. (Intelligent tree-shaking, Nanopass navigation, and Hyping Hypergraphs cover the mechanism and the Program Hypergraph counting that produces these numbers.)

Module signatures with platform-specific implementations

The language-level feature that operationalizes the cross-platform claim is signature conformance. A worked example with three platform-specific implementations appears in the module signatures section later in this post.

Memory model designed for arena and region allocation

Our memory model is free of garbage collection by design. Memory regions are explicit and lifetimes are inferred at compile time through Clef’s coeffect tracking (The native memory management design elaborates). On mobile, working-set memory tracks the application’s actual data footprint, and the cold-start budget belongs to the application’s own initialization and its HAL.

Coeffect tracking for resource invariants

Constant-time execution, allocation discipline, IO discipline, and syscall discipline are coeffect properties expressible in Clef’s compiler service and checked at design time through our coeffect-aware process (Context-Aware Compilation documents the mechanism). These properties are stated as preconditions, checked at compile time, and designed to be preserved through every nanopass to the emitted binary. Native compilation is the precondition that lets coeffect claims hold all the way to the loader, and on every Fidelity target, mobile included, the design carries coeffect-tracked properties to binary. This is the link between the mobile compilation account in this post and the broader verification story in Cryptographic Certainty.

All five were in place before mobile compilation was a question, and each of them is non-trivial to retrofit into an existing runtime-first framework.

The Platform Tuple as Compilation Substrate

A platform tuple in our framework is the triple (architecture, libc, packaging). Composer reads the target tuple, selects matching module-signature implementations, routes through the appropriate LLVM backend, and emits an artifact in the platform’s native format. The matrix spans six platform classes:

Platform	Architecture(s)	Libc	Output format	Packaging wrapper	UI substrate(s)
Linux desktop	x86_64, aarch64	glibc	ELF	none (or .deb, .rpm)	Wayland direct, GTK4, system WebView
postmarketOS / Phosh	aarch64, armv7	musl	ELF	apk	Wayland direct, GTK4, WebKitGTK
Samsung TV (Tizen)	aarch64 (some armv7, x86_64)	glibc	ELF	.tpk or .wgt	Wayland, EFL, system WebKit
Android	aarch64, armv7, x86_64	bionic	ELF (.so)	APK with JNI shim	NativeActivity, Compose host, System WebView
iOS	aarch64	libSystem	Mach-O	.ipa with Swift shim	Metal direct, UIKit / SwiftUI host, WKWebView
STM32, RA6M5	armv7m, armv8m	musl static, picolibc	bare ELF	bootloader image	LVGL, framebuffer direct, none

Our framework treats libc as a compilation parameter. The Fidelity.Libc bindings are abstracted through module signatures, and the implementation selected for any given build is the one whose ABI matches the target tuple. The bindings themselves are generated by Farscape from each library’s headers via clang AST extraction, which means new targets reduce to acquiring the headers and adding a target tuple.

Three binary formats appear. Linux, Android, and most embedded targets emit ELF. Deep-embedded targets emit bare ELF without dynamic linking, linked statically against the chosen runtime. iOS and macOS emit Mach-O. Composer routes ELF and bare-ELF through LLVM’s existing backend coverage. The iOS Mach-O path is designed to use a Swift-native-style compilation route. The resulting binary is built to link against libSystem and Apple’s framework set, conforming to the ABI Apple’s loader expects. Every artifact lands on its platform in the platform’s standard format.

The packaging wrapper is platform-specific glue around an unmodified binary. Each wrapper in the matrix is a metadata envelope plus a small amount of platform-native shim code. An APK with a JNI shim is roughly 100 lines of Kotlin plus a manifest. An iOS .ipa with a Swift shim is roughly 50 to 150 lines of Swift plus the project file. A Tizen .tpk is the manifest, the icon set, the signatures, and the ELF binary. A .desktop file for Linux is a key-value text file. The wrapper surrounds the Fidelity-compiled artifact and supplies the platform-specific metadata the loader looks for.

The signing model varies across platforms, and the binary inside each wrapper has the same shape as the binary the platform’s own toolchain would produce. Our contribution is to compile to that shape natively, with the same dialect-aware pipeline that compiles to STM32 or to Linux desktop.

The UI Strategy: Two Reference Implementations

Our framework supports two UI architectures today, each grounded in a published reference implementation. The two are complementary; neither subsumes the other, and the choice between them is an operational one.

The HelloWayland pattern: native widgets, direct surface

HelloWayland is our published reference for the native-widget pattern. The Clef program is 19 lines. It produces an ELF binary that links against wayland-client, libdrm, libgbm, and resvg. The UI primitives (label, svgImage, withColor, App.run) are Fidelity.UI types compiled directly into the binary, sharing no code with GTK, Qt, or any platform UI toolkit.

The architectural commitment of the HelloWayland pattern is to own the widget toolkit and to draw to the compositor’s surface directly. The widget lifecycle and the event loop are Fidelity-owned, and lowering produces direct compositor calls. The advantage is binary footprint, startup latency, and the ability to satisfy our own coeffect constraints (constant-time, allocation-free, syscall-free) end-to-end through the UI code, because the event loop is part of the verified compilation unit instead of a foreign toolkit’s runtime.

The pattern maps across the platform matrix. On Linux desktop, the pattern works as published with Wayland. On postmarketOS and Phosh, the same pattern applies with the mobile-specific Wayland protocols added (wlr-layer-shell for status bars and notification rolls, input-method-v2 and text-input-v3 for on-screen keyboards, wl_touch for touch input); the compositor on a Phosh device is a Wayland compositor, and the surface contract is the same. On Samsung TV with Tizen, Wayland surfaces are accessible to .tpk native applications, and the pattern works with Tizen’s compositor and .tpk packaging. On Android, NativeActivity (or GameActivity for the higher-level API) provides a single full-screen ANativeWindow surface, and Fidelity.UI primitives lower to Vulkan or OpenGL ES instead of to Wayland; the host shim is a small Kotlin or Java Activity. On iOS, an MTKView is hosted by a thin UIViewController, and Fidelity.UI primitives lower to Metal; the host shim is a small Swift class. On STM32 and RA6M5, LVGL is the target.

That last case is where this post intersects most directly with Leveraging Fabulous for Native UI. The compile-time UI lowering strategy described there (runtime widget diffing becomes compile-time widget resolution, runtime event-handler closures become static function pointers with explicit context) is the same compile-time lowering strategy that produces Metal calls on an iPhone and Vulkan calls on an Android phone, parameterized by the target tuple. The microcontroller and the smartphone are the same kind of compilation problem to Composer.

The WRENHello pattern: system WebView with native backend

WRENHello is our published reference for the WREN-stack pattern. The architecture has four parts: a SolidJS frontend compiled via Fable to JavaScript, a native backend compiled via Composer to a platform ELF or Mach-O, BAREWire over localhost WebSocket between the two, and the system WebView as the frontend’s host. The HTML and frontend assets are embedded in the binary’s .rodata section. The resulting binary is 2 to 10 MB depending on frontend assets, compared with Electron’s typical 150 to 300 MB.

The pattern is documented in detail in the WREN Stack post. The relevant property for the present discussion is that the boundary between frontend and backend is uniform across all platforms. The Shared/Protocol.fs types compile to both Fable (JavaScript for the WebView) and Composer (native machine code for the backend), and the same BAREWire schema describes the wire format in each direction. The platform-specific code is the WebView host integration, which is what each platform’s UI substrate is built around anyway.

Each platform’s WebView host substitutes for the GTK4 host on Linux: WebKitGTK on Linux desktop and postmarketOS, Tizen’s WebKit (with the backend as a separate .tpk Service Application and Tizen message ports as the BAREWire transport), Android System WebView (with the backend as a foreground Service and AIDL or local socket carrying BAREWire), and WKWebView on iOS (with the backend in-process and WKScriptMessageHandler carrying BAREWire, since iOS does not permit user-installable background services). The per-platform mechanics appear in the Per-Platform Walkthrough below.

When to use which pattern

The choice between the HelloWayland (native) and WRENHello (WebView) patterns is at the UI substrate, not at the language or compiler level. Both patterns share the same backend compilation pipeline and the same BAREWire interchange. The guidance:

Use the HelloWayland pattern when the UI is the computational artifact (signal visualization, custom renderer, high resolution instrument panel); when binary footprint is a hard constraint (embedded, kiosk, panel applet); when the application has security-relevant constant-time requirements that conflict with running inside a JavaScript engine; or when the deployment target lacks a system WebView (deep embedded, bare-metal).

Use the WRENHello pattern when the UI is form-driven, list-driven, or settings-driven (the majority of business applications); when responsive design across phone, tablet, and desktop is a requirement and CSS handles it without further effort; when the team’s UI expertise is web-shaped and SolidJS or a similar reactive framework is the productive tool; or when the deployment target has a system WebView readily available, which these days means every modern mobile target worth targeting.

Per-Platform Walkthrough

The walkthrough below assumes a technical appetite; its purpose is to differentiate what the platform-specific work actually consists of for each row in the matrix.

Linux desktop

The Fidelity-compiled ELF is the deliverable, with no further wrapping required. An optional .desktop file would provide menu integration for desktop environments; an optional .deb or .rpm package wraps the binary with distribution-standard metadata for the package manager. This is the baseline against which the other platforms add their packaging overhead.

postmarketOS and Phosh

Alpine APK packaging. The musl target tuple would be used because Alpine’s libc is musl. WebKitGTK is pre-built against musl in Alpine’s community repository, and GTK4 is similarly available. Its Phosh app metadata follows the standard Linux desktop conventions with a small set of mobile-specific hints (X-Purism-FormFactor for the legacy Phosh stack, mobile-friendly markers for newer revisions). Distribution would be through F-Droid for community-maintained applications or through Alpine’s repository system for distribution-curated applications. The postmarketOS device support matrix is wider than commercial Linux phone offerings; our Phosh target aspires to reach every device on which postmarketOS runs Phosh, which today covers several years of mainstream Android hardware.

Samsung TV (Tizen)

Two packaging paths exist: .tpk for native applications and .wgt for web applications. A native .tpk is a manifest, the Fidelity-compiled ARM ELF binary, the icon set, the resource directory, and the signature, signed with author and distributor certificates. The .wgt is the WREN-stack analogue: a packaged frontend with a Tizen manifest. Combining a .wgt frontend with a .tpk Service Application, communicating through Tizen message ports, gives the full WREN-on-Tizen deployment. Distribution would be through Samsung’s Seller Office.

Android

This is an APK packaging story. The wrapper is a thin Kotlin or Java Application and at least one Activity. For HelloWayland-pattern applications, the Activity is typically a NativeActivity (or GameActivity) that gives the native code direct access to a single full-screen ANativeWindow. For WRENHello-pattern applications, the Activity would host an Android System WebView starting a foreground Service that would own the native backend. The Fidelity-compiled binary would be placed at lib/<abi>/libfidelity.so inside the APK, compiled for arm64-v8a, armeabi-v7a, and x86_64 as needed for the target device set. The bionic libc target tuple applies, and Farscape would generate the JNI bridge stubs from the platform’s jni.h headers. Signing uses any Android keystore the developer controls.

iOS

Our wrapper would use a Swift AppDelegate and at least one UIViewController. For HelloWayland-pattern applications, the controller hosts an MTKView and forwards lifecycle events to Fidelity-compiled callbacks. For WRENHello-pattern applications, the controller hosts a WKWebView and registers WKScriptMessageHandler instances that exchange BAREWire payloads with the in-process native backend. The Fidelity-compiled binary would be a Swift-native-compatible Mach-O for arm64. The compilation path is designed to match what Apple’s own toolchain produces. The artifact links against libSystem and the framework set declared in the project (UIKit, Metal, QuartzCore, Foundation, and so on).

Embedded (STM32, RA6M5)

The artifact is a flashable image. It is produced by linking statically against the runtime the target tolerates and emitting raw ELF for the bootloader. The runtime selection is per-target. The libc surface is musl-static or picolibc, depending on flash budget. Vendor HAL bindings are brought in through Farscape from each vendor’s headers: CMSIS for Cortex-M cores, Renesas FSP for the RA family, ST’s HAL for STM32. The choice of libc and HAL is dictated by the flash budget, the peripheral set, and the runtime requirements (Fidelity on STM32 walks the constraints).

The Module Signatures That Make It Work

Cross-platform native compilation in our framework follows directly from a language-level mechanism: writing a single Clef module signature for a platform-dependent capability, binding that signature to multiple implementations, and letting Composer select the implementation matching the target tuple at compile time. The mechanism is the signature conformance feature in the Clef specification.

A concrete example illustrates. Our Fidelity.UI native surface abstraction declares its contract once:

// Fidelity.UI.Native.fsi (signature)
namespace Fidelity.UI.Native

module Surface =
 type SurfaceHandle
 type SurfaceConfig

 val create : config: SurfaceConfig -> SurfaceHandle
 val present : handle: SurfaceHandle -> unit
 val destroy : handle: SurfaceHandle -> unit

The platform-specific implementations satisfy that contract. The Wayland implementation is used on Linux desktop, postmarketOS, and Tizen:

// Fidelity.UI.Native.Wayland.fs (Linux, Phosh, Tizen)
namespace Fidelity.UI.Native

module Surface =
 open Bindings.WaylandClient
 open Bindings.Drm
 open Bindings.Gbm

 type SurfaceHandle =
 { Display: wl_display
 Surface: wl_surface
 Buffer: wl_buffer }
 type SurfaceConfig =
 { Width: int; Height: int; Format: uint32 }

 let create config = ... // wl_compositor_create_surface, gbm_bo_create
 let present handle = ... // wl_surface_attach, wl_surface_commit
 let destroy handle = ... // wl_surface_destroy, gbm_bo_destroy

The Metal implementation is used on iOS:

// Fidelity.UI.Native.Metal.fs (iOS)
namespace Fidelity.UI.Native

module Surface =
 open Bindings.Metal
 open Bindings.QuartzCore

 type SurfaceHandle =
 { Layer: CAMetalLayer
 Drawable: CAMetalDrawable
 Device: MTLDevice }
 type SurfaceConfig =
 { Width: int; Height: int; PixelFormat: MTLPixelFormat }

 let create config = ... // CAMetalLayer alloc, drawable acquisition
 let present handle = ... // currentDrawable.present
 let destroy handle = ... // layer release, drawable release

The Vulkan implementation is used on Android:

// Fidelity.UI.Native.Vulkan.fs (Android)
namespace Fidelity.UI.Native

module Surface =
 open Bindings.Vulkan
 open Bindings.AndroidNative

 type SurfaceHandle =
 { Instance: VkInstance
 Surface: VkSurfaceKHR
 Swapchain: VkSwapchainKHR }
 type SurfaceConfig =
 { Width: int; Height: int; Format: VkFormat }

 let create config = ... // vkCreateAndroidSurfaceKHR
 let present handle = ... // vkQueuePresentKHR
 let destroy handle = ... // vkDestroySurfaceKHR

The consuming code is the same across platforms:

open Fidelity.UI.Native

let runUi config =
 let surface = Surface.create config
 // application rendering logic using Surface.present
 Surface.destroy surface

Composer selects the implementation by target tuple. When the tuple specifies aarch64-apple-ios, the Metal implementation is selected. When it specifies aarch64-linux-android, the Vulkan implementation is selected. When it specifies aarch64-unknown-linux-musl for postmarketOS or aarch64-tizen-linux-gnu for Samsung TV, the Wayland implementation is selected. The conformance check runs at compile time and verifies that each implementation satisfies the signature’s type and arity contract.

The arity property quoted from the specification is load-bearing: “Arities in a signature must be equal to or shorter than the corresponding arities in an implementation, and the prefix must match… function arity affects compilation. Clef functions with known arity compile to direct function calls, while function values require closure allocation. Signatures must contain enough information to reveal the desired arity for efficient native code generation.” The consequence is that the cross-platform substitution does not introduce virtual dispatch, closure allocation, or runtime indirection; every call resolves to a direct platform-specific function call in the emitted binary.

Signature-and-structure separation is a well-established ML-family mechanism, in place since Standard ML in the 1980s and exercised at scale by OCaml’s functor system for decades. Our framework uses that mechanism as the load-bearing selector for platform-specific lowering, with the target tuple as the selection key and the requirement that the conformance check produce direct native calls in the lowered IR. We have found no other representative implementations that drive cross-platform native lowering this way in the standing literature we have reviewed.

Honest Tradeoffs

The architectural account above presents the cross-platform native path in its most direct form. The path has measurable costs, and an honest treatment must name them. The structure of this section follows the template established by the “What the Type System Catches and What It Does Not” section in Cryptographic Certainty.

Per-platform shim code is real and not zero

A WREN-style application needs platform-specific WebView host integration: Kotlin or Java on Android, Swift on iOS, GTK4 with WebKitGTK on Linux, a Tizen manifest with message-port handling on Tizen. A HelloWayland-style application needs the equivalent surface host: a NativeActivity on Android, a UIViewController hosting MTKView on iOS, a Wayland display connection on Linux. The shim is small, typically 50 to 200 lines per platform for a representative application, and it remains real work the developer has to write and maintain.

iOS specifically requires macOS for the build

Apple owns this constraint. The SDK requires Xcode, and Xcode requires macOS. A Fidelity-on-iOS build pipeline needs Mac hardware or a Mac cloud build service (MacStadium, GitHub Actions macOS runners, or an equivalent). The dependency is limited to the iOS target; every other target in the matrix builds on Linux or Windows. This is the only hard infrastructure constraint our framework inherits from a platform vendor, and it is durable because Apple owns the toolchain back end.

No hot-reload story comparable to Flutter’s

The iteration loop is compile, deploy, run. For UI iteration specifically, the WREN-stack pattern partially recovers the fast loop by hot-reloading the frontend with Vite or a similar tool while the backend runs on a “cold” reboot cycle. This is most of what Flutter’s hot reload provides in practice for UI development. For pure-native HelloWayland-pattern work, the iteration loop is the compile-and-run loop, faster than C++ on the same hardware because Composer’s incremental compilation is fast, and slower than a JavaScript runtime’s hot reload. Our framework offers incremental native rebuilds, and we anticipate that the cycle for backend work will measure in seconds rather than the sub-second feedback an HMR engine provides. This is something we will look to improve as the work continues.

App Store review may be more cautious on first submission

App Store review for certain targets may be slower or more opinionated for a novel binary architecture. Fidelity-compiled Mach-O is structurally identical to Swift Mach-O at the loader level, but the metadata and the symbol table may be visibly different. We anticipate some additional review time on the first submissions that uses our framework. Subsequent submissions within the same application’s lifecycle should pass at normal review pace once the reviewers have the application’s pattern on file.

We see these costs as bounded, and as acceptable initial costs of the choices we have made. The benefits compound across the platform matrix and across the verification story; the costs scale with the number of platforms targeted and have a logarithmic scaling factor that will eventually level off.

Strategic Implications

Device fleet longevity

The working-set numbers from the Conventional Path section are not abstract. For a productivity application, the gap between a small native footprint and a 150 MB runtime is the difference between feeling responsive and feeling sluggish. For a credentialing application, it is the difference between sub-second handshake completion and a five-second cold-start handshake. Device fleet longevity, in the field, is dominated by working-memory pressure. Reducing that pressure extends the useful life of the install base.

Security surface

A native binary with no JIT, no garbage collector, no interpreter, and a small dependency set is a tractable target for formal-methods auditing. For our broader commercial thesis, this property is load-bearing. A QuantumCredential client compiled to run natively on an iPhone would preserve the constant-time and side-channel-resistance properties that our coeffect verifier reasoned about, because the binary on the device is the binary the verifier saw. Constant-time is a pillar of any verification stack, and it depends on the executing binary matching the audited code.

Cross-cutting verification across all targets

The four-tier verification architecture from Cryptographic Certainty rides on the same substrate as the coeffect tracking described earlier: Tier 1 structural correctness, Tier 2 range and inequality reasoning over QF_LIA and QF_BV, Tier 3 rejection-sampling termination through library-instantiated probabilistic lemmas, and Tier 4 relational reasoning checked against a Rocq-proved rule library. Every tier emits to the same Composer pipeline, and the pipeline emits to every row in the platform matrix. Mobile inherits the full verifier reach because mobile is a row.

Where mobile lands on this roadmap

Linux desktop and embedded targets are working today, and the HelloWayland UI is our clearest demonstration of that arc. Linux mobile (primarily postmarketOS) and Tizen TV are scoped additions to the target tuple matrix. The libc and packaging differences are real, and the architectural work is within reach. Android and iOS are the next tier up in platform-specific shim tooling; our design accounts for them, and we are building the Composer pipeline to produce the right output formats. The path forward from here is tooling and certificate logistics.

Native goes where our Composer IR reaches, and we made that reach part of the design from the start. Mobile and the rest of the matrix are targets we will keep building toward as the pipeline and the per-platform tooling come into place.

Don't Assume All Proofs Are Bulletproof

Thu, 30 Apr 2026 00:00:00 +0000

On April 29, 2026, the cryptographer JP Aumasson posted the following observation on social media:

dont worry, BLAKE3 is post-quantum; unlike the provably secure hash VSH

The remark is compressed enough to read as a quip and substantive enough to deserve unpacking. Aumasson co-designed the BLAKE family of hash functions, and his statement carries a specific technical claim: a hash function with no formal security reduction (BLAKE3) survives the quantum transition, while a hash function with a formal security reduction to a well-studied hard problem (VSH) does not. The relationship between formal proof and practical security, in this comparison, runs in the direction opposite to the one the cryptographic community has traditionally treated as default.

Formal proofs are essential to the discipline. The load-bearing question is what cryptographic security proofs actually establish, and what the post-quantum transition reveals about the conditions of that establishment. Practitioners adopting cryptographic primitives must understand the structure of the proofs supporting those primitives, because the proof’s meaning is conditional on assumptions that may not hold across the threat models the primitive will face during its deployment lifetime.

Game-Based Security Definitions

The modern foundation for cryptographic security proofs is the game-based formalism developed in works such as Katz and Lindell’s Introduction to Modern Cryptography. A security property is defined as an experiment between a challenger and an adversary, parameterized by a security parameter $n$ . The adversary is a probabilistic polynomial-time (PPT) algorithm $\mathcal{A}$ . The experiment outputs $1$ if the adversary succeeds and $0$ otherwise.

For a hash function $H : \{0,1\}^* \to \{0,1\}^{\ell(n)}$ , the collision-resistance experiment $\mathsf{Coll}_{\mathcal{A}, H}(n)$ proceeds as follows. The adversary $\mathcal{A}$ outputs two distinct messages $x_1, x_2 \in \{0,1\}^*$ . The experiment outputs $1$ if $H(x_1) = H(x_2)$ and $0$ otherwise. The hash function $H$ is collision-resistant if for every PPT adversary $\mathcal{A}$ ,

\Pr[\mathsf{Coll}_{\mathcal{A}, H}(n) = 1] \leq \mathsf{negl}(n)

where $\mathsf{negl}(n)$ denotes a function that decreases faster than any inverse polynomial in $n$ . The advantage of the adversary is the probability above; collision resistance requires that this advantage be negligible across all PPT adversaries.

This formalism is precise but conditional. The quantification “for every PPT adversary” defines the threat model. A primitive proven secure against PPT adversaries is silent about adversaries with capabilities outside that class. Quantum adversaries with bounded-error quantum polynomial-time (BQP) algorithms are outside the class. The proof remains valid; its scope simply does not extend to the quantum setting.

The Reductionist Structure

The dominant proof technique for establishing computational security properties is the polynomial-time reduction. Given a primitive $\Pi$ with claimed security in some experiment and a mathematical problem $\mathcal{P}$ believed to be hard, a reductionist proof constructs a PPT algorithm $\mathcal{B}$ that uses any successful adversary $\mathcal{A}$ against $\Pi$ as a subroutine to solve $\mathcal{P}$ with related success probability and runtime.

Concretely, the reduction establishes an inequality of the form

\mathsf{Adv}^{\mathcal{P}}_{\mathcal{B}}(n) \geq \frac{1}{q(n)} \cdot \mathsf{Adv}^{\Pi}_{\mathcal{A}}(n) - \mathsf{negl}(n)

where $\mathsf{Adv}^{\Pi}_{\mathcal{A}}(n)$ is the adversary’s advantage against $\Pi$ , $\mathsf{Adv}^{\mathcal{P}}_{\mathcal{B}}(n)$ is the constructed algorithm’s advantage in solving $\mathcal{P}$ , and $q(n)$ is some polynomial overhead introduced by the reduction. The contrapositive yields the security claim: if $\mathcal{P}$ is hard (no PPT algorithm solves it with non-negligible advantage), then $\mathsf{Adv}^{\Pi}_{\mathcal{A}}(n)$ is negligible for every PPT $\mathcal{A}$ .

The pattern is exemplified by RSA. The factoring assumption states that for every PPT algorithm $\mathcal{F}$ ,

\mathsf{Adv}^{\mathsf{Factor}}_{\mathcal{F}}(n) = \Pr[\mathcal{F}(N) = (p, q) : N = pq, \, p, q \text{ prime}, \, |p|, |q| = n] \leq \mathsf{negl}(n)

The RSA cryptosystem’s one-wayness reduces to a related (and stronger) assumption, the RSA assumption, but the key structural feature is the same: the cryptosystem’s security is bounded by the assumption’s hardness, with polynomial loss. ECDSA’s unforgeability reduces (under the random oracle model and additional assumptions) to the elliptic curve discrete logarithm problem. The Diffie-Hellman key exchange reduces to the computational Diffie-Hellman problem. The cryptographic literature is, to a substantial degree, a catalog of such reductions.

The strength of the reductionist approach is that it grounds practical primitives in well-studied mathematical assumptions. The community has invested substantial effort in understanding integer factoring and discrete logarithms; the difficulty of these problems against classical adversaries is supported by extensive cryptanalysis. A primitive that reduces to factoring inherits the credibility of factoring’s hardness assumption against PPT adversaries. The proof is the bridge that transfers this credibility to the primitive.

The Bidirectional Consequence

The reduction $\mathcal{A} \to \mathcal{B}$ has a logical structure that bears explicit examination. The constructed algorithm $\mathcal{B}$ uses $\mathcal{A}$ as a subroutine, so any improvement in $\mathcal{A}$ ’s advantage propagates (modulo the polynomial loss) to $\mathcal{B}$ ’s advantage. If a new technique gives $\mathcal{A}$ non-negligible success, the same technique, run through the reduction, gives $\mathcal{B}$ non-negligible success against $\mathcal{P}$ . This is the property cryptographers want: tight coupling of the primitive’s security to the assumption’s hardness.

The same coupling runs in the reverse direction. Suppose a new algorithm $\mathcal{S}$ solves $\mathcal{P}$ with non-negligible advantage. The reduction does not directly give an attack on $\Pi$ ; reductions are typically constructive only in one direction. But the contrapositive of the security theorem fails: it is no longer true that “if $\mathcal{P}$ is hard then $\Pi$ is secure,” because $\mathcal{P}$ is no longer hard. The proof did not falsify; the assumption it relied on simply ceased to hold. The primitive’s security claim, as established by the reduction, is no longer in force.

In practice, the failure of the assumption tends to produce direct attacks on the primitive, often through cryptanalysis that exploits the specific structure the reduction relied on. RSA decryption can be performed by anyone who can factor the modulus; the reduction made this an explicit consequence rather than an incidental one. ECDSA signatures can be forged by anyone who can solve the elliptic curve discrete logarithm problem for the public key. The problems’ difficulty already determined that the attacks were possible; the reductions made the dependency precise.

Cryptographers have always understood this bidirectional structure. The reductionist methodology is a contract: the primitive is exactly as secure as the assumption is hard, with explicit polynomial loss. What has changed in the post-quantum era is that the “assumption holds” condition has become contingent on a quantum-computation timeline that may sit inside the deployment lifetimes of primitives currently being shipped. Shor’s algorithm demonstrates that integer factoring and discrete logarithm computation are in BQP, the bounded-error quantum polynomial-time class. A sufficiently capable quantum computer solves both problems efficiently. Against a quantum adversary, the factoring assumption and the discrete logarithm assumption both fail to hold.

The consequence follows directly from the reductionist proofs themselves. RSA, ECDSA, ECDH, EdDSA, Schnorr, and finite-field Diffie-Hellman are all broken against quantum adversaries. The proofs that ground these primitives are correct; their conclusions, in the quantum setting, are silent rather than supportive of the security claim. The community has known this since Shor’s 1994 paper. The post-quantum transition is the operational response to a structural property of the proofs that has been understood for three decades.

VSH and the Inverted Hierarchy

The starkest illustration of the reductionist trap is VSH (Very Smooth Hash), proposed by Contini, Lenstra, and Steinfeld in 2005. VSH is a hash function constructed so that its collision resistance reduces, in the formal sense, to a problem closely related to integer factoring. Specifically, finding a collision in VSH would yield an algorithm for the Very Smooth Number Nontrivial Modular Square Root (VSSR) problem, which Contini, Lenstra, and Steinfeld argue is at least as hard as factoring the underlying modulus $N$ .

The reduction takes the form: if $\mathcal{A}$ finds a collision in VSH with non-negligible advantage, then there exists a PPT algorithm $\mathcal{B}$ that solves VSSR with related advantage, and a further reduction connects VSSR to the factoring of $N$ . Under classical assumptions about the hardness of factoring, VSH’s collision resistance follows.

This is precisely the kind of foundation a cryptographic hash function is supposed to have under the dominant criterion. Most deployed hash functions, including SHA-2, SHA-3, and BLAKE3, lack any such reduction; their security rests on the absence of known attacks rather than on a formal proof of equivalence to a hard problem. The widely deployed hash functions are, in the strict reductionist sense, less rigorously founded than VSH. This was, for two decades, treated as a point in VSH’s favor by cryptographic theorists, even as performance considerations kept it from widespread deployment.

In the quantum setting, this rigor inverts. Shor’s algorithm factors $N$ in polynomial time on a quantum computer. The factoring assumption that grounds VSH’s collision resistance does not hold against quantum adversaries. The reduction that established VSH’s collision resistance, applied to a context where factoring is tractable, yields the consequence that VSH collisions are findable with the same efficiency that factoring is solvable. VSH is broken in the quantum setting as a direct consequence of the proof structure that established its security in the classical setting.

The hash functions without such reductions are unaffected by this argument. SHA-256, SHA-3, and BLAKE3 do not reduce to factoring or to any other Shor-vulnerable problem. Their security is grounded in resistance to known cryptanalytic techniques applied to their specific constructions, and Grover’s algorithm provides at most a quadratic speedup against pre-image search and collision search. At standard output sizes (256 bits and above), this leaves them at or above NIST’s post-quantum security floor of 128 bits.

The hash function with the formal proof falls. The hash functions without the formal proofs survive. This is the substance of the Aumasson observation: the specific formal proofs available before the post-quantum era reduced to the wrong problems. Cryptanalytic hash functions sit beyond that failure mode entirely; their security stands on resistance to the broad class of attacks.

The Post-Quantum Reductionist Response

The cryptographic community’s response to the quantum threat has been substantial and largely successful. The NIST post-quantum cryptography standardization process produced ML-KEM (formerly Kyber) for key encapsulation, ML-DSA (formerly Dilithium) for digital signatures, and SLH-DSA (formerly SPHINCS+) for hash-based signatures. These standards are now part of the formal cryptographic infrastructure of the United States and are being adopted internationally.

The structure of the security proofs for ML-KEM and ML-DSA preserves the reductionist pattern, and reading those proofs at the level of what they actually depend on is part of the deeper “deep tech” work we pursue at SpeakEZ Technologies with the Fidelity Framework. Their security reduces to lattice problems, primarily Module Learning With Errors (Module-LWE). The Module-LWE problem, due to Regev (2005) and its module-lattice generalization due to Langlois and Stehlé (2015), is defined as follows. Let $R = \mathbb{Z}[X]/(X^d + 1)$ be a polynomial ring with $d$ a power of two, and let $R_q = R/qR$ for a prime $q$ . The Module-LWE distribution with parameters $(n, m, q, \chi)$ samples a uniform matrix $\mathbf{A} \in R_q^{m \times n}$ , a secret vector $\mathbf{s} \in R_q^n$ drawn from the error distribution $\chi^n$ , and an error vector $\mathbf{e} \in R_q^m$ drawn from $\chi^m$ , and outputs the pair $(\mathbf{A}, \mathbf{A}\mathbf{s} + \mathbf{e})$ .

The Module-LWE problem is to distinguish samples from this distribution from samples drawn from the uniform distribution over $R_q^{m \times n} \times R_q^m$ . The decision-Module-LWE assumption states that for every PPT distinguisher $\mathcal{D}$ ,

\left| \Pr[\mathcal{D}(\mathbf{A}, \mathbf{A}\mathbf{s} + \mathbf{e}) = 1] - \Pr[\mathcal{D}(\mathbf{A}, \mathbf{u}) = 1] \right| \leq \mathsf{negl}(n)

where $\mathbf{u}$ is uniform in $R_q^m$ . ML-KEM’s IND-CCA security reduces to this assumption with a polynomial loss factor that depends on the specific parameters and the underlying KEM construction (Fujisaki-Okamoto transform applied to a CPA-secure scheme).

This is genuine progress, but the structural feature that produced the VSH problem is preserved. If a quantum algorithm is found that solves Module-LWE efficiently, every primitive whose security reduces to Module-LWE falls together. The lattice problems are believed hard against quantum adversaries, but “believed hard” is the same epistemological status that integer factoring held before Shor’s algorithm. The community’s confidence rests on the cryptanalytic record of failed quantum searches against these problems, with the underlying hardness question itself still open. The reductionist methodology has been re-aimed at quantum-resistant target problems; the structural risk profile carries over. The bet has been re-placed on different mathematics.

SLH-DSA is a different case. Its security reduces to the pre-image resistance of an underlying hash function, which is itself believed quantum-resistant under Grover’s quadratic bound. The reduction here targets a property of the hash function (pre-image resistance) whose security stands on cryptanalytic resistance to a broad class of attacks rather than on any single algebraic assumption. This is closer to the cryptanalytically grounded model of BLAKE3 than to the reductionist model of VSH or ML-KEM, and SLH-DSA’s foundation is correspondingly more conservative. The cost is operational: SLH-DSA signatures and keys are larger and slower than their lattice-based counterparts. This is a recurring pattern: reductionist security to a structured assumption typically affords better performance than cryptanalytic security to a less-structured assumption, and the tradeoff between these is a genuine engineering decision.

This is the engineering register our planned post-quantum work in the Fidelity Framework enters. Post-quantum primitive support is one line of work among several rather than the framework’s central concern; the standardized primitives and their reductionist proofs sit upstream of any compiler we build, and the broader engagement with quantum substrates lives in Quantum Optionality and the categorical structure of the quantum substrate. Our four-tier proof architecture is aimed at the implementation surface around standardized primitives, and the post-quantum primitive support we plan to ship will be built against that surface.

Three Categories of Foundation

Three categories of cryptographic security foundations come into focus, distinguished by what an adversary would have to accomplish to break the primitive.

Reductionist foundations establish security through a polynomial-time reduction to a specific mathematical problem assumed to be hard. The primitive’s security advantage is bounded by the assumption’s solvability advantage, with explicit polynomial loss. RSA, ECC, finite-field DH, and the lattice-based post-quantum standards all fall in this category, with different target problems. The advantage of this category is that the security has a precise formal grounding; the disadvantage is that the security is contingent on a single hardness assumption, and a breakthrough against that assumption breaks every primitive that reduces to it. Reductionist foundations also typically yield concrete advantage bounds that allow precise parameter selection for a given security level.

Information-theoretic foundations establish security without any computational hardness assumption. The one-time pad provides perfect secrecy: for every distribution over plaintexts and every ciphertext $c$ , the conditional distribution of the plaintext given $c$ equals the marginal distribution. This holds against unbounded adversaries, including quantum adversaries, because the security is independent of computational power. Information-theoretic message authentication codes provide unconditional unforgeability under similar assumptions, and quantum key distribution provides information-theoretic security under specific physical assumptions about quantum measurement. The advantage of this category is immunity to any computational advance, including quantum. The disadvantage is operational cost: the one-time pad requires key material as long as the message, information-theoretic MACs require fresh key material per authentication, and QKD requires specific physical infrastructure that does not generalize to typical deployment contexts.

Cryptanalytic foundations establish security through extensive failed attack. A primitive in this category has been subjected to focused cryptanalytic effort by skilled researchers over an extended period, and the cryptanalytic record is the security record. SHA-256, SHA-3, BLAKE3, AES, ChaCha20, and most deployed symmetric primitives fall in this category. The advantage of this category is the diffusion of failure modes: each primitive’s security stands on its own cryptanalytic record, so a change in any single mathematical problem’s hardness leaves the rest of the category unaffected. The disadvantage is empirical grounding: the security claim is supported by the cryptanalytic record rather than derived from a formal proof.

These categories are not exhaustive, and important primitives combine elements of multiple categories. Hybrid post-quantum constructions, for example, combine a classical reductionist primitive (such as X25519, whose security reduces to the elliptic curve Diffie-Hellman problem in Curve25519) with a post-quantum reductionist primitive (such as ML-KEM-768, whose security reduces to Module-LWE) into a single key exchange. The resulting hybrid is secure if either component is secure: an adversary must solve both Module-LWE and the elliptic curve Diffie-Hellman problem to break the hybrid. This is a defense-in-depth strategy that hedges against the failure of either reduction, at the cost of additional bandwidth and computation per handshake.

Implications for Primitive Selection

Practitioners selecting cryptographic primitives need to ask both the standard question (is the primitive proven secure?) and the load-bearing follow-up: what assumption would have to fail for the proof to no longer apply, what the reduction loses, and how robust that assumption is under the threat model the primitive will face. The fragility lives in the assumption; the proof correctly conveys the assumption’s status to the primitive, including the negative case. Migration planning should account for both the algorithm name and the foundation type of each primitive in the cryptographic stack.

For primitives with reductionist foundations targeting Shor-vulnerable problems, the migration is mandatory. RSA, ECC, and finite-field Diffie-Hellman must be replaced. The reductionist proofs grounding these primitives derive their vulnerability against quantum adversaries directly from the proof structure, and no parameter adjustment can recover the security: the reduction does not have parameters that can be tuned to protect against an adversary capable of solving the target problem efficiently.

For primitives with reductionist foundations targeting post-quantum problems, the migration is to these primitives, with the understanding that the security rests on assumptions that have received less cryptanalytic scrutiny than factoring or discrete logarithms received before Shor. The lattice-based standards are the current best available choice for general key exchange and signatures, and they have undergone substantial scrutiny through the NIST process. The structural property remains that a breakthrough against the underlying lattice assumption invalidates the entire family of lattice-based primitives simultaneously. This is the same risk profile that classical reductionist primitives carried before Shor; the community is now operating under it for a different set of assumptions.

For primitives with cryptanalytic foundations, the migration is largely a matter of parameter verification rather than algorithm replacement. AES-256 remains adequate against Grover. SHA-256, SHA-3, and BLAKE3 at 256-bit output remain adequate against Grover. AES-128 requires upgrade to AES-256 to clear the post-quantum security floor. The construction-level security of these primitives is independent of any single mathematical assumption that quantum computing might invalidate, so the migration burden is correspondingly lighter.

For applications where the additional cost is operationally tolerable, hybrid constructions provide defense in depth. The Cloudflare deployment of X25519Kyber768 in production traffic is an example of this strategy applied at internet scale: the hybrid retains the classical security of X25519 against any adversary while adding the post-quantum security of Kyber against quantum adversaries. If lattice cryptanalysis advances unexpectedly, the classical component preserves security against the (still classical) adversary that has not yet built a quantum computer; if classical cryptanalysis advances unexpectedly, the post-quantum component preserves security. The hybrid retains reductionist risk in both components, with the structural improvement that the construction breaks only when both reductions fail simultaneously, a meaningfully stronger requirement than either reduction holding individually.

For our part, the Fidelity Framework’s planned post-quantum primitive support is built to serve exactly this analysis. Each primitive we plan to ship will carry a per-tier certificate naming what our verification covers (memory safety, constant-time discipline, source-to-binary correspondence) and what residual concerns sit outside that scope. The shape of that per-primitive accounting parallels how our verification architecture treats other classes of obligation; see Building Proofs for the Real World for how the range-propagation tier handles physical-safety properties under the same dual-pass discipline. The Cryspen team’s recent “verification facade” discussion and accompanying ePrint paper are companion reading on why per-primitive accounting of this kind matters.

Terms and Conditions Apply

The lesson we take from this is about reading formal cryptography carefully, and about recognizing that proofs are precise instruments whose precision depends on the specific conditions to which they are applied. Inside those conditions, they remain as rigorous as the day they were written. Outside those conditions, the proofs do not protect the primitive. The shift from one regime to the other is the operational substance of the post-quantum transition, and the practitioner has to know, for each primitive in the deployed stack, on which side of the boundary a given claim resides.

A Triangle Without Mystery

Wed, 08 Apr 2026 00:00:00 +0000

The Bermuda Triangle was the great mystery of the 1970s, the kind of story that sold twenty million copies of Charles Berlitz’s paperback and gave Leonard Nimoy a full season of In Search Of… episodes to walk through in that baritone voice. That phenomenon was a product of marketing and pre-Internet pop-culture amplification. By contrast, while this topic area can seem mysterious, the math is firmly grounded in reality and confirms our design. When “the fog clears” and the right organizing principles emerge, we find that adjacent framing had its own way of lining up. We note our findings in order to share how this informs our design in a way that can provide greater capabilities while remaining approachable in a design-time context.

A note on vocabulary before the corners. Few people outside of academic communities converse in this material, and several of the terms are being introduced here for the first time for many readers. The post uses a handful of terms from category theory and order theory: category, functor, groupoid, poset, sheaf, and others. We try to give them a plain-English framing the first time each term appears below. We realize that plain English is doing a lot of work here. The math is real, but it should be possible to connect it back to our work with a willingness to take the casual definitions at face value.

Corner One: Dimensional Types

The Fidelity framework’s Dimensional Type System is our starting point, and has coverage here on this site and in a whitepaper on arXiv. It assigns to each value in a program an annotation that records its physical or mathematical units. Length, time, mass, temperature, currency, dimensionless count: each base unit is a generator of a free abelian group, and the annotation on a compound value is a vector in $\mathbb{Z}^n$ where $n$ is the number of base units in scope.

Operations propagate annotations: addition requires equal vectors, multiplication adds vectors, division subtracts. Dimensional consistency reduces to a system of linear equations over $\mathbb{Z}$ , decided in polynomial time by Gaussian elimination. The result is either a consistent assignment or a contradiction. The engineer never writes a proof, because the inference is complete and principal. This is the Tier 1 fragment of the framework’s verification architecture, and the post on free proofs from dimensional types makes the parametricity argument explicit.

The dimensional system has another structure that the parametricity story does not emphasize. The annotations live on every node of the Program Semantic Graph, on every operation in every MLIR dialect the program is lowered through, and on the residual evidence in the binary. The lowering passes are required to preserve them, and our design for a dual-pass architecture can re-discharge this requirement at each pass.

The compilation pipeline has an ordered structure that turns out to carry significant meaning in the mathematical framing introduced in the corners that follow. The stages form a chain: source < PSG < high-level MLIR < mid-level MLIR < low-level MLIR < binary. Each stage carries its $\mathbb{Z}^n$ -valued annotation bundle, and each lowering pass translates annotations from one stage to the next. The property the dual-pass architecture enforces is referenced as “compositionality”: two consecutive lowerings produce the same annotations as one direct lowering.

The classic category theory diagram: three objects, two morphisms $f$ and $g$ , and the composite morphism $g \circ f$ that the category laws require to exist whenever $f$ and $g$ are end-to-end. Every object also carries an identity arrow from itself back to itself, often omitted from the diagram for clarity.

Anyone who has done functional programming already has the intuition for composition: $g \circ f$ means running $f$ then $g$ gives the same result as running the combined operation directly. The pipeline’s compositionality condition is the same equation applied to annotation-preserving transformations rather than to values, and as we discovered, that condition is the defining equation of a larger mathematical structure that connects the framework’s engineering to results we didn’t expect when the pipeline was first designed.

Corner Two: Tarau’s Groupoid

Paul Tarau’s work on bijective combinatorial encodings starts from a question that sounds elementary and turns out to be deep. Can finite mathematical objects of different kinds (natural numbers, finite sets, binary trees, lambda terms, multisets, permutations) be put into computable bijection with one another such that the round trips compose? If you can convert a binary tree to a natural number and back, and a natural number to a finite set and back, can the conversions be made to compose so that going tree → number → set → number → tree returns the original tree?

Tarau’s answer is to route every encoding through a common type, called the Root, and to require that each encoder/decoder pair forms an isomorphism between its source type and the Root. With this discipline the round-trip equation falls out by composition. Any conversion between two sources is the composition of one source-to-Root encoding with one Root-to-target decoding, and the round trip is the composition of those compositions, which the isomorphism property guarantees is the identity.

The structure here is what category theorists call a connected groupoid: multiple objects (the encoded types) linked through Root-mediated isomorphisms, where every arrow has an inverse. Collapsing all objects to Root via those isomorphisms yields a group (the automorphisms of Root), which is a groupoid on a single object. Either reading produces the same round-trip guarantees. The natural numbers $\mathbb{N}$ are the canonical Root choice in much of Tarau’s work because their order structure makes the encodings effective.

We read this as a functor from a poset to a category. The base poset, in Tarau’s case, is the simplest one available: a single point with no order relations at all. The target category is the category of types and isomorphisms. The functor sends the single point to the Root. Compositionality is automatic because there are no non-trivial relations in the base for composition to fail on. Tarau’s contribution is the discipline (every encoder routes through Root) and the proof that the discipline is enforceable across a wide family of finite mathematical objects.

This showed us that our use of $\mathbb{Z}^n$ as the canonical representation for dimensional information is a Root in exactly Tarau’s sense: every annotation, regardless of which physical or mathematical domain it originates in, translates to a vector in $\mathbb{Z}^n$ , and the round trip is an isomorphism by construction. The compositionality equation Tarau requires (round trips compose) is the same equation the dual-pass architecture enforces across MLIR lowering passes, and the same equation our Program Hypergraph design had been relying on for years under the name “annotation consistency across hyperedges.” Tarau gave us the categorical name for a discipline the engineering had already committed to.

Corner Three: Cellular Sheaves

The third corner introduces the term that does most of the heavy lifting here: the sheaf. Sheaves originated in algebraic topology in the 1940s and acquired their modern abstract form in the work of Alexander Grothendieck. The recent paper on cellular sheaves on finite posets (arXiv:2502.15476) is an approachable introduction and treats them as a working tool for compositional information flow, with the abstract apparatus stripped down to what an engineer can use directly. Their definition: a cellular sheaf on a finite poset $P$ attaches a stalk (a value, a set, an algebraic structure) to each element of $P$ , together with structure maps $D(s_1 < s_2)$ for each ordered pair, satisfying the compositionality equation

D(s_0 < s_1) \,;\, D(s_1 < s_2) \;=\; D(s_0 < s_2).

A global section of the sheaf is an assignment of one element of each stalk such that the structure maps send the chosen elements correctly. The paper’s central computational fact is that to verify a global section it suffices to check the structure-map equations on the edges of the Hasse diagram of the poset. Transitivity propagates through compositionality, so checking the immediate edges checks every longer chain.

The paper treats hypergraphs as a special case via the membership poset. A hypergraph becomes a bipartite poset where vertex $v$ is below hyperedge $h$ when $v \in h$ , and a sheaf on this poset is a cellular sheaf in the standard sense. Sheaves carry their own measurement apparatus called cohomology, which turns the failure of local consistency to extend globally into a measurable algebraic invariant. Some sheaf cohomology groups vanish on graphs (one-dimensional cell complexes) but become non-trivial on cell complexes of higher dimension, and those non-vanishing groups are exactly the obstructions that prevent extending the data attached at individual items to a globally consistent assignment across the whole poset.

Read this as a functor. The base poset is whatever poset the application requires (a hypergraph’s membership relation, a process calculus’s reachability relation, a compilation pipeline). The target category is whatever category of values the application carries (abelian groups, distributions, capability lattices). The functor sends each poset element to its stalk and each ordered pair to its structure map, subject to the compositionality equation. The paper’s contribution is the algorithmic infrastructure for computing global sections and cohomology on arbitrary finite posets, including the hypergraph case.

As with the other two corners, the engineering was placed before the sheaf vocabulary arrived. Our work on the Program Hypergraph had been carrying joint constraints across three or more operations, alongside a Clifford algebra grade system that uses Cayley table elimination to verify that geometric products preserve their structural zeros through the computation graph. We recognize now that both of these are sheaf conditions in retrospect: the PHG’s “annotation consistency across hyperedges” is a global section condition on the hypergraph membership poset, and the Cayley table’s grade preservation through training is a global section condition on the differentiation sheaf. The cellular sheaf paper gave us the formal infrastructure to see them as instances of the same equation, and to use the cohomological machinery to diagnose the cases where local consistency fails to extend globally.

The hyperedge structure that sheaf theory justifies is also what makes our framework a general systems platform rather than a single-target compiler. Joint constraints across three or more operations are the norm in AI and machine learning workloads, in spatial dataflow on FPGAs and NPUs, and in the quantum substrate where gate fidelity, qubit connectivity, and error correction all impose irreducibly multi-way constraints on the circuit graph. The PHG is the structure that holds all of those domains in the same compilation framework, and the sheaf theory is what confirms that no simpler structure could do the same job.

The Triangle

The three corners are three instances of the same categorical construction:

	Base poset	Target category	Compositionality
DTS	Compilation pipeline	$\mathbb{Z}^n$ -annotation bundles	Enforced by dual-pass
Tarau	One point	Types and isomorphisms (Root-mediated)	Automatic; base is trivial
Cellular sheaves	Arbitrary finite poset (hypergraph membership)	Application-determined stalks	Theorem: check edges of Hasse diagram

In each case the object of interest is a functor from a finite poset to a target category. In each case the compositionality equation is what distinguishes a usable structure from a collection of disconnected encodings. In each case the value of the discipline is that it lets information flow from one part of the structure to another without the annotations degrading through composition.

The Dimensional Type System is the case where the base is a finite linear order (the compilation pipeline) and the target category has enough structure (abelian groups and homomorphisms) for compositionality to be checked algorithmically. Gaussian elimination decides the structure-map equations at each stage, and the dual-pass architecture is the witnessing mechanism for the global section.

Tarau’s groupoid is the degenerate case in which the base is a single point. The discipline reduces to “every encoder passes through Root.” Compositionality is automatic because there is nothing to compose against, and the entire content of the discipline is at the level of the target category.

The cellular sheaf framework is the general case in which the base may be any finite poset and the target category may be any category in which composition is meaningful. The hypergraph case unlocks higher cohomology, which is the categorical reason joint constraints in kernel fusion or spatial tile placement cannot be reduced to binary edges. The obstruction classes that joint constraints generate live in cohomology groups that do not exist on a one-dimensional cell complex.

This is why the Program Hypergraph is necessary. The PSG carries every constraint that can be represented as a relationship between two operations and discharges them through the dual-pass architecture cleanly, but the joint constraints that arise once three or more operations need to agree on a shared resource live in cohomology groups a one-dimensional semantic graph structure cannot reach. It would be a mistake to flatten those joint constraints into pairwise edges. The structure would then lose the agreement that has to hold across all participants at once. The PHG’s hyperedges are the structure that holds the more complex agreements together.

The Engineering Flywheel

Recognizing the three corners as instances of one categorical construction means that infrastructure built for any one of them transfers, in principle, to the other two.

From the Dimensional Type System, the engineering payoff is the dual-pass witnessing strategy: enforce the compositionality equation locally at each edge of the Hasse diagram, and let transitivity propagate the global section. The cellular-sheaf paper’s central computational fact, that edge-local checks suffice, is exactly the strategy our framework uses operationally. We arrived at the strategy from MLIR engineering pressure, and the paper supplies the categorical justification for why edge-local checks suffice.

From Tarau, the engineering payoff is the Root discipline: a single canonical type through which all encoders pass, with isomorphisms guaranteeing round-trip consistency. Our use of $\mathbb{Z}^n$ as the canonical representation for dimensional information is a Root in exactly Tarau’s sense. Every dimensional annotation, regardless of which physical or mathematical domain it originates in, gets translated to a vector in $\mathbb{Z}^n$ , and the round trip from a domain-specific type to $\mathbb{Z}^n$ and back is an isomorphism by construction. The Tarau discipline is what makes the compilation poset’s structure maps composable in the first place.

From cellular sheaves, the engineering payoff is the cohomological diagnosis of two situations our framework already encounters in its design. The first is a conservative range finding from Tier 2 propagation, and the cleanest place to ground it is the constraint-resolution problem at the heart of clinical decision support.

Here is an example that relates to a problem we are working on for a healthcare client, the kind of workload our framework’s verification stack is designed to address. The behavior described below is how we intend the analysis to work as the verification stack comes together.

Given a patient’s state (weight, age, organ function, current vitals) and a clinical indication, find a dosing regime that satisfies every safety constraint simultaneously and stays inside the therapeutic window.

This example is for a dosing calculation for a continuous infusion whose plasma concentration the prescriber needs to keep above the minimum effective level and below the toxic threshold across the full duration of the infusion:

open Fidelity.Physics.Clinical

// Therapeutic window for the drug
let min_effective = 5.0<mg/L>
let max_safe = 20.0<mg/L>

// Patient state, derived from chart at order time
let patient_weight = Range(2.5<kg>, 4.0<kg>) // term neonate
let creat_clearance = Range(15.0<mL/min>, 40.0<mL/min>) // immature renal fn

// Drug pharmacokinetics, parameterized by patient state
let volume_dist = volumeOfDistribution patient_weight
let elim_rate = eliminationRate creat_clearance // k = CL / Vd
let infusion_rate = Range(0.5<mg/kg/hr>, 2.0<mg/kg/hr>)
let infusion_time = Range(0.5<hr>, 6.0<hr>)

// Concentration at the end of an infusion (one-compartment model)
let c_at_t = fun t ->
 (infusion_rate * patient_weight / (elim_rate * volume_dist))
 * (1.0 - exp(-elim_rate * t))

let peak_concentration = c_at_t infusion_time

For the linear and ratio-of-linear parts of this calculation, Tier 1 range propagation produces an exact bound from the input ranges and a proof tool discharges the comparison against min_effective and max_safe cleanly. At the exp(-elim_rate * t) term, the analysis switches modes. The argument -elim_rate * t lies in a known interval (negative, with bounds derived from the elimination-rate and infusion-time ranges), and the framework’s generic interval rule for exp returns the conservative bound $[0, +\infty)$ . Propagating that through the rest of the calculation produces a peak_concentration bound wide enough to overlap both the sub-therapeutic region below min_effective and the toxic region above max_safe, and the compiler surfaces the result as a conservative finding rather than a clean verdict. The bound is sound (the true concentration always lies inside it) and it is wider than the therapeutic window needs.

The sheaf reading is that the global section over the computation DAG fails to be tight at the exp node: the structure map there (the generic interval extension of exp) over-approximates, producing a stalk value wider than the true image of the upstream interval. The topology of the DAG is fine; the deficiency is in the stalk computation, and the fix is a refined structure map (the monotonicity lemma) rather than a change in the base space. The engineering reading, which is what the prescriber actually needs, is that the compiler identifies the exact node where the analysis loses precision and names the Tier 3 lemma missing from Fidelity.Lemmas.Mathematics for this calculation. Specifically: a parameterized lemma of the shape

\forall a, b \in \mathbb{R} \text{ with } a \le b : \exp([a, b]) = [\exp a, \exp b]

proved once from the monotonicity of $\exp$ in an external proof assistant, instantiated by the compiler against the concrete interval the PSG carries at the exp node, and consulted to tighten the downstream bound. Every additional clinical model the library covers improves the set of patient-and-drug combinations the framework can give a safety verdict on. The cohomological framing is what makes “we need a lemma about $\exp$ on a negative interval” a categorical specification the lemma library can be planned against rather than a vague blind spot in requirements. Each new lemma shrinks the space of unaddressed cases, and an engineering team can track exactly which models remain uncovered and prioritize accordingly.

The second situation is kernel fusion under joint resource constraints, the original motivation for the Program Hypergraph in Hyping Hypergraphs. Suppose the compiler wants to fuse three operations that share an on-chip buffer on an FPGA tile: a load that reads sensor input into the buffer, a transform that updates the buffer in place, and a store that hands the buffer’s contents to a downstream consumer. The fusion is sound only if all three operations agree simultaneously on the buffer’s allocation region (BRAM block at a specific address), its lifetime (the duration of the fused kernel), its dimensional annotation (Span<float<celsius>>), and its access pattern (read, then read-modify-write, then read). A graph-based representation can record these properties pairwise:

graph LR
L["load<br>BRAM_0, t=[0,t1]<br>celsius, write"] -->|"buffer ref"| T["transform<br>BRAM_0, t=[t1,t2]<br>celsius, rw"]
T -->|"buffer ref"| S["store<br>BRAM_0, t=[t2,t3]<br>celsius, read"]

The pairwise edges admit annotation assignments the actual hardware cannot honor. Nothing in the graph rules out a Tier 1 lowering pass that, locally consistent on each edge, decides to allocate BRAM_0 for the load-transform pair but BRAM_1 for the transform-store pair, because the joint three-way constraint “all three operations share the same physical buffer” is not expressible as a property of any single edge. The hyperedge that the PHG provides is the structure that can express it:

graph LR
L["load"] --- H{{"hyperedge:<br>BRAM_0, celsius,<br>t=[0,t3], shared"}}
T["transform"] --- H
S["store"] --- H

The cohomological reading is that the obstructions to a globally consistent annotation assignment live in cohomology groups that exist only on cell complexes of dimension greater than one, and a graph is a one-dimensional cell complex. The engineering reading is sharper: if you try to express kernel fusion with binary edges, the representation will admit annotation assignments that the actual hardware cannot honor, because the binary edges do not have the structural capacity to rule them out. Hyperedges are the minimum structure that does. This is a categorical impossibility result on what graphs can carry, and the PHG is the minimal cell complex on which the obstruction classes for joint constraints can even be stated, much less resolved.

In most cases none of this is meant to surface at the engineer’s desk. The math justifies the tooling, and our work on Composer and Clef is aimed at keeping the math invisible to the engineer who only wanted to write a clinical dosing calculation or a fused kernel. We intend that a software engineer can write a one-compartment infusion model in plain Clef with Fidelity.Physics.Clinical measure types and get either a clean peak_concentration bound that fits the therapeutic window or a conservative finding that names the lemma our library still needs. A developer can write a kernel fusion that shares an on-chip buffer across three operations and get either a fused schedule that respects every joint constraint or a diagnostic that points at the specific constraint the schedule cannot honor. In both cases the engineer reasons in the vocabulary of their domain, and the categorical machinery underneath earns its place by staying out of view. The cohomological diagnosis is our way of explaining to the verification stack, in terms it can act on, what to do at the moments the engineer’s tools need help. The intended experience is that the tools work, the diagnostics are specific, and the cases where an engineer has to consult the lemma library directly are rare and well-isolated.

Leading Indications

A triangle as metaphor is rarely the end of a story. In our view, it likely signals a fourth corner waiting to be identified: some other concept/object that ties the three together from above, or some application that requires all three at once.

The fourth corner this triangle suggests to us is a verification framework that uses multiple sheaves over the same base poset simultaneously. The four-tier Hoare correspondence is one sheaf over the compilation poset (the functional-correctness sheaf, with stalks ranging from $\mathbb{Z}^n$ at Tier 1 to relational pRHL judgments at Tier 4). Access Hoare logic (Beckmann & Setzer, arXiv:2511.01754) is a second sheaf over the same poset, with capability-lattice stalks. Symmetry Hoare logic (Mehta & Hsu, OOPSLA ‘25, arXiv:2509.00587) is a third sheaf, with group-action stalks. Each sheaf has its own global-section problem, its own dual-pass discharge, and its own cohomological obstructions. They share the compilation poset and the Tarau-style Root discipline, and they can be checked simultaneously without interfering with one another.

Our framework is designed with the operational infrastructure for this. The PSG is meant to carry the annotations for any number of sheaves simultaneously, the MLIR pipeline preserves them as opaque attributes, and the dual-pass architecture re-discharges them at each lowering. The triangle adds the categorical justification: the same base poset supports any number of compatible sheaves, and the global sections of distinct sheaves do not interfere because they are checked against distinct stalk categories. The same map applies to access control, symmetry verification, and probabilistic relational reasoning, which is why the map is worth drawing. It confirms the direction we have taken and marks the paths we intend to follow next.

An engineer working in Clef will write a clinical dosing calculation, a signal processing pipeline, or a fused FPGA kernel, and our framework will return either a report that the calculation is sound across the declared input ranges or a specific diagnostic naming what would need to change to make it sound. Memory safety, dimensional consistency, range bounds, and physical symmetries are all checked at compile time. Cryptographic verification is available for the modules that need it. All of it runs on the dual-pass certificate machinery the categorical content confirms for us here.

An FPGA example is already realized in a direct, approachable form in HelloArty, where Clef source compiles through Composer to a working Artix-7 bitstream with our dimensional and coeffect guarantees intact across the boundary into hardware. The example artifacts directory shows what the diagnostics look like in practice. That diagnostic is the categorical content outlined here, surfaced in the terminal in a form the engineer can act on directly. Working in Clef will provide more than reliable memory safety and expressive ergonomics in a concurrency-based functional model. Our goal is to make it a development environment that rewards a variety of software engineering tasks with quiet, efficient builds and the ability to maintain depth in high-leverage design spaces when the domain calls for it.

The fog around this triangle cleared in the way actual fog usually does: first gradually, then all at once. Paul Snively mentioned Tarau to Houston, and the link to the sheaf paper shortly followed. The constellation of sources that shed light on our engineering track continues to expand, and what has emerged is more guidance to build on. It’s gratifying to find so much in mathematics that confirm our direction, and it does significant work in showing the path forward. We will continue de-mystifying this domain for ourselves and for others, in theory and in practice as the work continues.

A Runtime Revolution, sort of...

Mon, 06 Apr 2026 00:00:00 +0000

The Fidelity framework is primarily designed to target hardware in the most direct manner possible. You know the acronyms: CPUs, GPUs, FPGAs, MCUs, NPUs, and other accelerators too. Every compilation target goes through Alex (the “Library of Alexandria”), our MLIR middle-end, where dimensional verification, escape analysis, and BAREWire schema derivation all happen in one place. It is the heart of our framework.

So why is a JavaScript target cause for excitement?

We have to admit, this is a special case. We have an affinity for Cloudflare’s cloud edge model for its scale, speed, and security. Cloudflare Workers execute in V8 isolates, so JavaScript is the deployment format for edge workloads on their platform.

Fable is going to stay where it’s been, serving the F# and broader .NET ecosystem that depends on it. Partas.Solid frontends, WrenHello’s Fable-driven WebView, and Fidelity.CloudEdge’s existing F# bindings continue on the well-understood F#-through-Fable path. What we’ve been designing around is a parallel Clef-native path to JavaScript that lives inside Composer’s MLIR middle-end rather than as a separate pipeline alongside it.

The closer architectural reference point for what we’re doing is not Fable. It is js_of_ocaml, which has been compiling OCaml to JavaScript since 2010 by lifting bytecode into an internal SSA-style IR, running optimization passes, and emitting JavaScript through a conventional compiler back end. js_of_ocaml proved at production scale, over fifteen years, that an ML-family language can reach JavaScript through a compiler IR rather than through source-AST walking. In 2024 the same project shipped wasm_of_ocaml, a WebAssembly backend sharing the same IR with the JavaScript backend. That’s the multi-target-from-shared-IR pattern, and it’s exactly what Composer’s middle-end generalizes over MLIR’s dialect infrastructure.

Then recently, Google published an RFC to upstream JSIR (JavaScript Intermediate Representation) into MLIR. As mentioned above, using Multi-Level Intermediate Representation is the central strata for the Fidelity framework’s compilation path. So while we were looking at LLVM for certain ’legacy’ targets and other back ends to carry to specific types of processors, the introduction of a JavaScript pathway through MLIR was a pleasant surprise. For our purposes, JSIR places JavaScript inside the same lowering pathway that every other Clef target already uses. That means the JavaScript path can now go through Alex, through the same verification passes, through the same BAREWire schema derivation, and to a dedicated “BackEnd” via JavaScript to be packaged for various uses.

The technical details are worth reading if you want to understand how JSIR’s ops map to Alex’s dialect infrastructure and where the trust boundaries fall. This post is about what the unification means in practice.

Why This Matters for Cloudflare Workers

The Fidelity framework deploys actors to two substrates. Native actors run as OS processes with IPC and shared memory. Edge actors run as Cloudflare Workers, Durable Objects in V8 isolates. The unified actor architecture describes how Prospero supervisors and Olivier workers communicate across this boundary using BAREWire.

The Fable-based path handles type safety for JSON-over-HTTP traffic by compiling shared F# types to both sides of the wire: native .NET on the server, JavaScript on the client, with the same source definitions producing compatible shapes by construction. That closes the loop for the SAFE-stack request/response pattern and has been production-tested for years. BAREWire’s cross-substrate binary framing is a different concern. The critical question for binary framing is whether the native serializer and the JavaScript deserializer agree on the byte layout, and before JSIR, that agreement was verified by testing. If the two compilers disagreed about field order in a discriminated union, the mismatch showed up at runtime, in production.

With JSIR in Alex, both serializers derive from the same BAREWire dialect ops. The pipeline forks after the schema is derived, not before. Cross-substrate compatibility becomes a structural property of the compiler as opposed to after-build validation using standard testing. The design-time specification article walks through each verified property and how it behaves after emission to JavaScript.

BAREWire at the Wire Boundary

Our BAREWire format was designed for exactly this kind of cross-substrate communication. The binary wire format encodes type structure in the layout of the bytes. No field names. No schema negotiation. The deserializer reads positions, not keys.

What JSIR adds is confidence that the JavaScript-side deserializer was produced by the same pipeline that produced the native-side serializer. The schema identity that BAREWire enforces at the wire level now has a compilation-level counterpart: both sides of the conversation were lowered from the same verified IR.

This is where the dimensional type system and BAREWire intersect in a new way. Schema identity serves as a proxy for dimensional agreement. If the dimensional structure of a type changes, the schema changes. If the schema changes, the tag changes. If the tag changes, the receiver rejects the frame. Dimensional disagreement surfaces as transport-level rejection, before any handler code executes. The Schema Identity section of the JSIR article covers this mechanism in depth.

How the Runtime Carries the Freight

Here’s the part that gets genuinely fun if you like programming language history. Clef has no obj type, no null, no universal root. Within Clef source code, every value has a specific static type and there is no escape hatch. JavaScript, meanwhile, has obj everywhere and null as a first-class citizen. How does a compiler with a closed type system emit safe, useful code for a runtime whose default vocabulary is open and dynamic?

The answer is historical, and it runs deeper than the modern ecosystem discussion usually lets on. Brendan Eich designed JavaScript in 1995 with Scheme as one of his explicit references. JavaScript’s Object, in its essential structure, is a direct descendant of LISP’s association lists and hash tables. A JavaScript object is a tagged associative structure: keys (property names) are accessible at runtime, values carry their type tags (typeof, Array.isArray, instanceof), and the shape is observable through standard reflection. V8’s hidden classes, which optimize stable-shape objects by specializing access paths, are the modern rediscovery of Common Lisp’s DEFSTRUCT with declared slots. The same pattern, different vocabulary, forty years apart.

The architectural consequence is that Clef’s interop layer inherits four decades of work on narrowing tagged dynamic data to static types. Typed Racket, which took dynamically-typed Racket (a LISP descendant) and added compile-time type annotations that produce runtime validators, has been refining this pattern since the 2000s. The pattern is: declare the target type at compile time, and the compiler generates a validator that walks the runtime tags according to the declaration. If the runtime value matches the shape, you get a typed result. If it doesn’t, you get a structured error. No assumptions, no partial access to malformed data, no crashes mid-handler.

When Clef code says response.Json() : UserProfile, the compiler inspects the PSG to see what UserProfile is structurally (field names, types, nullability, nested shapes) and emits a validator that reads the runtime Object by walking its property-name tags and checking its value type-tags. The return type is Result<UserProfile, DeserializationError>. Missing required field? Error. Wrong type for a field? Error. Unexpected null? Silently becomes None if the field is Option<_>, Error otherwise. The Clef programmer pattern-matches on the Result; the validator is invisible code the compiler generated from the type annotation.

This is the fun academic twist on a real-world production system: the runtime is actually able to carry the freight. V8 carries runtime tags: property names, typeof results, prototype chains, hidden classes. Those tags are the LISP-descended metadata that make shape-directed narrowing possible at all. If JavaScript were genuinely untyped at runtime (no tags, opaque bytes), the approach wouldn’t work. Because JavaScript’s Object is structurally a tagged associative value in the LISP tradition, the mature LISP-family techniques for narrowing dynamic tagged data to static records transfer directly. Clef’s compiler gets to be clean-slate static with no runtime type pollution; V8’s runtime gets to be LISP-descended dynamic with full tag visibility; the schema-directed narrowing at the interop boundary bridges the two using a pattern that has been production-tested since Typed Racket.

BAREWire handles the complementary case, when the tags live in bytes at known positions rather than in runtime metadata. Both patterns are projections of the same principle: narrow dynamic tagged data to static types using compiler-generated code. The wire-side freight is carried by the byte layout; the object-side freight is carried by the runtime’s tag system. In both cases, the compiler knows the target shape and emits the right narrowing; in both cases, the programmer sees typed results, not raw access to untrusted data.

Streaming Tokens from Containers

The unification also opens a path for inference streaming. When a BitNet ADM running in a container generates tokens, each token can become a BAREWire frame, streamed through a Worker to a client over WebSocket.

The conventional approach is Server-Sent Events with JSON. Every frame carries field names as strings. The schema is rediscovered from every message. A typical token payload is 60+ bytes of redundant structure.

With BAREWire, the same token is roughly 18 bytes. No field names. No JSON parsing. The frame is self-contained, independently parseable, and typed by the discriminated union that the compiler verified. The streaming inference article describes the full path from spatial dataflow inside the container through demand-driven frame emission to WebSocket relay at the edge.

Multiple concurrent inference streams multiplex over a single WebSocket, demultiplexed by correlation ID on the client. Each stream accumulates independently. With MoQ/QUIC in the future, each stream could become a separate QUIC stream with independent ordering and no head-of-line blocking.

What Would Not Change

JSIR would affect the compilation pipeline. It would not affect the actor model’s design, the management API surface, or the deployment infrastructure. Olivier workers would still communicate via WebSocket. Prospero supervisors would still manage lifecycle. BAREWire would still carry the messages. The RAII patterns for actors would be unchanged.

Fable continues as the F#-to-JavaScript path for everything already built on it: Partas.Solid components, WrenHello’s Fable-rendered WebView layer, the F# Worker bindings in Fidelity.CloudEdge. Two coexisting models reach JavaScript through different routes. The F#/.NET path goes through Fable, forced to source-AST walking because the .NET CLR’s IL erases F# idioms before any IR stage could recover them. The Clef path goes through Composer-and-JSIR, free to use a compiler IR because Clef owns its post-frontend representation end-to-end. Neither replaces the other; each serves the source language it was built for.

Related to this, Fidelity.CloudEdge itself will likely transition to FSharp.CloudEdge at some point, distinguishing itself as a direct F# implementation for the F#/.NET ecosystem. The current naming conflates the F# binding layer with the broader Fidelity framework; renaming better reflects what the package actually is, the F# community’s Cloudflare SDK, independent of what Clef and Composer are doing. Details on timing, community maintenance (likely through fsprojects), and the transition plan will be worked out over time. The point for this post is that the F# path and the Clef path are going to be distinct first-class citizens with distinct identities, not one subordinate to the other.

The Fidelity.CloudEdge (or FSharp.CloudEdge, post-transition) management layer would still provision Workers, create D1 databases, and deploy scripts through the same REST clients. Whether the actual Worker runtime code was compiled by Fable or by Composer via JSIR would be invisible to the management layer.

Where We See the Trust Boundary Moving

The part we are most interested in is what JSIR could do for the trust boundary between native and edge actors. The JavaScript itself would still be dynamically typed and garbage collected. The design-time contract between actors, enforced by our BAREWire schema identity, could extend further at that boundary than the alternatives surveyed below reach today.

The closest prior art is Erlang/OTP. The BEAM VM provides runtime pattern matching on tagged tuples at the receive site, and OTP’s supervision trees enforce structured concurrency across distributed nodes. Erlang gets closer to verified actor communication than anything else in production. But the verification is at runtime: a mismatched message crashes the receiving process, and the supervisor restarts it. The contract is enforced by convention and testing, not by the compiler. Elixir inherits the same model with improved ergonomics but the same runtime trust boundary.

Conventional edge deployment frameworks don’t get even that far. gRPC-Web serializes to protobuf but relies on code generation that is disconnected from the application’s type system. tRPC provides end-to-end TypeScript types but those types are erased at runtime and carry no verification beyond what TypeScript’s structural checking provides. Remix and Next.js server actions cross the client/server boundary through JSON serialization with no schema enforcement at all.

With JSIR in the pipeline, we anticipate that the BAREWire schema governing actor communication would be derived from verified, well-structured types and records in a shared set of data specifications. The native serializer and the JavaScript deserializer would both lower from the same MLIR ops. Schema identity at the wire level would become a structural consequence of the compilation, not a property validated after the fact. The verification internals describe what the pipeline is designed to enforce: dimensional consistency, memory safety, representation fidelity, optimization correctness. The JavaScript binary would not be fully verified the way a native binary is. The contract between actors, the messages they exchange, and the schemas they enforce, could be verified at design time and carried through to the wire format. For an edge framework carrying that contract to the wire, we have found no other representative implementations in the standing literature we have reviewed.

Looking Forward

The posit arithmetic design describes how the DTS would select numeric representations based on dimensional range analysis. On native targets, this means tapered precision concentrated where the computation operates. On JavaScript, it means IEEE 754 float64 every time. We’re exploring how the compiler could surface this divergence as a design-time diagnostic so the developer can make informed decisions about what computation belongs at the edge and what belongs on native hardware.

JSIR also opens a path beyond Workers that we’re eager to explore. The WREN stack uses Partas.Solid components rendered in a system WebView, with native Clef logic communicating over BAREWire through a local WebSocket. Today, Partas.Solid compiles F# to JavaScript through Fable for the front-end layer, and we expect that to continue for F# frontends. What JSIR adds is an additional option: Clef-authored frontends could be compiled by Composer through the same verified middle-end as the native backend. A WREN stack whose frontend was written in Clef would have both sides of the IPC boundary going through Alex, with the same BAREWire schema derivation governing the messages between them. The two paths coexist: F#-and-Fable on one, Clef-and-Composer on the other, both producing JavaScript that runs in the same WebView and talks to the same native backend through the same BAREWire wire format.

The three articles in the JavaScript targeting section cover the technical foundations we’re building toward: how JSIR welds to our pipeline, how verified properties could carry through to emission, and how BAREWire would enable streaming inference from containers through Workers to clients.

JavaScript You Can Be Proud Of

Credit where it’s due: “JavaScript you can be proud of” is Fable’s tagline, coined by Alfonso García-Caro years ago. The phrase belongs to Fable, and Fable earned it. Composer-through-JSIR aims at the same quality from a different direction, and we borrow the banner in that spirit.

What we don’t get on the JavaScript target. Dimensional types don’t survive to the runtime; they inform representation selection in Alex and become debug metadata at emission. Deterministic memory management isn’t available; the V8 garbage collector owns the heap, and native-side techniques like arena allocation, stack allocation for non-escaping values, and region-based lifetime management don’t translate. Representation selection is trivial; JavaScript’s Number is IEEE 754 float64 and nothing else. Posit arithmetic, fixed-point, and tapered precision all belong to native targets. The compiler’s best work on those axes doesn’t reach the JavaScript output.

What we do get is substantial. We get full use of what JavaScript actually offers. We get JavaScript’s LISP heritage via V8’s tagged associative structures; schema-directed narrowing works because the runtime carries the tags that make it work, the same pattern Typed Racket has been refining since the 2000s applied to the runtime JavaScript happens to provide. We get BAREWire over WebSocket, which preserves typed structure across the erasure boundary by carrying it in the byte layout rather than relying on runtime type information. We get byte-identical cross-substrate communication because the native serializer and the JavaScript deserializer both lower from the same MLIR ops in the shared middle-end. We get SIMD where it helps numerical work, Promise-integrated async via JSPI once runtime support settles in, and Component Model interoperability as it matures in 2026. Within the scope of what JavaScript can be, these are substantial advantages that ordinary JavaScript toolchains do not provide.

The security argument stands on its own. The js_of_ocaml-shaped compile-through-IR approach produces JavaScript that does not depend on the npm package ecosystem at runtime. The Clef standard library compiles through the same pipeline as application code, the same way OCaml’s stdlib compiles through js_of_ocaml; the output has no node_modules, no transitive dependency graph, no trust decisions about other authors’ code shipped into production. Every published npm supply chain incident (typosquatting, compromised maintainer accounts, transitive takeover, protestware, outright malicious packages) has structural immunity here. The artifact’s bytes are traceable end-to-end to auditable Clef source. For security-sensitive domains where this property is contractual or regulatory (defense, finance, healthcare, regulated edge infrastructure), this alone justifies the work.

A Principled Compromise

For a framework designed around native compilation, JavaScript through JSIR would become another backend reached through the same dialect infrastructure, subject to a subset of the verification passes the native targets run. That convergence onto our tool chain carries both performance and security positives, and it keeps the JavaScript output traceable to the same Clef source as every other artifact we produce. We will keep designing toward that path as the JavaScript targeting work comes into place alongside the native targets it descends from.

Building Proofs for the Real World

Thu, 02 Apr 2026 00:00:00 +0000

Beyond Dimensional Consistency

Our companion post that precedes this one expands on the formal lineage connecting Reynolds’ abstraction theorem and Wadler’s parametricity result to the Dimensional Type System’s design-time verification. That lineage is one of several converging influences on the DTS design. Kennedy’s Units of Measure [5] demonstrated that dimensional inference is practical in an ML-family language. Syme’s work on F# [6] proved that the approach scales to production engineering. Gustafson’s posit arithmetic [7] showed that numeric representation can be matched to the value ranges that dimensional analysis reveals. Wadler’s contribution [1] provides the formal guarantee that the inferred types generate correct theorems. Together, these four lines of work inform a type system where dimensional annotations persist through compilation, guide representation selection, and, as this post argues, generate physical safety proofs that hold the computation graph to declared physical bounds.

This raises a useful question: how far can the proofs extend without annotation? Dimensional consistency tells us that a force in Newtons will not be accidentally added to a velocity in meters per second. That is valuable. But a practitioner working in aerospace, finance, or clinical medicine needs more than dimensional consistency. An aerospace engineer needs to know that the shear stress on a wing span never exceeds the yield strength of the material. A financial risk analyst needs to know that a portfolio’s value-at-risk stays within regulatory limits. A medical clinician needs to know that a drug dosage stays within the therapeutic window for a given patient weight. Can the compiler derive these constraints from the computation graph, and can it show proof that those constraints hold?

The answer is yes, within well-characterized boundaries, and the mechanism is one the DTS/DMM paper already describes as foundational to Fidelity Framework’s design.

Range Propagation as Proof Machinery

The DTS includes a representation selection function (detailed in Section 2.6 of the DTS/DMM paper). Given a value with a declared dimensional range, the compiler evaluates candidate numeric representations (IEEE 754, b-posit at various widths) against worst-case relative error within the range. The mechanism is interval arithmetic over the computation graph: declared ranges at the inputs propagate through arithmetic operations to produce derived ranges at the outputs.

This mechanism was designed to select posit widths. It also generates safety proofs.

Fidelity.Physics supplies the type that makes this possible. Alongside its measure type declarations, it provides a generic Range type:

type Range<[<Measure>] 'T, [<Measure>] 'U> =
 Range of lower: 'T<'U> * upper: 'T<'U>

Both type parameters are measures in the abelian group algebra. Range is what distinguishes a pair of propagation bounds from an ordinary pair of values. When the application writes Range(5000.0<kg>, 80000.0<kg>), the compiler knows to propagate that interval through every arithmetic operation in the computation graph.

The interval arithmetic that propagates ranges through multiplication, division, addition, and subtraction is deterministic, bounded-time, and closed under the same operations that dimensional types are closed under. When the compiler propagates a range through a computation graph and the output range exceeds a declared bound, the violation is a design-time finding. No annotation is required beyond the input ranges and the material property that defines the bound. The computation graph determines the rest.

Domain Case: Aerospace Structural Integrity

The following examples illustrate how range propagation produces domain-specific safety findings. Each uses the same underlying mechanism: interval arithmetic over the computation graph, compared against a declared bound. The domains differ; the compiler’s role is identical. Fidelity.Physics provides the dimensional algebra, the measure types that make quantities type-safe. The application code declares constants, material properties, operating ranges, and the computation graph. The compiler provides the proof.

Flight Envelope Analysis

Consider a simplified structural analysis. Fidelity.Physics provides the measure types, derived measures like Pa, and universal constants like gravitational_acc. The application declares its own material properties and operating ranges:

open Fidelity.Physics
// Fidelity.Physics provides:
// [<Measure>] type kg
// [<Measure>] type m
// [<Measure>] type s
// [<Measure>] type Pa = kg * m^-1 * s^-2
// let gravitational_acc = 9.81<m * s^-2>

let yield_strength = 2.7e8<Pa> // aluminum 7075-T6

let aircraft_mass = Range(5000.0<kg>, 80000.0<kg>)
let load_factor = Range(1.0, 9.0)
let wing_spar_area = Range(0.01<m^2>, 0.1<m^2>)

The computation graph encodes the physics:

let lift_force = aircraft_mass * gravitational_acc * load_factor
// inferred: float<N>, range [49050, 7063800]

let shear_stress = lift_force / wing_spar_area
// inferred: float<Pa>, range [490500, 706380000]

The compiler propagates the ranges through the arithmetic. Multiplication of intervals multiplies the endpoints (with appropriate handling of signs). Division divides with the corresponding endpoint pairing. At the output, the computed range of shear_stress is [490500, 706380000]. The declared yield strength is $2.7 \times 10^8$ Pa. The upper bound of the computed range ( $7.06 \times 10^8$ ) exceeds the yield strength.

The compiler reports this without any safety assertion from the programmer:

⚠ Range exceeded: shear_stress upper bound 7.06e8 <Pa>
exceeds yield_strength 2.70e8 <Pa>
Occurs when load_factor > 3.44
(derived from computation graph)
Trace:
lift_force = aircraft_mass * gravitational_acc * load_factor
shear_stress = lift_force / wing_spar_area
Dimensional consistency: verified ✓
Range analysis confidence: exact
(monotonic arithmetic, all inputs range-declared,
no control flow, no iteration)

The diagnostic tells the engineer not only that a violation exists but where in the operating envelope it occurs: load factors above 3.44 produce shear stresses that exceed the material limit. The engineer can then make an informed decision: restrict the operating envelope, increase the spar area, or select a stronger material. Each of these changes modifies the input ranges or the computation graph, and the compiler re-evaluates automatically.

Domain Case: Financial Risk Constraints

The same mechanism applies to financial computation, where the “physical” constants are regulatory limits and the “material properties” are risk thresholds. Fidelity.Physics.Finance provides the currency and time measure types. The application declares its regulatory constants and portfolio-specific operating ranges:

open Fidelity.Physics.Finance
// Fidelity.Physics.Finance provides:
// [<Measure>] type USD
// [<Measure>] type EUR
// [<Measure>] type days
// [<Measure>] type years
// let tradingDaysPerYear = 252.0<days * years^-1>

// Regulatory and statistical constants
let confidence_z = 2.326 // 99th percentile
let regulatory_limit = 5.0e7<USD> // Tier 1 capital

// Portfolio-specific operating ranges
let notional = Range(1e4<USD>, 1e9<USD>)
let leverage_ratio = Range(1.0, 30.0)
let volatility = Range(0.05, 0.80) // annualized
let holding_period = Range(1.0<days>, 10.0<days>)

The computation graph encodes the risk model:

let exposure = notional * leverage_ratio
// inferred: float<USD>, range [1e4, 3e10]

let var_estimate = exposure * volatility * confidence_z * sqrt(holding_period)
// inferred: float<USD>, range [1162, 1.76e10]

The compiler reports:

⚠ Range exceeded: var_estimate upper bound 1.76e10 <USD>
exceeds regulatory_limit 5.0e7 <USD>
Occurs when leverage_ratio > 1.27 at max volatility
(derived from computation graph)
Dimensional consistency: verified ✓
Range analysis confidence: exact

The dimensional types prevent a category of error that financial systems encounter routinely: confusing notional with exposure, mixing annualized volatility with daily volatility (different time dimensions), or computing VaR in one currency and comparing against a limit denominated in another. The range propagation then adds a second layer: even when the dimensions are correct, the computation may produce values that breach regulatory thresholds at certain points in the parameter space. The compiler identifies those points.

The computation graph is the risk model. The compiler verifies both dimensional consistency and regulatory compliance as design-time properties of the same graph traversal.

Domain Case: Clinical Dosage Safety

In clinical pharmacology, the “material property” is the therapeutic window: the range of drug concentrations that are effective without being toxic. Fidelity.Physics.Clinical provides the clinical measure types and derived concentration dimensions. The application declares pharmacokinetic constants, therapeutic bounds, and protocol-specific operating ranges:

open Fidelity.Physics.Clinical
// Fidelity.Physics.Clinical provides:
// [<Measure>] type mg
// [<Measure>] type hr
// [<Measure>] type L
// [<Measure>] type concentration = mg * L^-1
// [<Measure>] type doseRate = mg * kg^-1 * hr^-1

// Drug-specific pharmacokinetic properties
let clearance_rate = 0.15<hr^-1>
let volume_dist = 0.25<L * kg^-1>

// Therapeutic window (the safety constraint)
let min_effective = 10.0<mg * L^-1>
let max_safe = 40.0<mg * L^-1>

// Protocol-specific operating ranges
let patient_mass = Range(40.0<kg>, 150.0<kg>)
let dose_rate = Range(0.5<mg * kg^-1 * hr^-1>, 5.0<mg * kg^-1 * hr^-1>)
let infusion_duration = Range(0.5<hr>, 4.0<hr>)

The computation graph encodes the pharmacokinetic model:

let total_dose = dose_rate * patient_mass * infusion_duration
// inferred: float<mg>, range [10, 3000]

let peak_concentration = total_dose / (volume_dist * patient_mass)
// inferred: float<mg * L^-1>, range [0.4, 300]

The compiler reports two findings:

⚠ Range exceeded: peak_concentration upper bound 300 <mg * L^-1>
exceeds max_safe 40.0 <mg * L^-1>
Occurs when dose_rate > 1.0 at patient_mass = 40 kg,
infusion_duration = 4.0 hr
(derived from computation graph)
⚠ Below range: peak_concentration lower bound 0.4 <mg * L^-1>
below min_effective 10.0 <mg * L^-1>
Sub-therapeutic when dose_rate < 1.25 at patient_mass = 150 kg,
infusion_duration = 0.5 hr
(derived from computation graph)

The dimensional types catch a class of clinical error that dimensional analysis was originally designed to prevent: confusing mg/kg (dose per body mass) with mg/L (plasma concentration), or applying a dose rate in mg/kg/hr to a duration in minutes without converting. These errors have caused patient harm in practice. The range propagation adds the therapeutic window check: the computation is dimensionally correct, but the dosage at certain combinations of patient mass, rate, and duration falls outside the safe range. The compiler identifies the specific parameter combinations.

Note that the compiler does not know pharmacology. It knows dimensional types, declared ranges, and interval arithmetic. The pharmacological knowledge is encoded in the application’s constants, therapeutic bounds, and range declarations. Fidelity.Physics.Clinical contributes only the measure types that make those declarations dimensionally safe. The compiler’s contribution is propagating the consequences through the computation graph exhaustively and exactly.

The Common Pattern

An aerospace engineer, a risk analyst, and a clinical pharmacologist all receive the same class of compiler guarantee from the same toolchain. The three cases share a single underlying mechanism. The domains differ in their dimensional vocabularies. The physical constants, safety bounds, and operating ranges are all application concerns. The compiler’s role is identical in all three: propagate ranges through the computation graph, compare the result against declared bounds, report findings with traces and confidence levels.

Fidelity.Physics provides dimensional algebra, not domain knowledge. Each sub-namespace (Aerospace, Finance, Clinical) declares the measure types that make a domain’s quantities type-safe. The constants, material properties, regulatory limits, therapeutic windows, and operating ranges belong to the application. A financial risk model and a structural analysis both use the same compiler machinery. They differ in the measure types they import and in the constants and ranges they declare.

The Mechanism Is Not New; the Application Is

The engineer declares constants, bounds, and operating ranges using the measure types that Fidelity.Physics provides. The compiler propagates them, compares against the declared bounds, and produces the diagnostics as part of the same compilation pass. The interval tracking and the constraint check are steps in the build, not a separate verification stage the engineer runs by hand.

Interval arithmetic itself is not novel; it has been studied extensively since Moore’s foundational work in the 1960s. What is new is the integration with dimensional types and the computation graph. In conventional interval arithmetic, ranges are manually specified and manually propagated. In our Fidelity framework, the compilation graph provides the structure through which ranges propagate, and the DTS ensures that every propagation step is dimensionally consistent.

The same PSG node that carries a dimensional annotation (<Pa>) and a representation selection hint (posit32, bias at 1 MPa) also carries a propagated range ([490500, 706380000]). The three are computed by the same elaboration and saturation process. Dimensional consistency, representation adequacy, and physical range safety are three views of the same graph traversal.

Higher-Order Range Propagation

The range propagation extends through higher-order functions by the same mechanism that dimensional types extend through them.

A first-order function with declared input ranges produces output ranges deterministically:

let computeStress (load : float<N>) (area : float<m^2>) : float<Pa> =
 load / area

The range of the output is the range of the numerator divided by the range of the denominator. When this function is called from a higher-order context, the compiler inlines the range propagation:

let safetyEnvelope (massRange : Range<kg>) (gRange : Range<1>) =
 let liftRange = massRange * g * gRange
 computeStress liftRange sparArea

The function computeStress has a known computation graph (a single division). Its range behavior is deterministic: output range equals input range divided by input range. The call from safetyEnvelope passes a derived range (the product of mass, gravitational acceleration, and G-force ranges) into computeStress, and the range propagation continues through the function boundary without interruption.

Parametricity ensures this works. The function computeStress is parametric in the magnitude of its inputs; it divides whatever it receives. The range propagation commutes with the function application for the same reason that dimensional annotations commute with compilation passes: the function does not inspect or modify the metadata, only the computational structure.

For functions that compose multiple range-carrying operations, the propagation chains:

let fullAnalysis mass gForce sparArea yieldStrength =
 let lift = mass * g * gForce
 let stress = lift / sparArea
 let safetyMargin = yieldStrength - stress
 safetyMargin
 // compiler derives the range of safetyMargin

The safety margin’s range is computed from the chain of operations. If the lower bound is negative, the compiler reports the specific combination of input values that produces a negative margin. This is not a test; it is a proof over the declared operating envelope.

Where Range Propagation Is Exact

The compiler’s range analysis produces exact bounds (no false positives, no false negatives) when the computation satisfies three conditions:

Monotonic arithmetic. Multiplication of positive values, addition, subtraction, and division by positive values are all monotonic: the output range endpoints correspond to specific input range endpoints. The compiler evaluates the operation at the endpoint combinations and takes the extremes. This is exact.

No control flow. The computation is a straight-line sequence of arithmetic operations. There are no branches, no conditionals, no pattern matches that would split the range into cases. The range at every node is determined by a single arithmetic path from the inputs.

All inputs are range-declared. Every leaf value in the computation graph has a declared range or a known value (from explicit annotation, from a constant declaration, or from a material property). There are no unranged inputs whose range defaults to the full representable interval.

When all three conditions hold, the range analysis is a proof: the output is guaranteed to lie within the computed range for all inputs within the declared ranges. The compiler reports this as “range analysis confidence: exact.”

Stated in Hoare logic, the analysis discharges the triple:

\{\,\forall i.\; \mathit{input}_i \in \mathit{declared\_range}_i\,\} \quad \mathit{computation\_graph} \quad \{\,\mathit{output} \in \mathit{computed\_range} \;\wedge\; \mathit{computed\_range} \subseteq \mathit{safe\_bound}\,\}

The declared input ranges are the precondition. The straight-line arithmetic is the command. The propagated output range is the strongest postcondition derivable from those preconditions and the operations. The bound check is an application of the consequence rule: if the computed range implies the declared safety bound, the triple is valid, and Z3 discharges that implication as a QF_LIA obligation. Range propagation, in this reading, is forward strongest-postcondition computation over the PSG.

Structural analysis, thermal analysis, fluid dynamics boundary conditions, sensor fusion pipelines, and electrical power budgets are typically straight-line arithmetic over declared physical ranges. They satisfy all three conditions. For these computations, the proofs are free for the same reason dimensional consistency is free: the computation graph determines the result.

Where Range Propagation Is Conservative

The analysis becomes conservative (may report violations that cannot actually occur) in cases that fall into two distinct categories. The distinction matters: the first category is resolved by Tier 2 annotations the engineer writes locally; the second is resolved by Tier 3 lemmas that the compiler instantiates from a growing library. They are not the same kind of gap.

Resolved at Tier 2 by an annotation

Branching control flow with narrowing. When a computation branches on a runtime condition, the range at the join point is the union of the ranges from both branches. For branches that correspond to operating regimes (subsonic vs. supersonic, laminar vs. turbulent), the union is the correct range. But for branches that narrow the range (a clamp, a saturation, a guarded reciprocal), the conservative analysis may report a violation that the clamp itself prevents. The engineer resolves this by inserting a range assertion at the join point. In Hoare logic this is the conjunction rule: instead of carrying {Q₁ ∨ Q₂} forward as the postcondition of the branch, the engineer asserts {bound} and Z3 discharges bound ⊆ Q₁ ∧ bound ⊆ Q₂ as a QF_LIA obligation. The asserted bound then becomes the precondition for the rest of the graph, and propagation continues.

Bounded loops with linear invariants. A loop whose iteration count depends on runtime data produces a range that the compiler must compute by fixed-point iteration over the interval. For bounded loops with a linear invariant, the engineer states the invariant as an annotation and Z3 discharges the Hoare while rule: the invariant holds initially, is preserved by each iteration, and implies the postcondition on exit. This stays inside the QF_LIA fragment and remains a Tier 2 obligation.

External inputs. Values that enter from outside the compilation boundary (sensor readings, API responses, user input) carry only their declared range. The compiler cannot verify that the external source respects the declared range. The analysis is sound with respect to the declaration; it is not sound with respect to the actual source. This is a boundary condition rather than a tier issue. The declaration is the contract.

Resolved at Tier 3 by a parameterized lemma

Wide-interval transcendentals. The sine of a wide interval is [-1, 1]. The exponential of a wide interval is [0, ∞). The naive interval rule for these functions is too coarse for useful constraint checking. An annotation cannot resolve this case, because the engineer cannot honestly assert a tighter range without invoking a real-analysis fact. The resolution is a lemma in Fidelity.Lemmas.Mathematics, parameterized over the interval, proved once in Rocq, and instantiated automatically by the compiler from the specific interval values in the PSG. The Tier 2 discharge confirms that the concrete interval satisfies the lemma’s precondition; the lemma supplies the tighter postcondition. The compiler reports a conservative finding today because the relevant lemma is not yet in the library, not because the approach cannot express it.

Loops with nonlinear or transcendental invariants. A loop whose invariant requires reasoning beyond QF_LIA (convergence of an iterative solver, monotonic decrease of a Lyapunov function, a fixed-point bound on a nonlinear recurrence) is a Tier 3 obligation. The lemma is parameterized over the recurrence and the operating interval; the instantiation comes from the PSG. As with the transcendental case, the conservative diagnostic today is an honest acknowledgment that the relevant lemma is not yet available, not a structural limitation.

In each of these cases, the compiler reports the confidence level alongside the finding:

⚠ Potential range exceeded: thermal_stress upper bound 3.1e8 <Pa>
may exceed yield_strength 2.7e8 <Pa>
Range analysis confidence: conservative
(branch at line 47 widens interval; actual range may be narrower)
Consider: add range assertion or narrow input range declaration

The diagnostic distinguishes between exact findings (proofs) and conservative findings (warnings). For the first category above, the engineer tightens the finding into an exact one by adding a Tier 2 annotation. For the second, the resolution arrives when the relevant lemma lands in Fidelity.Lemmas.Mathematics; from that point forward, every program whose PSG annotations fall within the lemma’s parameter types is instantiable for free. The conservative region shrinks as the lemma library grows.

The categorical reading of this gap is precise about where the witness is missing. Each conservative finding corresponds to a non-trivial obstruction class in the first cohomology of the relevant verification sheaf: the local arithmetic cannot be extended to a globally consistent range assignment without an additional witness, and the witness is exactly what a Tier 3 lemma supplies. The space of conservative findings is therefore the space of obstruction classes for which no witness has yet been added to the library. Adding a lemma kills the corresponding obstruction class, and every program whose PSG falls within the lemma’s parameter types becomes exact at that boundary. The compilation sheaf design document treats the four-tier verification stack as a graduated sequence of sheaves over the same compilation poset, and the conservative-finding diagnostic is the operational face of an $H^1$ obstruction in that framework.

The Revised Tier Boundary

The Fidelity framework’s formal verification design defined Tier 1 as dimensional consistency, escape classification, allocation verification, and capability checking. Range consistency and physical safety constraint checking belong in Tier 1 as well, when the computation satisfies the exactness conditions above.

The revised Tier 1 coverage:

Property	Mechanism	Free?
Dimensional consistency	Abelian group algebra over type annotations	Yes
Escape classification	Coeffect propagation through PSG	Yes
Allocation verification	Escape classification mapped to target memory	Yes
Capability checking	Coeffect requirements against target profile	Yes
Representation selection	Interval arithmetic over dimensional ranges	Yes
Physical range safety	Same interval arithmetic, compared against physical bounds	Yes, when exact

The last two rows use the same mechanism. The difference is what the computed range is compared against: a representation’s dynamic range (for representation selection) or a physical property’s value (for safety checking). The comparison target determines whether the finding is “use posit32” or “the wing breaks at G-force 3.44.” The propagation is identical.

For the majority of safety-critical arithmetic, the engineer writes no proof code and no verification annotations. The compiler does the work at Tier 1. Tier 2 (scoped assertions) is needed only when the range analysis is conservative in a way a local annotation can resolve: a clamp at a join point, a linear loop invariant. Tier 3 is needed for properties that range propagation cannot express: convergence, termination of iterative procedures, transcendental bounds, and correctness of algorithms whose safety depends on control flow rather than arithmetic. Tier 3 lemmas in Fidelity.Lemmas.Mathematics are parameterized over interval bounds and instantiated by the compiler from the Tier 2 facts already present in the PSG. The marginal cost of a new lemma is the proof itself; instantiation is automatic. The same lemma-reuse mechanism applies to cryptographic protocols in the formal verification design, where one proof in the library discharges every program that falls within its parameter types.

The coeffect algebra that emerged from evaluating and departing from F*’s dependent type approach is precisely what makes range propagation composable: the same algebraic structure that tracks escape classification and memory lifetimes also tracks value ranges through the computation graph. The verification internals cover this design path in full.

Implications for Domain Libraries

Fidelity.Physics provides the dimensional vocabulary: measure types and the Range<'T, 'U> generic introduced earlier. Building a domain library is declaring which measure types exist and how they compose. The application then uses those types, along with Range, to declare its own constants, material properties, safety bounds, and operating ranges. The compiler enforces the consequences across every computation.

Three domain libraries illustrate the pattern:

module Fidelity.Physics.Aerospace =
 [<Measure>] type gForce // dimensionless load factor
 [<Measure>] type knot // nautical miles per hour

module Fidelity.Physics.Finance =
 [<Measure>] type USD
 [<Measure>] type EUR
 [<Measure>] type days
 [<Measure>] type years

module Fidelity.Physics.Clinical =
 [<Measure>] type mg
 [<Measure>] type hr
 [<Measure>] type L // volume

Each library declares its own dimensional vocabulary. The application declares everything else: constants, bounds, and ranges for the specific design or protocol under analysis. The compiler treats all domains identically: propagate the application’s ranges through the computation graph, compare against the application’s bounds, report the findings. The dimensional algebra is in the library; everything else is in the application; the verification is in the compiler.

An aerospace application would use these types to declare its own material properties:

let aluminum7075 = {
 YieldStrength = 2.7e8<Pa>
 UltimateStrength = 5.7e8<Pa>
 Density = 2810.0<kg * m^-3>
 ThermalExpansion = 23.6e-6<K^-1>
}

The compiler combines these application-level declarations with the computation graph to produce safety findings without any per-function annotation. The library provides the type algebra; the application provides the constants, bounds, ranges, and computation graph; the range propagation is the verification. All compose at design time, and the proofs, where they are exact, are free.

From Findings to Certificates

The design-time diagnostics shown in the examples above are derived from Z3 proof obligations that CCS generates as it saturates the PSG. When the range analysis for shear_stress produces an exact finding, that finding corresponds to a resolved QF_LIA assertion in the PSG node. The release path we are designing carries those resolved assertions forward: a clef build --release would aggregate them into a global SMT problem, verify it with Z3, and hash the resulting witness cryptographically alongside the compiled binary into a .proofcert artifact.

The certificate guarantees that every range finding reported at design time holds in the compiled output. The range exceeded at G-force 3.44, the VaR breach at leverage ratio 1.27, the therapeutic window boundary at dose rate 1.0 for a 40 kg patient: each is a Z3-verified constraint that survives MLIR lowering through translation validation. The verification internals document this pipeline from PSG saturation through the cryptographic release certificate.

The practical consequence is that the engineer who sees a range finding in Lattice during development can trust that the same constraint is enforced in the release binary. The proof ships with the artifact.

Better Design, Safer Results

This approach satisfies proofs for a large portion of the physical computations that safety-critical engineering depends on: structural loads, thermal gradients, pressure differentials, electrical power budgets, and sensor operating envelopes. Range propagation through the computation graph produces exact safety proofs for a well-defined class of computations: straight-line arithmetic over declared ranges with monotonic operations.

The class does not include computations whose safety depends on control flow, iterative convergence, or runtime-dependent data. For those, the compiler produces conservative warnings that may require Tier 2 annotation to resolve. The boundary between “free proof” and “requires annotation” is determined by the computation’s structure, not by the engineer’s diligence. The compiler knows which case applies and reports accordingly.

Representation selection and safety constraint checking are the same mechanism applied to different comparands. The infrastructure for safety checking was present in the DTS from the beginning; it was designed for representation selection and deployed for that purpose. Extending it to physical safety constraints requires no new machinery, only the recognition that a range bound compared against a material property is the same operation as a range bound compared against a numeric format’s dynamic range.

The four influences on this design each contribute a specific element. Kennedy’s Units of Measure showed that dimensional inference is practical in a production language. Syme’s F# proved it scales. Gustafson’s posit arithmetic connected dimensional ranges to numeric representation selection, creating the interval propagation machinery. Wadler’s parametricity result provides the formal guarantee that the propagated types generate correct theorems. Range propagation extends these theorems from dimensional consistency into the physical, financial, and clinical constraints that the types were designed to represent.

The proofs follow the computation graph as far as the arithmetic is monotonic, the inputs are declared, and the control flow is absent. For a significant and practically important class of computations across multiple domains, that is far enough. The domain library is the design-time decision that scopes the proofs to a specific field. We have found no other representative implementation in the standing literature we have reviewed that derives physical, financial, and clinical range proofs from the same dimensional propagation that selects numeric representations, and that is the direction we will keep extending as the lemma library grows and the conservative region shrinks.

References

[1] P. Wadler, “Theorems for free!” in Proceedings of the Fourth International Conference on Functional Programming Languages and Computer Architecture, pp. 347-359, ACM, 1989.

[2] H. Haynes, “Dimensional Type Systems and Deterministic Memory Management: Design-Time Semantic Preservation in Native Compilation,” arXiv:2603.16437, 2026.

[3] R. E. Moore, Interval Analysis, Prentice-Hall, 1966.

[4] H. Haynes, “Decidable By Construction: Design-Time Verification for Trustworthy AI,” arXiv:2603.25414, 2026.

[5] A. Kennedy, “Types for Units-of-Measure: Theory and Practice,” in Central European Functional Programming School, Springer LNCS 6299, 2009.

[6] D. Syme, A. Granicz, and A. Cisternino, Expert F# 4.0, Apress, 2015.

[7] J. L. Gustafson and I. T. Yonemoto, “Beating Floating Point at its Own Game: Posit Arithmetic,” Supercomputing Frontiers and Innovations, vol. 4, no. 2, 2017.

[8] J. C. Reynolds, “Types, abstraction and parametric polymorphism,” in Information Processing 83, pp. 513-523, North-Holland, 1983.

'Free' Proofs from Dimensional Types

Tue, 31 Mar 2026 00:00:00 +0000

In 1989, Philip Wadler published “Theorems for free!”, a paper that demonstrated a remarkable property of polymorphic type systems: the type of a function, by itself, determines non-trivial theorems about that function’s behavior. No implementation needs to be examined. No test cases need to be run. The type is the theorem.

This result, grounding John Reynolds’ earlier abstraction theorem (1983) in a form accessible to working programmers, has quietly underpinned the design of ML-family type systems for over three decades. It is a direct influence on our Dimensional Type System, and the lineage is worth tracing.

The Parametricity Result

Wadler’s central observation is best illustrated by his own method. Write down the type of a polymorphic function. Do not look at the function’s definition. From the type alone, derive a theorem that every function of that type must satisfy.

Consider a function with type $\forall a.\ [a] \to [a]$ . This function takes a list of any type and returns a list of the same type. Parametricity guarantees that for any such function $g$ and any total function $f$ :

\operatorname{map}\ f \circ g = g \circ \operatorname{map}\ f

The function $g$ cannot inspect the elements of the list (they are abstract; $g$ does not know what $a$ is). It can only rearrange, duplicate, or drop elements. Whatever rearrangement it performs must commute with any element-wise transformation. This is not a property of any specific function; it is a property of the type. Every function with this type satisfies the theorem.

Wadler called these “free theorems” because they cost nothing: no annotation, no proof effort, no verification step. They fall out of the type.

The Connection to Dimensional Types

The DTS extends Hindley-Milner unification with dimensional annotations drawn from finitely generated abelian groups. A function with type float<'d> -> float<'d> -> float<'d * 'd> (multiply two dimensioned values) carries a dimension variable 'd that is polymorphic in exactly the sense Wadler describes: the function cannot inspect the dimension. It must behave uniformly across all dimensional instantiations.

Parametricity guarantees this uniformity. A multiplication function that works correctly for meters must work correctly for kilograms, for seconds, for any dimension, because the dimension variable is abstract. The function has no mechanism to dispatch on the dimension and do something different. The type prevents it.

This is the formal reason that dimensional type inference is sound. When our DTS infers that a computation is dimensionally consistent, it is deriving a free theorem from the computation’s polymorphic type. The inference is decidable (polynomial time, complete, principal) because the dimensional constraints form a system of linear equations over the integers, solved by Gaussian elimination. But the correctness of that inference, the reason the inferred types actually guarantee dimensional consistency of the compiled artifact, rests on parametricity.

Persistence Through Lowering

The DTS paper’s central claim is that dimensional annotations persist through multi-stage MLIR lowering. Each lowering pass transforms the program’s structure (from high-level operations to target-specific instructions) while preserving the dimensional annotations as MLIR attributes. Parametricity provides the formal justification for this claim.

Each lowering pass is a structure-preserving transformation, a function from one program representation to another. The dimensional annotations are polymorphic metadata that the pass carries through. Wadler’s map-commutation theorem applies directly: if the lowering pass is parametric in the dimension (it does not inspect or modify dimensional annotations, only the computational structure), then lowering and reading the dimension gives the same result as reading the dimension and lowering.

In more concrete terms: it does not matter whether the compiler checks dimensional consistency before or after lowering to the LLVM dialect. The result is the same, because the lowering is parametric in the dimension. This is not an implementation property of specific MLIR passes; it is a consequence of the type structure that Wadler formalized.

The DTS paper’s information accrual principle (Section 6.6) states that each compilation stage has strictly more information than its predecessor. Parametricity is the mechanism that ensures this accrual is monotonic: dimensional information established at an early stage cannot be contradicted at a later stage, because the later stage’s transformations are parametric in the dimensions.

Free Theorem Cascade in the Fidelity Framework

Several properties that the DTS paper establishes as design-time verification results are, in Wadler’s terminology, free theorems. They cost nothing beyond the type declarations that the programmer provides (or that the inference engine derives):

Dimensional consistency of the chain rule. If $f$ maps values with dimension $d_1$ to values with dimension $d_2$ , then the derivative $df/dx$ carries dimension $d_2 \cdot d_1^{-1}$ . This is a free theorem: the chain rule’s dimensional behavior follows from the polymorphic type of differentiation. The DTS verifies it without examining the computation’s structure, because the type determines it.

Cross-target transfer fidelity. When a value crosses a hardware boundary (FPGA to CPU, NPU to GPU), the dimensional annotation determines whether the precision conversion is acceptable. The representation selection function operates on the dimensional range, which is a type-level property. The transfer fidelity analysis is a free theorem of the value’s dimensional type and the target’s representation profile.

Coeffect propagation. The escape classification system (StackScoped, ClosureCapture, ReturnEscape, ByRefEscape) is a coeffect discipline in the sense of Petricek et al. The propagation of coeffects through the compilation graph follows the same parametricity structure: a transformation that is parametric in the coeffect annotation cannot change the escape classification.

Grade preservation in geometric algebra. The PHG paper (arXiv:2603.17627) introduces grade as a dimension axis within the DTS abelian group framework. Grade preservation through training, the theorem that forward-mode autodiff with quire-exact accumulation preserves the structural zeros of the Cayley table, is a free theorem of the grade-annotated type. The grade variable is polymorphic; operations that are parametric in grade cannot introduce grade corruption.

The Connection to Reynolds

Wadler’s paper is explicitly an accessible reformulation of Reynolds’ abstraction theorem. Reynolds proved in 1983 that types can be read as relations: a type denotes not just a set of values but a relation between different interpretations of the type. Polymorphic functions must preserve these relations. Wadler showed that this relational reading generates useful theorems for specific types.

Reynolds also independently discovered continuations (as documented in his 1993 survey “The Discoveries of Continuations”), which provide the formal basis for Clef’s DCont mechanism. The two contributions, abstraction (parametricity) and continuations (DCont), are the two formal pillars of the porous loop’s typed interface:

DCont provides the suspension and resumption mechanism. In the design we are building toward, the recurrent model suspends mid-computation, passes its state as a delimited continuation to a domain actor, and resumes with the response.

Parametricity provides the guarantee that the suspension and resumption are dimensionally consistent. The continuation’s type is polymorphic in the dimension; the domain actor’s response must satisfy the same dimensional constraints regardless of which specific dimension is instantiated. This is a free theorem of the continuation’s type.

The convergence of these two contributions in one researcher’s body of work is not coincidental. Both are consequences of taking types seriously as specifications of program behavior: types determine what continuations can capture (Reynolds 1972), and types determine what theorems functions satisfy (Reynolds 1983, Wadler 1989).

Implications for DTS

The DTS paper’s Section 2.2 (dimensional inference) derives its soundness from parametricity. The claim that dimensional annotations survive lowering (the persistence property) is a consequence of parametric polymorphism applied to compilation passes. The decidability result (polynomial time, complete, principal) establishes that the inference algorithm terminates; parametricity establishes that the inferred types mean what they claim to mean.

In short, the full summary of Clef’s innovation stems from:

Reynolds’ abstraction theorem: types are relations; polymorphic functions preserve relations.
Wadler’s free theorems: specific types generate specific theorems about all functions of that type.
DTS inference: dimensional types generate dimensional consistency theorems about all functions whose types the DTS infers.
Persistence: compilation passes that are parametric in dimensional annotations preserve the inferred dimensional consistency, by the same reasoning that Wadler’s map-commutation theorem follows from parametricity.

The Free Theorem Boundary

The properties enumerated above (dimensional consistency, grade preservation, coeffect propagation, transfer fidelity) all share a structural feature that makes them genuinely free: each is a statement about an abelian-group-valued annotation whose preservation under polymorphic operations is forced by parametricity. The dimensional algebra is a free abelian group on the base units. Grade is an integer index in that same abelian setting. Escape classifications form a finite lattice that the compiler propagates without engineer intervention. In every case, the theorem follows from the type structure alone, and the engineer’s annotation cost is zero.

This boundary is sharp, and it is worth being explicit about where it falls. Properties that involve inequalities over the integers (a buffer index lies within bounds, a temperature stays below a threshold, a sampled coefficient stays inside a norm bound) are not free theorems. They are not derivable from type structure by parametricity alone, because parametricity is silent about which specific values the variables take. Such properties live one tier up in the Fidelity framework’s verification stack: the engineer declares a range, the compiler propagates it through the computation, and Z3 discharges the resulting QF_LIA obligations. The cost is modest, and the obligations are decidable, but the discharge requires establishing a precondition rather than deriving one from a type. Properties that involve probability distributions (rejection-sampling termination, support equality of uniform distributions over lattice cosets) and properties that involve pairs of program runs (the probabilistic relational reasoning at the heart of cryptographic indistinguishability proofs) live further still from “free,” because the obligations are no longer about a single value at a single point but about a distribution or a relation between two computations.

The framework treats the four logical fragments ( $\mathbb{Z}^n$ equality, QF_LIA, the restricted probabilistic fragment, probabilistic relational Hoare logic) as distinct sheaves over a shared compilation poset, where the stalk category is what changes between tiers and the dual-pass discharge mechanism is what remains constant. The compilation sheaf design document makes that structure precise, and the triangle without mystery post sketches why the same categorical scaffolding shows up in Tarau’s combinatorial isomorphisms and in the recent cellular-sheaf literature on compositional information flow. The point worth carrying out of this post is the fence: parametricity does the work in the abelian fragment, and only in the abelian fragment.

Lower Bounds Framing

A lower bound is rarely a property of a problem alone. It is a property of a (problem, technique) pair: a statement that the known techniques for solving the problem have not done better than the bound, established by a proof that exploits some structural assumption about how those techniques work. When the literature reports “problem P has lower bound B,” the load-bearing content is “every technique we have tried so far inherits a reasoning step that forces B.” The bound looks structural because the reasoning step is shared across all the techniques. It stops looking structural the moment someone finds a technique that does not depend on that step.

Engineering value follows directly. If a lower bound is a property of (problem, technique) pairs, then progress comes from finding techniques whose structural assumptions do not bind on the specific instance class you care about. The distinction between “the problem is intractable” and “the problem is intractable under reasoning step S” is the difference between a closed door and a door that has not yet been examined for an appropriate handle.

Two results from different fields make the distinction concrete. In type theory, the standard argument against fully automated verification runs: program correctness needs dependent types, type inference and proof search in dependent type systems are not fully automatable (the engineer must supply proof terms by hand), therefore complete annotation-free verification is impossible. The undecidability of inference and proof search in general dependent type systems is real. The reasoning step that fails is the implicit assumption that every useful program property requires the full expressive range of dependent types. Dimensional consistency does not. It lives in a free-abelian-group fragment whose obligations are systems of linear equations over the integers, and within that fragment inference is decidable, complete, and principal via Gaussian elimination. The engineer supplies no proof terms; the bound on proof search has nothing to bind on. The DTS escape route operates by observing that the dependent-type undecidability result’s load-bearing assumption (every property needs the full machinery) does not hold for the property the framework actually verifies.

In algorithms, a 2025 paper from Tsinghua, Stanford, and Max Planck (arXiv:2504.17033) makes the same kind of move against the long-standing $O(m + n \log n)$ bound for directed single-source shortest paths. The bound had stood since the Fibonacci heap result of 1987 and was widely believed to be tight, on the reasoning that any shortest-path algorithm establishes a distance order over the vertices and therefore inherits the $\Omega(n \log n)$ lower bound that comparison sorting carries. The 2025 algorithm (BMSSP) defeats the bound by organizing the work as recursive divide-and-conquer over bounded vertex sets, compressing the search frontier in a way the comparison-sorting reduction cannot account for. The new bound is $O(m \log^{2/3} n)$ . The failure point in the older argument was the implicit transfer step: the comparison-sorting lower bound is a property of techniques that establish a total order through pairwise comparisons, and BMSSP does not work that way. The comparison-sorting bound holds for the techniques it was proved for, and it stops being a constraint the moment a technique appears that does not inherit the reasoning step it depends on.

The discipline that the (problem, technique) framing imposes is symmetric. Treating a lower bound as universal closes off research programs that turn out to be tractable in the structured subclass where the engineer actually works. Treating the structured subclass as broader than it is ships systems whose guarantees do not survive contact with workloads that fall outside the subclass. The honest position requires three things at once: identify the (problem, technique) pair the lower bound was actually proved for, identify the structural assumption in the original technique that fails on an instance class, and demonstrate a technique that exploits the failure constructively.

The Deeper Pattern

Wadler’s paper demonstrates a principle that recurs throughout our framework’s design: structure that is present in the type system generates properties of the compiled artifact for free. Dimensional consistency, escape classification, grade preservation, coeffect propagation, and cross-target transfer fidelity are all instances of this principle. None requires runtime enforcement. None requires separate verification tooling. Each falls out of the type structure through parametricity.

This is the formal content of the claim that verification is a compilation byproduct: the types determine the theorems, the compiler infers the types, and the theorems follow. The cost is the type system’s design. Once that design is in place, the theorems are free in Wadler’s original sense, and that is the property we will keep building on as the verification stack fills out the tiers above the abelian fragment.

References

[1] P. Wadler, “Theorems for free!” in Proceedings of the Fourth International Conference on Functional Programming Languages and Computer Architecture, pp. 347-359, ACM, 1989.

[2] J. C. Reynolds, “Types, abstraction and parametric polymorphism,” in Information Processing 83, pp. 513-523, North-Holland, 1983.

[3] J. C. Reynolds, “The discoveries of continuations,” Lisp and Symbolic Computation, vol. 6, pp. 233-248, 1993.

[4] H. Haynes, “Dimensional Type Systems and Deterministic Memory Management: Design-Time Semantic Preservation in Native Compilation,” arXiv:2603.16437, 2026.

[5] H. Haynes, “The Program Hypergraph: Multi-Way Relational Structure for Geometric Algebra, Spatial Compute, and Physics-Aware Compilation,” arXiv:2603.17627, 2026.

[6] T. Petricek, D. Orchard, and A. Mycroft, “Coeffects: A calculus of context-dependent computation,” in Proceedings of the 19th ACM SIGPLAN International Conference on Functional Programming, pp. 123-135, 2014.

FPGA and Hardware Inference

Fri, 20 Feb 2026 00:00:00 +0000

There’s a question that comes up early in any FPGA design course: is this a Moore machine or a Mealy machine? Students learn the distinction, draw the state diagrams, and internalize the rule. Moore machines have outputs that depend only on state. Mealy machines have outputs that depend on both state and inputs. It’s clean, it’s testable, and it’s one of those classifications that feels settled.

What doesn’t get enough attention is that this classification is not really a design choice. It’s a property of your circuit design. If your outputs reference inputs, you wrote a Mealy machine. If they don’t, you wrote a Moore machine. The distinction is already in the dependency graph before you ever annotate anything.

That observation is the starting point for what we’re building into the Clef language and Fidelity Framework.

flowchart LR
Infer["Compiler infers from code<br>Dependency analysis on step body"]
Match{"Match?"}
Dev["Developer declares intent<br>[‹Moore›] / [‹Mealy›] / fidproj default"]
Pass["✓ Proceed to codegen"]
Diag["⚠ Diagnostic:<br>Declared model ≠ inferred model<br>Change annotation or refactor outputs"]
Infer --> Match
Dev --> Match
Match -->|"Declared = Inferred"| Pass
Match -->|"Declared ≠ Inferred"| Diag

The Three Ms

In FPGA design, sequential circuits fall into three broad categories.

Moore machines produce outputs that depend solely on the current state. Because the output logic has no combinational path from the inputs, Moore outputs can be registered directly off the state flip-flops. This makes timing closure straightforward and eliminates glitching on outputs. The trade-off is latency: a Moore machine’s outputs reflect the state after the clock edge, so input changes don’t appear in the output until the next cycle.

Mealy machines produce outputs that depend on both state and inputs. The combinational path from inputs to outputs means the circuit can respond within the same clock cycle. This is faster by one cycle compared to Moore, but the combinational output path can create timing hazards, especially in high-speed designs where the critical path through input-to-output logic becomes the bottleneck.

Mixed machines are what most real designs actually are. Some outputs depend on inputs (Mealy paths), others depend only on state (Moore paths). A counter display that also has an input-responsive status LED is a mixed machine. Treating the entire design as one or the other means either over-registering outputs that need to be fast, or leaving timing-sensitive outputs exposed to glitches.

The conventional approach in Verilog, VHDL, and even higher-level tools like Chisel or Clash is to leave this classification to the engineer. You decide what your machine is, you code accordingly, and if you get it wrong, timing analysis tells you after synthesis. There’s no structural validation at the language level.

Why This Matters for Clef

The Fidelity Framework is built on the principle that types carry truth through the compilation pipeline. Dimensional types flow unchanged from application code through to hardware synthesis. Lifetime inference provides memory safety without annotation burden. In each case, the compiler observes a property of the code and carries it forward rather than asking the developer to declare it separately.

Machine classification in FPGA circuit design fits this pattern exactly.

The step function’s dependency graph already contains the answer. The compiler just needs to read it.

We have an early HelloArty design, a blinky LED project for the Digilent Arty A7-100T board that we’ve been compiling through the Clef toolchain:

[<HardwareModule>]
let helloArtyTop = {
 InitialState = { Counter = 0L; PeriodMs = 500L }
 Step = Behavior.step
 Clock = Endpoints.clock
}

The step function takes the current BlinkState and Inputs<ArtyInputs> (switch positions, button states) and returns a tuple of next state and Outputs<ArtyReport> (LED values, UART report). Some of those outputs, the LED colors, depend on which switches are active. That’s a Mealy path. The UART report depends only on state. That’s a Moore path.

HelloArty is a mixed machine. Not because someone declared it that way, but because that’s what the code says.

Declared or Inferred, with Validation

This follows the same pattern we established with DTS (Dimensional Type Safety) and DMM (Deterministic Memory Management). The developer can declare their intent:

[<HardwareModule>]
[<MachineModel(Moore)>]
let myDesign = { ... }

Or they can leave it undecorated and let the compiler infer the classification from the step function’s dependency structure. Either way, the compiler performs the analysis. When a declaration is present, the analysis validates it. When there’s no declaration, the analysis drives code generation directly.

The interesting case is when declaration and analysis disagree. If you annotate a design as Moore but your step function’s outputs depend on inputs, the compiler produces a diagnostic: your declared intent doesn’t match your code. Either refactor the outputs to depend only on state, or change the annotation. The mismatch becomes a compile-time failure, not a post-synthesis discovery.

The fidproj can also set a project-wide default:

[compilation]
target = "fpga"
machine_model = "inferred"

Setting machine_model = "moore" as a project default means every [<HardwareModule>] in the project is validated against Moore constraints unless individually overridden.

For safety-critical designs where registered outputs are a requirement, this provides a project-level guarantee.

What the Compiler Sees

The analysis itself is a natural fit for our nanopass architecture. A MachineDependencyAnalysis pass walks the step function body in the PSG (Program Semantic Graph) and traces, for each output field, whether any path in the dependency graph reaches an input parameter.

The mechanics:

Identify the output fields from the step function’s return type
For each output field, walk backwards through the PSG
If any transitive dependency reaches an input parameter: Mealy
If all dependencies resolve to state or constants: Moore
Attach the per-output classification as a coeffect on the Design node

This runs as a Phase 5+ nanopass, after Baker’s type resolution but before Alex’s MLIR generation. By the time the witness walks the Design node, the classification is pre-computed and available as a coeffect. The witness observes it. It does not compute it. This is the codata principle at work: the analysis is mise-en-place, prepared before the walk begins.

What Changes in the Generated Hardware

The classification drives concrete differences in the generated CIRCT output.

For a Moore output, the top-level module registers the output through seq.compreg:

// Moore: output registered for timing safety
%output_reg = seq.compreg %computed_output, %clk : !hw.struct<...>
hw.output %output_reg : !hw.struct<...>

For a Mealy output, the combinational result wires directly to the output port:

// Mealy: combinational output for same-cycle response
hw.output %computed_output : !hw.struct<...>

For a Mixed design, each output gets the treatment its dependency structure calls for. In HelloArty, the LED outputs (which depend on switch inputs) would remain combinational for same-cycle response to switch changes, while the UART report (which depends only on state) gets registered automatically for clean timing.

This is where inference provides a real advantage over manual classification. A human engineer looking at a complex step function might not immediately see which outputs have Mealy paths and which don’t. The compiler traces every dependency and makes the per-output decision systematically.

The Design-Time Experience

ClefAutoComplete participates in this analysis. When editing a [<HardwareModule>] design, the editor can surface the inferred classification:

ℹ Design 'helloArtyTop': Inferred Mixed
Mealy paths:
Outputs.Leds depends on Inputs.Sw0..Sw3 (color selection)
Outputs.Leds depends on Inputs.Btn0..Btn3 (cadence control)
Moore paths:
Outputs.UartReport depends on state only
Suggestion: Consider [<MachineModel(Mixed)>] to document intent

This feedback loop is where Clef’s approach diverges most from traditional FPGA workflows. You don’t wait for synthesis to discover timing issues. You don’t manually audit your step function for input-to-output paths. The analysis runs continuously in the editor and tells you what you built.

Width Inference and Timing Budget

Machine classification answers one question about your design: what kind of state machine did you build? There’s a second question that matters just as much for FPGA: how wide does each wire need to be?

On a CPU, this question has a trivial answer. int means “whatever the machine register holds,” typically 64 bits on modern architectures. The width is a platform property, fixed before your program runs. On FPGA, there are no machine registers. Every wire, every flip-flop, every arithmetic unit is exactly as wide as the design requires. Width is a design property, not a platform property.

Width as a Property of Code

Consider the Counter field in HelloArty’s BlinkState. It counts clock ticks and resets via modulus against maxCounterTicks, which equals maxPeriodMs * ticksPerMs * 2, approximately 400 million. The counter never exceeds that value. Its range is [0, 399,999,999], which fits in 29 bits.

The PeriodMs field is clamped between 100 and 2000 by the clampPeriod function. Its range is [100, 2000], which fits in 11 bits.

The Color discriminated union has 8 cases. Its tag value ranges from 0 to 7: 3 bits. Mode has 2 cases: 1 bit.

None of these widths are declared anywhere. They are consequences of the program’s structure: the constants, the modulus operations, the clamp bounds, the number of DU cases. The information is already in the code.

Interval Analysis

The same PSG (Program Semantic Graph) that carries dependency information for machine classification also carries the constants and operations that bound value ranges. An IntervalAnalysis nanopass walks the enriched PSG after type resolution and propagates intervals through the graph:

Constants have exact intervals: 100_000 is [100000, 100000]
Modulus x % K bounds the result to [0, K-1]
Clamp via min(x, upper) or max(x, lower) tightens the interval from one side
Addition and subtraction propagate through operand intervals
Multiplication takes the product of operand ranges
DU tags have [0, numCases - 1]
Boolean values are [0, 1]: 1 bit

The minimum bit width falls out directly: for an unsigned value with maximum M, you need ceil(log2(M + 1)) bits. For signed values, one additional bit for the sign.

Applied to HelloArty:

Value	CPU width	Inferred width	Reduction
Counter	64 bits	29 bits	55%
PeriodMs	64 bits	11 bits	83%
ticksPerMs	64 bits	17 bits	73%
Color tag	8 bits	3 bits	63%
Mode tag	8 bits	1 bit	88%

Every one of those narrower widths means fewer flip-flops, fewer LUT stages, shorter carry chains, and smaller multipliers. The 17-by-64-bit multiplier that dominated the critical path becomes 17-by-29. The seven 64-bit adders become 29-bit adders. The CARRY4 count drops proportionally.

Timing Budget

The width reduction isn’t just about area. It directly affects whether the design meets timing.

Carry chain depth on Xilinx 7-series is roughly proportional to bit width; each CARRY4 primitive handles 4 bits. A 64-bit adder needs 16 CARRY4 stages. A 29-bit adder needs 8. The combinational delay drops by half.

Traditionally, you discover this after synthesis. The STA (Static Timing Analysis) output from Vivado, generated after place-and-route, tells you whether your design met its clock constraint. If you have negative slack, you go back to your HDL, restructure your logic, re-synthesize, and check again. The feedback is accurate but late. A full synthesis-and-route cycle on even a modest design can take minutes; on a larger one, hours. The edit-compile-check loop for timing is one of the slowest feedback cycles in hardware development.

The Clef Compiler Service provides an early warning through a two-layer timing architecture:

Layer 1 is a structural depth heuristic that runs during compilation. A DepthAnalysis pass walks the PSG using the same semantic-edge-following traversal that drives code generation, counting weighted combinational operation depth between register boundaries. Multiplies and divides carry weight 2 (DSP slice or multi-LUT chain), adds and compares carry weight 1.

The threshold is not hardcoded. It is calibrated from two values that flow through the platform binding:

threshold = floor(clock_period_ns / ns_per_weight_unit)

The platform binding declares ns_per_weight_unit, a fabric-specific constant representing the average delay per weighted depth unit, calibrated from Vivado post-route timing. The Arty A7-100T binding declares ns_per_weight_unit = 1.6, derived from the HelloArty ground truth: weighted depth 8 produced a total path delay of 12.635 ns, giving approximately 1.58 ns per unit (rounded conservatively to 1.6).

The project declares clock_mhz in its fidproj, which can override the binding’s default. At the binding’s default of 100 MHz, the threshold computes to floor(10 / 1.6) = 6. HelloArty declares clock_mhz = 25 because its LED chaser design has no timing-critical paths. A 4-second breathing cycle does not need 100 MHz evaluation. At 25 MHz, the threshold becomes floor(40 / 1.6) = 25, well above the smoothstep’s depth of 12.

When depth exceeds the threshold, the compiler emits a CCS0100 warning with a two-sided diagnostic that tells the developer both knobs they can turn:

Behavior.clef:100: warning CCS0100: Combinational depth 12 exceeds threshold 6 (100 MHz)
Chain: op_Multiply → op_Multiply → op_Multiply → op_Division → op_Subtraction → op_Division → op_Addition
Hint: either reduce depth to ≤ 6, or relax clock to ≤ 52 MHz (currently 100 MHz).
To reduce: break the arithmetic/DSP chain with register stages

The diagnostic does not prescribe a solution. It presents the tradeoff: pipeline the logic to fit the clock, or relax the clock to fit the logic. The developer picks which knob to turn based on what they know about their design.

Layer 2 is Vivado’s post-route WNS (Worst Negative Slack). This is the ground truth. HelloArty’s smoothstep chain produces WNS = −2.635 ns at 100 MHz on Artix-7: the design needs 12.635 ns but only has 10 ns. Layer 1’s structural heuristic flagged this before synthesis ever ran.

The ns_per_weight_unit constant is calibrated against Layer 2 ground truth. HelloArty provides the first calibration data point. As more designs are compiled across different device families, the per-fabric constant gets refined: the feedback loop from real Vivado runs is meant to back-annotate the compiler model over time. Different fabrics (Kintex, Zynq, ECP5) will carry their own calibration constants in their platform bindings.

With --warnaserror, the CCS0100 warning promotes to an error, stopping compilation before Verilog is generated. This gives the developer the choice: fix the combinational depth (by pipelining or restructuring), relax the clock, or proceed to synthesis knowing the design will likely violate timing.

This is hardware autocomplete. The compiler understands the structural implications of your design and reports them as diagnostics, the same way it reports type errors.

The DTS Connection

This is the FPGA realization of the Dimensional Type System’s promise. Source code expresses intent: a counter, a period, a color. Operations express bounds: modulus, clamp, DU case count. The compiler derives precision from the intersection of intent and bounds. The platform binding provides physical constraints: clock frequency, device characteristics. The intersection of precision and constraints determines feasibility, checked at design time.

On CPU, DTS carries units of measure and dimensional correctness through arithmetic. On FPGA, it carries bit widths and timing feasibility through synthesis. Write your counter, write your modulus bound, and the compiler gives you 29 bits automatically. You don’t count bits. You don’t guess. You don’t find out after synthesis that your timing budget was blown by widths you never chose.

The Constructive Constraint

There’s a pattern in language design sometimes called the “pit of success.” The idea is that the easiest path through the language should lead to correct results, and deviation from correctness should require deliberate effort. Machine model inference creates exactly this kind of constructive constraint for FPGA design.

Consider a project with machine_model = "moore" set in the fidproj. If a developer writes a step function where an output depends on an input, the diagnostic appears immediately in the editor, before anything is compiled to hardware.

The constraint is constructive because it doesn’t prevent you from building what you need. If a particular module genuinely requires same-cycle input-to-output response, you annotate it [<MachineModel(Mealy)>] and the project default steps aside for that module. The override is explicit and local. Every other module in the project still gets the Moore guarantee.

Moore classification is not just a timing property; it is a physical property of the generated circuit. A Moore output has no combinational path from any input, which means it changes only on clock edges, without data-dependent timing variation. In cryptographic hardware, that variation is exactly what leaks key material through power analysis or timing side channels. The standard mitigation is manually ensuring constant-time execution paths, which is error-prone and difficult to verify. A compiler-enforced Moore constraint eliminates the category entirely: no combinational path, no data-dependent timing to exploit.

machine_model = "moore" is not just a design preference. It’s a security property expressed as a project configuration.

That proof compiles into physical hardware where the property holds by construction, not by convention. You don’t audit the netlist for timing leaks. You don’t add pipeline stages to break combinational paths. The language made the secure path the easy path, and deviation requires explicit annotation that documents where and why the constraint was relaxed.

Where This Sits in the Broader Picture

For FPGA targets, this principle plays out across the full pipeline:

Platform resolution determines the target board and its physical capabilities and constraints
Type mapping resolves Clef types to CIRCT hw.struct types with correct bit widths
Width inference determines minimum bit widths from value range analysis
Timing estimation validates that inferred widths meet the clock period budget
Machine classification determines which outputs need registration
The state machine builder generates the top-level hw.module with the correct output strategy
Pin assignment maps logical ports to physical package pins via XDC constraints
Synthesis takes the generated SystemVerilog through to a bitstream

Machine classification and width inference are two instances of the same principle: the compiler reads your code, derives a physical property from its structure, and acts on it before you reach the synthesis tool. One reads dependency graphs. The other reads value ranges. Both produce coeffects that downstream passes observe without re-deriving.

The FPGA is not just a standalone target. In the Fidelity model, it’s one processor in a heterogeneous system where Clef source compiles to native binaries on CPU and hw.module definitions on FPGA. The same type system that carries dimensional units through CPU arithmetic carries bit widths through hardware synthesis. The same dependency analysis that classifies Moore and Mealy outputs could classify which parts of a computation benefit from spatial execution versus temporal execution on a conventional processor.

We’ve written previously about posit arithmetic and the case for bringing Gustafson’s tapered-precision number system into the Fidelity toolchain. Posit’s strength is exactly the kind of sustained-accuracy computation where IEEE 754’s uniform precision wastes bits at extreme ranges. The quire accumulator, which provides exact dot products regardless of intermediate magnitudes, is a natural fit for FPGA implementation where the wide accumulator can be mapped directly to fabric without the overhead of a general-purpose ALU. Width inference is directly relevant here: the quire’s accumulator width is determined by the posit format parameters, and the compiler should derive it from the type, not from a hardcoded constant.

The planned demonstration for this is a numerical simulation where accuracy-dependent calculations are compiled to FPGA and connected to a CPU host over UART. The CPU orchestrates. The FPGA computes. Machine classification determines which parts of the FPGA design need registered outputs for clean data transfer and which can respond combinationally to streaming input. Width inference ensures every arithmetic unit uses exactly the precision the computation requires.

This is where the “same compiler, multiple targets” principle takes shape as a pipeline property. As we conceive it, the compiler reads what you wrote, classifies what you built, infers how wide your values need to be, checks whether your design fits in the timing budget, and targets the hardware that fits.

Machine classification and width inference are the first FPGA-facing results we have from our Dimensional Type System, and they are the ones we will keep building on as the rest of the heterogeneous pipeline comes into place.

Doubling Down on DMM and DTS

Fri, 23 Jan 2026 00:00:00 +0000

Two capabilities define the architectural foundation of the Fidelity framework:

Deterministic Memory Management (DMM)
Dimensional Type System (DTS)

While each addresses distinct engineering challenges, they rest on a single insight: memory semantics and physical semantics are both dimensions. They survive compilation the same way, constrain code generation the same way, and enable verification the same way. We have found no other representative implementation of this cohesion in the standing literature we have reviewed.

This document explores why these systems belong together, how they enable hardware-aware compilation from a single source, and what this means for developers building systems that interface with physical reality.

The Era of Managed Assumptions

To understand why DMM and DTS matter now, we must understand the constraints that shaped the previous generation of language design.

When Java, C#, Python, and F# emerged, the computing landscape looked very different. RAM was measured in single-digit megabytes. “Hardware” meant x86, perhaps SPARC. The operating system itself competed with applications for scarce resources. A single CPU with a single floating-point unit defined the execution model. In that context, language designers made reasonable engineering tradeoffs: erase type information early to reduce memory pressure, let garbage collectors handle memory so developers could focus on business logic, and treat the runtime as a trusted intermediary between programmer intent and machine execution.

Python’s lineage illustrates this particularly well. ABC was a teaching language at CWI Amsterdam in the 1980s. Python emerged in 1991 as Guido van Rossum’s scripting tool. NumPy arrived in 2005 to give researchers fast arrays. PyTorch and TensorFlow followed a decade later. At every step, the assumption was consistent: a knowledgeable human supervises the computation. The professor knows that column three contains velocity in meters per second. The graduate student knows which tensor axis represents batch versus sequence. The researcher validates results against physical intuition before publication.

Back then, the code was a calculator, not a specification.

Dimensional semantics lived in lab notebooks, paper appendices, comments, and in the researcher’s head. This worked for its intended purpose: quick iteration on numerical experiments where humans closed the loop on correctness.

Year later, that stack emerged to become the foundation for autonomous vehicle perception, robotic surgery systems, industrial process control, drug discovery pipelines, and power grid management. Systems where no human supervises each inference. Where the model runs millions of times without a domain expert checking outputs. Where a dimensional error doesn’t produce a bad plot; it produces a wrong action in the physical world.

We’ve used illustrative historical examples in other blog posts to outline where that can go wrong. The Mars Climate Orbiter disintegrated in 1999 because one team used pound-force seconds while another expected newton-seconds.¹ The Ariane 5 rocket exploded in 1996 because a 64-bit floating-point value was converted to a 16-bit signed integer without range checking.² These failures occurred in systems built with extraordinary care by talented engineers. The tooling simply couldn’t catch dimensional and type errors that humans missed.

The managed runtime generation optimized for different concerns: developer productivity, rapid iteration, web services at scale. Dimensional constraints left to documentation and testing made sense for those domains. But the systems we build today increasingly interface with physical reality, and the tools we inherited were never designed to verify physical consistency.

The Clef Position

The Fidelity framework addresses this gap through Clef, a concurrent systems language that targets MLIR and native executables rather than the .NET Common Language Runtime.

This distinction matters. Clef is to F#’s compiler heritage what Chris Lattner’s Mojo intended to be to Python. In our case, we started from a foundation that didn’t require splitting the language into two function types to achieve systems programming performance.

F# already has:

Computation expressions that directly encode monadic and categorical patterns. Where other languages require complex type gymnastics to express effectful composition, F# makes it natural.

Active patterns that recognize and destructure complex patterns without verbose visitor hierarchies.

Quotations that preserve type-carrying structure for compile-time analysis and transformation.

MailboxProcessor that provides underpinnings for actor-based concurrency as a core language feature.

Units of Measure that express dimensional constraints at the type level with zero runtime cost.

Statically Resolved Type Parameters that enable compile-time specialization without source-level duplication.

These aren’t incidental features; they form a coherent toolkit for expressing the kinds of abstractions that systems programming requires. Clef inherits them from a 25-year track record in F#. The syntax remains largely unchanged. The semantics are preserved. The idioms work the same way. What changes is what survives “in the machinery” through our compilation pathway.

In standard F#, Units of Measure erase before IL generation because the .NET BCL and CLR cannot represent dimensional metadata. The inline keyword is opt-in because aggressive inlining conflicts with the CLR JIT’s assumptions. Both features are held out as optional features by platform constraints, not design flaws.

Clef has no such constraints. It targets MLIR, not IL. Dimensional types don’t erase; they flow through the Program Semantic Graph to inform code generation. Memory lifetimes are tracked through coeffect analysis, as opposed to being relegated to a managed runtime. The result is native executables that preserve the semantic richness of the source language.

Deterministic Memory Management

DMM in Fidelity draws from C++’s RAII patterns while expressing them through F# idioms. Resources have deterministic lifetimes. Cleanup happens at precise points in the program, not whenever a garbage collector decides to run. Memory allocation strategies are explicit and verifiable.

Consider a function that processes sensor data:

let processSensorBatch (sensors: Span<SensorReading>) (output: Span<ProcessedData>) =
 for i in 0 .. sensors.Length - 1 do
 output.[i] <- transform sensors.[i]

In a managed runtime, this code depends on the garbage collector to eventually reclaim any intermediate allocations. The timing is unpredictable. Memory pressure can cause GC pauses at inopportune moments. The developer has limited visibility into when and why allocations occur.

In Fidelity, the same code compiles with explicit memory semantics. The Span types carry lifetime information. The compiler verifies that output outlives the write operations. No intermediate allocations occur because the transformation happens in place. The generated code is predictable, auditable, and free of GC pauses.

This matters for real-time systems where latency spikes are unacceptable. It matters for embedded systems where memory is constrained. It matters for safety-critical systems where every allocation must be accounted for. It also happens to support large server platform work, as recent industry event have raised the spectre of “memory pressure” once again imposing a significant design constraint.

Arena Allocation and Lifetime Tracking

Fidelity supports multiple allocation strategies through its memory model:

// Arena allocation for batch processing
use arena = Arena.create (megabytes 16)
let buffer = Arena.alloc<float> arena 1024
let result = processInArena buffer
// Arena released here, all allocations freed together

// Stack allocation for small, short-lived data
let localBuffer = stackalloc<float> 64
let localResult = processLocal (Span localBuffer)
// Stack frame cleanup, zero overhead

// Reference-counted allocation for shared ownership
let shared = RefCount.create initialValue
let reference = RefCount.share shared
// Released when last reference drops

The allocation strategy is not a type parameter that forces source-level duplication. It flows through the coeffect system, resolved at call sites, verified at compile time. A function that operates on Span<float> works with arena-allocated, stack-allocated, or reference-counted memory. The proofs verify that the allocation strategy satisfies the function’s requirements. The compiler generates appropriate code for each call site.

The Dimensional Type System

DTS makes dimensional constraints intrinsic to the type system. This extends F#’s Units of Measure from an erasable annotation to a preserved semantic property that survives compilation.

To situate this precisely: the type-theoretic spectrum runs from simple types at one end to Martin-Lof’s intuitionistic type theory (MLTT) at the other. MLTT provides full dependent types with propositions-as-types; the foundation for Agda, Coq, Lean, and the theoretical ancestor of F*. In that lineage, any proposition can be expressed as a type, and any proof is a program. The power is immense; the cost is that type inference and proof search are undecidable in general, so the developer carries the burden of supplying annotations and interactive proof terms wherever the solver cannot reach.

DTS occupies a specific, well-understood position on this spectrum:

System	Expressiveness	Decidability	Survives Compilation
Simple types (Hindley-Milner)	Type structure only	Always decidable	Erased
Phantom types / F# UoM	Type-level tags, abelian group algebra	Always decidable	Erased
Refinement types (Liquid Haskell)	Predicates from decidable SMT theories	Always decidable	Erased
DTS (Fidelity)	Predicates from decidable SMT theories	Always decidable	Preserved
Dependent types (F*, Agda, Coq)	Arbitrary propositions-as-types	Inference / proof search undecidable	N/A (extraction)

In their original form, F#’s Units of Measure (including the FSharp.UMX extension library) are phantom type parameters. They constrain the type checker but erase before code generation because the .NET runtime cannot represent them. By contrast, our DTS materializes these phantom types as structured refinements that persist through the compiler’s intermediate representation. The pivotal restriction, inherited from the Liquid Haskell tradition, is that each dimensional category maps to a decidable SMT theory: physical unit algebra maps to abelian group theory, memory space compatibility to enum sorts, width constraints to bitvector theory, and so on. Because each theory is decidable, the solver always terminates with a definitive answer; no fuel heuristics, no “unknown” results, no divergence.

The deliberate tradeoff: our DTS gives up MLTT’s full propositions-as-types in exchange for decidability and the ability to preserve type information through compilation to multiple hardware targets. This choice rests on an observation about systems programming constraints. Physical units, memory access modes, wire layouts, and substrate compatibility are inherently structured and finite. They do not require the full power of dependent types. They require the solver to always say yes or no.

Samples of Dimensions in Fidelity

Intrinsic Memory semantics: ReadOnly, WriteOnly, Peripheral, Stack, Arena. Making memory access patterns explicit at the type level.

Intrinsic Tensor indices: batch, channel, height, width. Distinguishing axes that shape-only systems conflate.

Abstract measures: tokens, embeddings, attention heads. The vocabulary of machine learning architectures.

Physical units: meters, seconds, joules, amperes. The vocabulary of engineering and physics.

Financial units: USD, EUR, basis points, annualized rates. The vocabulary of quantitative finance.

Domain identifiers: customerId, orderId, sessionId. Preventing the common bug of passing the wrong ID to an API.

These dimensions don’t erase after type checking. They flow through our Program Semantic Graph and inform code generation for any target. They are erased eventually, as a zero-cost abstraction, but not until after they have supplied the information needed for an efficient and safe compute graph.

// Physical computation with verified dimensions
let computeDisplacement
 (velocity: float<meters/seconds>)
 (time: float<seconds>)
 : float<meters> =
 velocity * time // Compiler verifies: (m/s) * s = m

// Attempting to add incompatible dimensions fails at compile time
let invalid = velocity + time // Error: cannot add meters/seconds to seconds

// Memory-mapped peripheral with dimensional semantics
type TemperatureSensor = {
 Address: nativeint
 Access: ReadOnly
 Value: float<celsius>
}

let readTemperature (sensor: TemperatureSensor) : float<celsius> =
 MemoryMapped.read sensor.Address

The practical consequence: when you write velocity * time, the compiler doesn’t just check that you’re multiplying two floats. It verifies that the dimensional algebra produces meters. When you read from a peripheral register, the compiler knows the access pattern and the unit of the returned value. This information guides code generation for the target architecture.

The Unifying Insight

DMM and DTS appear to address different concerns: memory management and unit safety. The architectural insight that makes Fidelity coherent is recognizing they’re the same mechanism.

A pointer to peripheral memory at address 0x40000000 has dimensional properties:

type PeripheralRegister<[<Measure>] 'unit> = {
 Address: nativeint
 Access: AccessMode // ReadOnly | WriteOnly | ReadWrite
 Alignment: int<bytes> // Dimensional: memory layout
 Value: float<'unit> // Dimensional: physical meaning
}

let temperatureSensor : PeripheralRegister<celsius> = {
 Address = 0x40000000n
 Access = ReadOnly
 Alignment = 4<bytes>
 Value = 0.0<celsius>
}

The ReadOnly access mode is a dimension. The 4<bytes> alignment is a dimension. The celsius unit is a dimension. They all constrain how the compiler generates code for this memory location. They all survive to inform register selection, barrier insertion, and instruction scheduling on the target hardware.

This unification enables the Program Semantic Graph to represent both control-flow (for CPU/sequential targets) and data-flow (for FPGA/spatial targets) interpretations of the same source program. Dimensional constraints are invariant across both interpretations. The pivot between representations preserves semantic meaning because dimensions survive the transformation.

graph TB
subgraph "Source Program"
FSH[Clef with DMM + DTS]
end
subgraph "Program Semantic Graph"
PSG[Unified Representation]
PSG --> DIM[Dimensional Constraints]
PSG --> MEM[Memory Semantics]
PSG --> PROOF[Proof Obligations]
end
subgraph "Target Interpretations"
CF[Control-Flow View]
DF[Data-Flow View]
end
subgraph "Code Generation"
CPU[CPU / LLVM]
GPU[GPU / SPIR-V]
FPGA[FPGA / HLS]
WASM[WebAssembly / WAMI]
end
FSH --> PSG
DIM --> CF
DIM --> DF
MEM --> CF
MEM --> DF
CF --> CPU
CF --> GPU
DF --> FPGA
DF --> WASM

Coeffects: The Resolution Mechanism

A reasonable objection to this architecture concerns code duplication. If allocation strategies are part of the type, does every function need separate versions for arena, stack, and reference-counted memory?

The answer is no. Allocation strategy is a coeffect, not a type parameter.³

Traditional effect systems track what code does to its environment: performs I/O, throws exceptions, mutates state. Coeffects flip this to track what code needs from its environment: network access, specific memory patterns, suspension capability. This distinction enables allocation-agnostic source code with call-site resolution.

// The function signature expresses requirements, not strategies
let inline processData (input: Span<float>) (output: Span<float>) =
 for i in 0 .. input.Length - 1 do
 output.[i] <- transform input.[i]
 // Implicit requirements:
 // - input: ReadCapability, Lifetime(L_function)
 // - output: WriteCapability, Lifetime(L_function)

// Call sites provide capabilities through allocation strategy
let arenaInput = Arena.allocSpan<float> arena 1000
let stackOutput = stackalloc<float> 1000

processData arenaInput (Span stackOutput)
// Compiler verifies:
// - Arena satisfies ReadCapability ✓
// - Arena lifetime ⊇ function scope ✓
// - Stack satisfies WriteCapability ✓
// - Stack lifetime ⊇ function scope ✓

The function processData is written once. At each call site, the compiler determines the allocation strategy from context, verifies that the strategy satisfies the function’s requirements, and generates specialized code. No source-level duplication. No runtime dispatch. All verification happens at compile time.

The coeffect system in Fidelity tracks these contextual requirements through the Program Semantic Graph:

Function: processData
├── Parameter: input
│ ├── Type: Span<float>
│ ├── Requirement: ReadCapability
│ └── Requirement: Lifetime(L_function)
├── Parameter: output
│ ├── Type: Span<float>
│ ├── Requirement: WriteCapability
│ └── Requirement: Lifetime(L_function)
└── Proof Obligations:
├── input.Lifetime ⊇ L_function
└── output.Lifetime ⊇ L_function

This approach draws from Koka’s effect system⁴ and Rust’s lifetime elision, adapting these ideas to our dimensional type system and MLIR’s compilation model.

Proofs and Verification

For safety-critical systems, decidable verification matters. MISRA guidelines, DO-178C certification, and similar standards require traceable reasoning about program behavior. The question arises: if allocation strategy is resolved at call sites, how do proofs remain decidable?

The crux of our position: proofs constrain abstract requirements, not concrete implementations.

Abstract Requirement	Proof Obligation	Arena	RefCount	Stack
Outlives scope L	Lifetime ⊇ L	✓	✓	✓
Mutable access	WriteCapability	✓	✓	✓
No aliased writes	Uniqueness	Checkable	Checkable	Guaranteed
Thread-safe	SyncCapability	✗	✓	✓

A function requiring WriteCapability and Lifetime(L) can be called with any allocation strategy that provides those capabilities. The proof verifies the abstract requirements. The compiler verifies that each call site’s allocation strategy satisfies those requirements. Both verifications happen at compile time.

This approach integrates with Z3 and cvc5 for SMT solving, as described in Proof-Aware Compilation. The compilation ledger records each verification decision: which requirements a function has, which capabilities a call site provides, why the capabilities satisfy the requirements.

// SMT specification for verified memory operation
[<SMT Requires("writable(output) && length(output) >= length(input)")>]
[<SMT Ensures("forall i. i < length(input) ==> output[i] = transform(input[i])")>]
let processData (input: Span<float>) (output: Span<float>) =
 for i in 0 .. input.Length - 1 do
 output.[i] <- transform input.[i]

The proof obligations are allocation-agnostic. Whether output lives in an arena, on the stack, or in reference-counted memory, the proof that every element is correctly transformed remains valid.

Differentiation Through Scope

Other projects also target MLIR. Modular’s Mojo compiles Python-syntax code through MLIR to optimized machine code. The question naturally arises: what distinguishes our approach?

The answer lies in scope, not competition. Modular has built excellent infrastructure for composing expert-written kernels for ML inference. Their graph compiler operates on tensor graphs that are already dataflow representations. When they fuse matmul+bias+relu into a single kernel, they’re optimizing arithmetic on tensors of shape [batch, sequence, hidden]. This is valuable work that serves the ML inference use case well.

What Modular cannot preserve is dimensional semantics, because those semantics were never in Python or PyTorch to begin with. When Chris Lattner says “we’re not an AI compiler, we’re throwing that concept away,”⁵ he’s being precise about their scope: kernel infrastructure for experts who carry domain knowledge in their heads.

Fidelity addresses a different problem. The Program Semantic Graph represents general Clef programs with control flow, recursion, closures, and pattern matching. Via the equivalence between SSA and functional programming established by Appel⁶, the same structure can be interpreted as control-flow graph (for CPU/sequential targets) or dataflow graph (for FPGA/spatial targets). Dimensional constraints are invariant across both interpretations.

This isn’t criticism. Different solution spaces require different architectures. But the asymmetry is worth noting: our PSG can represent tensor computations and lower them through MLIR to optimized kernels. That’s just dataflow with arithmetic, which the graph handles natively. The reverse isn’t true. Modular cannot reconstruct the semantic information it never had: dimensional constraints, physical invariants, or the control-flow/dataflow duality that enables targeting other advanced accelerators. The infrastructure Modular has built serves its scope well, and the PSG covers that same tensor-kernel ground while retaining the semantic information their pipeline discards.

This matters concretely. A physics-dependent neural network built with other tools may produce outputs that fit training data. But lurking behind the scenes are violations of conservation of energy, impossible accelerations, and mixed reference frames. This is “AI hallucination” in a nutshell. No standard compiler catches these errors because dimensional information was never encoded. Our Fidelity design changes this. Auto-differentiation is designed to respect dimensional constraints, gradients carry physical units, and loss functions are checked for dimensional consistency. The training dynamics stay grounded in physical reality because the type system won’t represent the alternative.

The Ada/VHDL Heritage

This approach isn’t novel. Ada understood that physical types guide compilation. VHDL understood that timing constraints in nanoseconds affect synthesis decisions.

Ada’s type system supports dimensional analysis through derived types:

type Meters is new Float;
type Seconds is new Float;
type Meters_Per_Second is new Float;

function "/" (Left : Meters; Right : Seconds) return Meters_Per_Second;

VHDL’s physical types carry units through synthesis:

type TIME is range implementation_defined
 units
 fs;
 ps = 1000 fs;
 ns = 1000 ps;
 us = 1000 ns;
 ms = 1000 us;
 sec = 1000 ms;
 end units;

These languages emerged in contexts where software controlled hardware directly: avionics, telecommunications, industrial control. The engineers who designed them understood that dimensional information serves dual purposes: correctness verification and synthesis guidance. Timing constraints in VHDL don’t just verify that a design meets specifications; they inform the synthesizer’s routing and resource allocation decisions.

The managed runtime generation largely abandoned this discipline. Reasonable enough for web services and business applications where meters per second don’t matter. But as software increasingly interfaces with physical systems, the Ada/VHDL insight becomes relevant again.

Fidelity restores that discipline in a modern functional language, with the added capability that dimensions guide code generation for any target: CPU, GPU, FPGA, spatial accelerator. The comparison point isn’t Python or C#; it’s Ada and VHDL, languages that understood what systems programming requires.

Adoption and Innovation Budget

Every framework asks developers to learn something new. The question is whether the learning investment yields proportionate capability. Fidelity’s approach minimizes the innovation budget while maximizing the capability payoff.

For F# developers, the language is familiar. Computation expressions work the same way. Active patterns work the same way. Units of Measure work the same way, except now they don’t erase. The new learning is understanding what compilation to native code means: deterministic memory lifetimes, explicit allocation strategies, preservation of dimensional information through to machine code.

The verification depth is opt-in. Most code needs no SMT proofs. Developers add verification where the domain demands it: safety-critical paths, regulatory requirements, high-assurance components. The design is meant to span this range, from casual tooling to stringent certification requirements.

// Simple code: no proofs needed
let average (values: float[]) =
 values |> Array.average

// Safety-critical code: full verification
[<SMT Requires("length(values) > 0")>]
[<SMT Ensures("result >= min(values) && result <= max(values)")>]
let verifiedAverage (values: float[]) =
 values |> Array.average

The same language, the same idioms, different levels of assurance based on the application’s requirements.

The Multi-Stack Reality

Modern systems don’t run on single architectures. A robotics application might process sensor data on an FPGA, run inference on a GPU, execute control logic on a CPU, and communicate over a network. Current practice requires different languages, different toolchains, and manual coordination at every boundary.

Our unified representation is designed to open a new set of options for designers, integrators and decision-makers. The same Clef semantics would compile to appropriate code for each target. Dimensional constraints verified once apply across all targets. Memory semantics appropriate to each platform are generated from the same source.

// Single source, multiple targets
let processAndControl (sensor: SensorInput<celsius>) =
 // Runs on FPGA: signal processing
 let filtered = lowPassFilter sensor.Signal

 // Runs on GPU: inference
 let prediction = neuralPredict filtered

 // Runs on CPU: control logic
 let command = computeControl prediction

 // Dimensional consistency verified across all stages
 // Memory semantics appropriate to each target
 command

This is what our DMM and DTS aim for together: a coherent foundation for systems that span architectural boundaries while maintaining semantic consistency throughout. The mechanism is preserving the information that makes consistency verifiable.

Conclusion

Our Fidelity framework bets on a specific thesis: that preserving semantic information through compilation yields capabilities worth the engineering investment. The zero-cost abstraction that carries this is what makes those capabilities reachable. DMM provides deterministic memory management that makes resource lifetimes explicit and verifiable. DTS provides dimensional types that survive compilation to inform code generation. We have found no other representative implementation of this combination in the standing literature we have reviewed. Together, they are designed to support hardware-aware compilation from a single source with verified consistency across targets.

While this might not right approach for every domain, it has a high degree of reach. Web services don’t need meters per second verified at compile time, but both .NET and Fable compiled versions of F# can handle that domain admirably. Business applications don’t need deterministic microsecond-level resource cleanup, but F# has handled that efficiently for a generation. The tooling that exists serves those domains well.

And while we see our Fidelity framework as a general systems platform that can span from the server to the microcontroller, we see particular strength in targeting systems that interface with physical reality. The language is concise and familiar. The compilation target is native code via MLIR. The semantic information that previous models would discard is meant to survive and do mission-critical work.

The multi-stack world is already here, and it is the work we will keep building toward as the design comes into place.

Stephenson, A. G., et al. “Mars Climate Orbiter Mishap Investigation Board Phase I Report.” NASA, November 10, 1999. The report identified the root cause as failure to convert English units to metric units in ground-based software. ↩︎
Lions, J. L. “ARIANE 5 Flight 501 Failure Report.” European Space Agency, July 1996. The report details how a 64-bit to 16-bit conversion overflow caused the guidance system failure. ↩︎
Petricek, Tomas, Dominic Orchard, and Alan Mycroft. “Coeffects: A calculus of context-dependent computation.” ACM SIGPLAN Notices 49.9 (2014): 123-135. This paper formalizes how coeffects track what computations require from their context. ↩︎
Leijen, Daan. “Koka: Programming with row polymorphic effect types.” Electronic Proceedings in Theoretical Computer Science 153 (2014): 100-126. Koka’s effect system demonstrates how effect tracking enables compiler optimization without runtime overhead. ↩︎
Lattner, Chris. High Performance AMD GPU Programming with Mojo, 2025. In discussing Modular’s approach to ML compilation, Lattner explicitly distinguished their kernel-infrastructure focus from traditional AI compiler approaches. ↩︎
Appel, Andrew W. “SSA is functional programming.” ACM SIGPLAN Notices 33.4 (1998): 17-20. This paper establishes the equivalence between SSA form and functional programming, enabling the dual interpretation central to Fidelity’s Program Semantic Graph. ↩︎

The WREN Stack

Tue, 06 Jan 2026 00:00:00 +0000

There’s a particular kind of frustration that comes from watching a progress bar crawl across your screen while an Electron app loads its bundled Chromium instance. Somewhere in those three-plus seconds of initialization, a complete web browser is waking up, allocating its 200 megabytes of baseline memory, spawning its constellation of processes, all so you can run what amounts to a glorified file picker. The frustration isn’t about the seconds themselves. It’s about knowing that the underlying operation, reading a directory and displaying a list, should take microseconds, not seconds.

This mismatch between what desktop applications could be and what they’ve become isn’t a new complaint. Developers have grumbled about Electron’s weight since it first appeared. But the grumbling rarely produces alternatives. The web development ecosystem is simply too good: DaisyUI’s component library, Tailwind’s utility classes, SolidJS’s fine-grained reactivity. These tools represent decades of collective innovation in user interface design. Abandoning them for Qt or GTK means abandoning that innovation, trading the expressiveness of modern CSS for the rigidity of native widget toolkits.

Here we offer a new hybrid concept - the WREN Stack

W ebView
R eactive
E mbedded
N ative

The False Dichotomy

The conventional wisdom often only presents two paths for desktop development. You can have web technologies, but you pay for the convenience with Electron’s bloat. Or you can have native performance, close to the metal, but often developers are forced to write imperative widget code that hasn’t evolved much since the 1990s. The up-front burden is non-trivial and the maintenance of that crufty surface only grows as time marches on.

This framing assumes that web technologies require a web runtime. That if you want CSS and JavaScript reactivity, the assumption is that you must bundle a browser engine and accept its overhead. But the assumption is wrong. Every modern desktop operating system already ships with a web rendering engine. macOS has WKWebView. Windows has WebView2. Linux has WebKitGTK. These aren’t foreign components that need to be bundled; they’re already present, already maintained, already consuming zero additional memory until your application needs them.

The realization that unlocks the WREN Stack is simple: use the system’s WebView as a rendering surface, but run application logic as native code compiled by Composer. The WebView displays your Partas.Solid components, styled with DaisyUI, rendered by a modern browser engine. But the “backend” that drives those components isn’t JavaScript running in Node.js. It’s Clef compiled to native machine code, communicating through typed binary messages over a local WebSocket connection.

The distinction matters. In Electron, your application bundles an entire Chromium browser: hundreds of megabytes of code that duplicates what the operating system already provides. A WREN Stack application uses the system’s WebView, already installed, already maintained, already sharing memory with other applications that use it. The binary footprint drops from hundreds of megabytes to single digits. And while the WebView does run in a separate process (all modern WebViews use multi-process architectures for security and stability¹), the communication between your native logic and the UI uses BAREWire over localhost WebSockets, the same efficient binary protocol pattern that works across any IPC boundary.

%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph WREN["WREN Stack Application"]
direction TB
W1[Your Code] --> W2[Native Binary]
W2 --> W3[System WebView]
end
subgraph Electron["Electron Application"]
direction TB
E1[Your Code] --> E2[Node.js Process]
E2 --> E3[IPC Serialization]
E3 --> E4[Chromium Process]
E4 --> E5[V8 Engine]
end

The Acronym

The acronym names what the architecture achieves: Webview as the rendering surface, Reactive UI through Partas.Solid, Embedded assets frozen into the binary’s read-only memory, and Native Clef logic compiled by Composer. Like its namesake, the small bird known for its speed and resilience, WREN Stack applications are designed to be lightweight and quick to start.

The embedded assets deserve particular attention. In a traditional web application, the browser fetches HTML, CSS, and JavaScript from a server or the filesystem. These fetches take time. The page appears white while resources load. Scripts parse and execute. Only then does the interface appear.

The WREN Stack eliminates this startup latency entirely. During the build process, Composer triggers Fable to compile your Partas.Solid components to JavaScript, then runs Vite to bundle everything into a single HTML string. The CSS is purged to only the Tailwind classes you actually use. Images, icons, and fonts are inlined as Base64 data URLs, frozen directly into the markup. The result is a completely self-contained HTML document with no external references.

This string becomes a Clef literal, and Composer compiles it directly into the .rodata section of the native binary. When your application starts, WebView.setHtml passes a pointer to read-only memory. The UI doesn’t load from anywhere; it wakes up from RAM. Every image, every icon, every font glyph is already present in the binary. There are no fetches, no cache misses, no 404s.

The practical effect is immediate appearance. Users click your application icon and see the interface without delay. There’s no white flash, no loading spinner, no visible initialization. The UI appears as fast as the operating system can composite a window.

Two Perspectives, One Binary

The most significant conceptual shift in the WREN Stack is nuancing the “frontend/backend” concept. In web development, frontend and backend are easy-to-distinguish separate deployments, separate processes, often separate systems. They communicate through highly layered network protocols. This separation makes sense when the frontend runs in a user’s browser and the backend runs on a server.

A unified desktop application experience doesn’t have thes constraints. The user’s machine runs everything. And yet most desktop frameworks preserve a ‘hard’ separation, often because they evolved from web frameworks (like Electron) or because they assume network-based architectures by default. Here we’re keeping some of the design-time conventions, for clarity, while unifying “the two worlds” of frontend and backend in a way we have found no other representative implementation of in the standing literature we have reviewed.

The WREN Stack maintains two perspectives at design time to create a single executable. Your Partas.Solid code, compiled by Fable, defines the visual interface and its reactive behavior. Your Composer code defines the native logic that responds to user actions, interacts with the filesystem, and communicates with hardware. Both compile to the same binary. The “communication” between them is message passing through a local WebSocket, but the overhead is minimal: binary frames over localhost, decoded directly into typed structures on both sides. No JSON parsing, no string escaping, no schema negotiation. Just bytes that both sides understand at compile time.

The shared vocabulary lives in a Shared project that both sides reference. You define your message types once, in ordinary Clef discriminated unions and records. Fable compiles these types to each component’s representations. Composer compiles them to native structs. BAREWire ensures binary compatibility between the two representations. When the UI sends a LoadFile command, it’s sending the same bytes that the native handler will interpret as a LoadFile command. There’s no string parsing, no field name matching, no runtime type checking.

// This file is compiled by both Fable and Composer
type Command =
 | LoadFile of path: string
 | SaveFile of path: string * content: byte[]
 | QueryHardware

type Event =
 | FileLoaded of content: byte[] * metadata: FileMetadata
 | FileSaved of success: bool
 | HardwareStatus of status: HardwareInfo

The type definitions look like ordinary Clef because they are idiomatic. The magic happens at compile time. Fable produces JavaScript with codecs for these types. Composer produces native code with matching codecs. When the UI sends a command, it’s sending bytes that the native handler decodes directly; no JSON parsing, no runtime schema validation. How those bytes travel between the two worlds is the “nervous system” we’ll examine shortly. The schema itself is simply the source code, shared between both perspectives.

The Weld

Composer’s build process for WREN Stack projects coordinates two parallel tracks. The first track runs Fable on your frontend project, producing JavaScript, then runs Vite to bundle the result with your CSS and any static assets. The second track runs the Composer pipeline on your backend project, producing MLIR, then LLVM IR, then a native object file.

The tracks merge in what we call the “weld.” Composer reads the bundled index.html produced by Vite and embeds it as a string literal in the native code. This isn’t a file read at runtime; it’s a compile-time inclusion, like #include in C but for complete web assets. Composer places the string in the .rodata section alongside other static data. The linker produces a single executable containing both the native logic and the embedded UI.

%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph Build["The Weld: Dual-Track Build"]
direction LR
subgraph FableTrack["Face Track"]
F1[Frontend/*.fs] --> F2[Fable]
F2 --> F3[JavaScript]
F3 --> F4[Vite Bundle]
F4 --> F5[index.html]
end
subgraph ComposerTrack["Brain Track"]
B1[Backend/*.fs] --> B2[CCS]
B2 --> B3[PSG]
B3 --> B4[Composer/MLIR]
B4 --> B5[LLVM]
end
F5 --> Weld["Weld Point"]
B5 --> Weld
Weld --> Binary[Native Binary]
end

The result is a standalone binary with no runtime dependencies beyond the system WebView. You can copy it to another machine and run it directly. There’s no installer, no framework prerequisites, no node_modules directory that somehow grew to 500 megabytes. Just an executable.

The Nervous System

Communication between the UI and native logic happens through BAREWire over a local WebSocket connection. When the application starts, the native code spawns a WebSocket server bound to localhost. The JavaScript side connects with the standard WebSocket API. From that point forward, the two sides exchange binary frames directly.

This architecture leverages a capability that every modern WebView provides: full WebSocket support, including binary messages. The BAREWire-encoded bytes flow directly onto the wire as ArrayBuffer frames on the JavaScript side and raw byte buffers on the native side. There’s no JSON serialization, no Base64 encoding, no string escaping. Binary in, binary out.

BAREWire itself is a serialization framework designed for efficiency and type safety. If you’ve worked with Protocol Buffers or Cap’n Proto, the concept is familiar: define your message schemas, generate encoders and decoders, pass binary payloads instead of text. But BAREWire differs from these systems in one crucial respect: the schema is ordinary Clef code. You don’t write .proto files or learn a schema DSL. You write discriminated unions and records in Clef, and BAREWire derives the binary format from the type definitions. This means your types work normally in both the frontend and backend code. Pattern matching, field access, type inference; all the Clef conveniences apply.

The binary encoding is compact. A LoadFile command with a typical file path encodes to perhaps 50 bytes, compared to 100+ bytes for a JSON equivalent with its field names, quotes, and colons. More importantly, the decoding is essentially free on the native side. BAREWire decodes directly into stack-allocated structs without heap allocation or parsing overhead. The WebSocket layer adds minimal framing, and because everything stays on localhost, there’s no network latency to speak of.

Threading Without Tears

JavaScript’s single-threaded nature is both its curse and its blessing. The curse is obvious: block the main thread and the UI freezes. The blessing is subtler: single-threaded code doesn’t have race conditions.

Modern WebViews run in separate processes from your application, an architectural choice that provides security isolation and crash protection.¹ The WREN Stack embraces this separation rather than fighting it. The WebView’s UI process handles rendering and JavaScript execution. Your native binary handles application logic, file I/O, hardware access, and compute-intensive work. The two communicate through BAREWire over a localhost WebSocket connection.

This might sound like overhead, but it’s actually a feature. The same BAREWire protocol that connects your UI to your native logic is the same pattern that connects microservices, IPC channels, and distributed systems throughout the Fidelity ecosystem. The localhost socket adds negligible latency (sub-millisecond round trips) while providing a clean async boundary that prevents either side from blocking the other.

Your Partas.Solid components run in the WebView’s JavaScript context, updating the DOM through SolidJS’s fine-grained reactivity. This context must never block. When you need to read a file, query a database, or poll a hardware device, you send a BAREWire command over the WebSocket and continue rendering. The native side receives the command on a worker thread, processes it without concern for UI responsiveness, and sends the response back asynchronously.

The result is the same 60fps responsiveness, but achieved through clean async IPC rather than shared-memory threading. Progress bars update smoothly. Cancel buttons respond instantly. The user never sees the spinning beachball of death because the WebSocket’s async nature prevents either side from waiting on the other.

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant UI as WebView Process
participant WS as localhost WebSocket
participant Logic as Native Process
UI->>WS: Command (BAREWire binary frame)
WS->>Logic: Dispatch to worker thread
activate Logic
Logic->>Logic: Process (blocking OK)
Logic->>WS: Event (binary frame)
deactivate Logic
WS->>UI: Async receive
UI->>UI: Decode → Update signals

let processLargeFile (conn: WebSocket) (path: string) =
 // This runs on a logic thread, not the UI thread
 let data = Sys.readFile path |> Result.get
 let total = data.Length

 let mutable processed = 0
 for chunk in data |> Array.chunkBySize 4096 do
 processChunk chunk
 processed <- processed + chunk.Length

 // Non-blocking send to UI
 WebSocket.send conn (ProgressUpdated (float processed / float total))

 WebSocket.send conn (ProcessingComplete total)

The WebSocket.send function is non-blocking by design. It hands the binary frame to the OS kernel’s socket buffer and returns immediately. The logic thread never waits for the UI to acknowledge receipt, and the UI processes incoming frames asynchronously through its WebSocket onmessage handler. Both sides run at their own pace, synchronized only by the natural flow of messages.

Close to the Metal

The native logic in a WREN Stack application isn’t a thin wrapper around .NET APIs. It’s genuine native code, compiled through MLIR to LLVM IR to machine instructions. When you call Sys.readFile, you’re not invoking a .NET method that eventually calls a Windows API. You’re executing a syscall wrapper that the compiler generated specifically for your target platform.

This matters for more than performance. Native code can access hardware directly. WREN Stack applications can read thermal sensors, communicate over serial ports, enumerate USB devices, and interact with GPIO pins, all through CCS intrinsics that compile to the appropriate platform primitives. A .NET application would need P/Invoke declarations, marshaling attributes, and careful memory management. A WREN Stack application just calls the function.

// These aren't .NET APIs. They're CCS intrinsics
// compiled to platform syscalls.
let temp = Hardware.getCpuTemp ()
let serial = Serial.open "/dev/ttyUSB0" 115200
let devices = USB.enumerate ()

The absence of a garbage collector is equally significant. Memory in a WREN Stack application is either stack-allocated (for local variables and small structures) or explicitly managed through arenas. There are no GC pauses, no unpredictable latency spikes, no background threads consuming CPU cycles to compact heaps. The application runs exactly when you expect it to run and stops exactly when you expect it to stop.

For applications that interact with real-time systems, whether hardware protocols, audio processing, or high-frequency trading, this determinism isn’t a nice-to-have. It’s a requirement.

The Face

Partas.Solid brings the expressiveness of SolidJS to Clef, compiled through Fable to JavaScript. If you’ve used React, the component model will feel familiar: functions that return virtual DOM trees, composed from smaller components, styled with utility classes. If you haven’t, the learning curve is gentle. Components are functions. State is signals. Updates are reactive.

[<SolidComponent>]
let Counter () =
 let count, setCount = createSignal 0

 div [ class' "p-4" ] [
 p [ class' "text-xl mb-2" ] [
 text (sprintf "Count: %d" (count()))
 ]
 button [
 class' "btn btn-primary"
 onClick (fun _ -> setCount (count() + 1))
 ] [ text "Increment" ]
 ]

The createSignal function returns a getter and a setter. The getter is a function because signals in SolidJS track their dependencies at call time. When count() appears inside the text helper, SolidJS notes that this text node depends on the count signal. When setCount updates the signal, only that text node re-renders. The button doesn’t re-render. The containing div doesn’t re-render. SolidJS updates the minimal DOM slice necessary to reflect the new state.

This fine-grained reactivity is fundamentally different from React’s virtual DOM diffing. React re-renders entire component subtrees and diffs the result against the current DOM. SolidJS tracks the graph of signal dependencies and updates precisely the nodes that changed. For complex UIs with many dynamic elements, the performance difference is substantial.

The styling comes from DaisyUI, a component library built on Tailwind CSS. The btn btn-primary classes in the example produce a styled button without writing any CSS. Tailwind’s purge process, run during the Vite build, eliminates any classes you don’t use, so the final CSS payload contains only what your application needs.

Pages Without Page Loads

A common concern when building desktop applications is navigation. Users expect to move between different views: a file browser, a settings panel, a detail view, a dashboard. In traditional web development, these might be separate HTML pages. In a single-page application, they’re components swapped by a router. The WREN Stack follows the SPA model, but with a twist that makes it even more natural for desktop contexts.

Solid Router, the navigation library for SolidJS, provides the familiar abstraction of routes mapping to components. During development, you organize your code into logical pages:

// Routes defined at the application root
[<SolidComponent>]
let App () =
 Router [
 Route [ path "/"; component' Dashboard ]
 Route [ path "/files"; component' FileBrowser ]
 Route [ path "/files/:id"; component' FileDetail ]
 Route [ path "/settings"; component' Settings ]
 ]

Each “page” is a separate component, often in its own file, developed and reasoned about independently. The mental model mirrors traditional multi-page development: Dashboard.fs handles the dashboard, Settings.fs handles settings, and so on.

But there’s no server to route requests. There’s no browser URL bar to display paths. In a WebView, “routes” are really application states. Solid Router can operate in memory mode, where navigation updates internal state rather than browser history. The user clicks a sidebar link, the router swaps the active component, and the transition feels like moving between pages. Under the hood, it’s a signal update that triggers fine-grained DOM changes.

The implications for the WREN Stack are significant. Every “page” of your application is already loaded in memory. Navigation is instantaneous because there’s nothing to fetch. The router simply unmounts one component tree and mounts another. With SolidJS’s transition primitives, you can animate between views, creating a polished experience that feels native despite running in a WebView.

This architecture also simplifies state management. Since the entire application lives in one JavaScript context, shared state is just shared signals. A user preference set in Settings is immediately visible in Dashboard. There’s no need for the elaborate state synchronization mechanisms that plague traditional multi-page applications.

When the Stack Applies

The WREN Stack shines in applications that need native capabilities wrapped in modern UI. System utilities that monitor hardware, configure services, or manage files. Hardware interfaces for USB devices, serial terminals, or IoT dashboards. Specialized tools for data analysis, scientific visualization, or industrial control. Kiosk applications for point-of-sale, digital signage, or self-service terminals.

The common thread is applications where the “backend” needs direct access to the system and the “frontend” needs to look good. The WREN Stack provides both without the overhead of Electron or the complexity of cross-platform native toolkits.

There are applications where the WREN Stack is less are appropriate. Some detailed editing benefits from native controls with platform-specific behaviors. And while WebGPU is a tantalizing possibility, right now we see high performance Games needing direct access to graphics APIs that WebView can’t provide efficiently. And of course for applications outside of the desktop, where WebView is not available, this ‘stack’ it at its boundary.

But for the broad middle ground, desktop applications that need more than a command line but less than a full native visual engine, the WREN Stack offers a compelling option.

The Comparison

The numbers tell part of the story. An Electron application starts with 150-300 megabytes of bundled Chromium, 100-200 megabytes of baseline memory consumption, and 2-5 seconds of cold start time. A WREN Stack application starts with 2-10 megabytes on disk, 10-30 megabytes of memory, and 50-200 milliseconds to first paint.

These aren’t theoretical projections. They’re the natural consequence of using system components instead of bundling them. WebKitGTK on Linux, WebView2 on Windows, and WKWebView on macOS are already present, already loaded into memory for other applications, already maintained by the operating system vendor. WREN Stack applications add only their own code and embedded assets.

Both the WREN Stack and Electron use multi-process architectures; WebViews isolate rendering in separate processes for the same security and stability reasons that Chromium does. The difference is what you bundle. Electron ships its own Chromium, its own V8, its own Node.js. The WREN Stack ships only your application logic and embedded assets. The WebView comes from the operating system, shared with every other application that uses it, updated by the platform vendor, consuming zero additional disk space in your distribution.

Looking Forward

The WREN Stack extends the pattern established by SAFE and SPEC. The SAFE stack pioneered the “shared library” model where Clef code compiles to both server (.NET) and client (Fable/JavaScript) from the same source. The SPEC stack adapted this model for edge computing, targeting Cloudflare Workers instead of traditional servers. The WREN Stack brings the model to the desktop, targeting native binaries instead of cloud runtimes.

The shared vocabulary, types defined once and compiled for multiple targets, becomes increasingly powerful as the Fidelity ecosystem matures. An application might use the same domain types across a WREN Stack desktop client, a SPEC edge API, and a SAFE server backend. The schemas stay synchronized because they’re the same source files. The binary formats stay compatible because BAREWire derives them consistently.

This is near-term design rather than a distant vision. We intend the WebView bindings, the BAREWire dual-targeting, and the Composer build process that coordinates the Fable and native compilation tracks to carry a WREN Stack application from source to a running binary.

What remains is the ordinary work of any framework: expanding the API surface, polishing the developer experience, documenting the patterns, and building the community of practice. That is where our attention turns next as we keep building the WREN Stack out toward the rest of the Fidelity ecosystem.

All modern WebView implementations use multi-process architectures for security and stability. See: WebView2 Process Model (Windows), WKWebView Architecture (macOS), and WebKit2 Design (Linux/WebKitGTK). ↩︎ ↩︎

XOR: A Post-Quantum Case Study

Sun, 28 Dec 2025 00:00:00 +0000

In a universally contested future, every cryptographic operation rests on a single foundation: entropy. The keys that protect communications, the nonces that prevent replay attacks, the seeds that initialize secure protocols; all derive their strength from randomness that adversaries cannot predict or influence. When that foundation weakens, everything built upon it becomes vulnerable to harvest-now-decrypt-later strategies that patient adversaries are already executing.

Post-quantum cryptography demands larger keys and more entropy. Air-gapped systems cannot rely on network-based entropy sources. Embedded devices in contested environments face active attempts to bias or destabilize their random number generation. The question of where random numbers come from, long ignored by application developers who simply call a function and move on, becomes existential when the security of critical infrastructure depends on the answer.

This case study examines what appears to be a narrow problem: generating 4,096 bytes of cryptographic-quality random data from hardware entropy sources. The same principles extend beyond this application. Mathematical properties of the XOR operation, combined with hardware/software co-design and native compilation, produce results that managed runtimes do not reach. Using mathematics to guide both the hardware architecture and the software implementation establishes patterns for building systems where security guarantees are provable.

From Physical Entropy to Usable Bytes

The Spectrum of Randomness

Not all random numbers are created equal. Understanding the hierarchy of randomness sources reveals why this case study matters for security-critical applications.

Pseudo-Random Number Generators (PRNGs) form the foundation of most software randomness. .NET developers encounter these through System.Random, which implements a deterministic algorithm seeded by an initial value. Given the same seed, a PRNG produces the identical sequence every time. This determinism is useful for reproducible simulations and testing, but it creates a false sense of security when developers assume the output is unpredictable.

// The vulnerability: time-based seeding is guessable
var rng1 = new Random(); // Seeded from Environment.TickCount
var rng2 = new Random(); // Same millisecond = same seed
// rng1.Next() == rng2.Next() - not random at all!

// An attacker who knows when your service started
// can try seeds around that time and find your sequence

The limitation is structural. PRNGs are finite state machines with small internal state. For System.Random, knowing the seed means knowing the entire infinite sequence. The default constructor seeds from system uptime in milliseconds: roughly $2^{25}$ possibilities for a system that’s been running less than a year. An attacker can try all of them in seconds.

Cryptographically Secure PRNGs (CSPRNGs) improve on basic PRNGs by using cryptographic primitives and continuously mixing in environmental entropy. .NET’s RandomNumberGenerator.Create() delegates to operating system facilities: CryptGenRandom on Windows, /dev/urandom on Linux. These systems harvest entropy from various sources (keyboard timing, mouse movements, disk I/O patterns, network packet arrival times) and use it to re-seed cryptographic algorithms.

// .NET CSPRNG - cryptographically secure but still algorithmic
using var rng = RandomNumberGenerator.Create();
var bytes = new byte[32];
rng.GetBytes(bytes); // Draws from OS entropy pool

CSPRNGs are adequate for most security applications, but they have subtle weaknesses. The entropy sources are environmental. An attacker who controls or observes the environment (a virtual machine host, a compromised kernel, a system shortly after boot before entropy accumulates) may be able to predict or influence the “random” values. The cryptographic algorithms that expand entropy into output bytes are public; only the entropy inputs provide unpredictability.

Hardware True Random Number Generators (TRNGs) found in modern microcontrollers represent a significant improvement. ARM Cortex-M4 and M7 processors often include dedicated TRNG peripherals that sample physical noise sources: thermal noise in resistors, shot noise in transistors, or oscillator jitter. These provide non-deterministic values rooted in physical processes.

// ARM TRNG peripheral access
uint32_t random_value;
HAL_RNG_GenerateRandomNumber(&hrng, &random_value);

Hardware TRNGs are genuinely unpredictable, but they’re not immune to attack. The physical noise sources can be influenced by environmental factors: temperature changes, electromagnetic interference, power supply manipulation. Sophisticated attackers have demonstrated that hardware TRNGs can be biased or destabilized through fault injection attacks. The fundamental issue is that thermal and electrical noise, while unpredictable under normal conditions, are classical physical phenomena susceptible to classical manipulation.

Quantum Random Number Generators (QRNGs) occupy the top of the hierarchy. They derive randomness from quantum mechanical processes that are unpredictable in principle as well as in practice. The uncertainty is a property of quantum mechanics itself, confirmed by Bell test experiments that rule out local hidden variable theories. True QRNGs typically rely on single-photon detection, vacuum-state homodyne measurement, or similar mechanisms where the quantum signal survives intact to the digitizer.

Our case study implements the third tier of that hierarchy: a four-channel hardware TRNG built from avalanche diodes biased into a broad breakdown regime. The breakdown current carries both quantum tunneling and classical thermal contributions. We claim the measured entropy of that combined noise; a QRNG claim would rest instead on isolating its quantum part. The quantum-dominated operating point, where the tunneling contribution leads, is sharp. Holding a circuit on it across temperature, supply drift, and device aging is past what manufacturing tolerances allow, so a quantum claim would stake the security argument on an operating point that moves with temperature and age. What the hierarchy ranks is the source; what carries the security guarantee is how the source is combined. The honest classification is a true random number generator drawing on physical breakdown noise, with the cryptographic assurance coming from four independent channels combined under XOR. The same architecture accepts a true quantum front-end in place of the avalanche stage where an application requires that stronger physical assurance.

Avalanche Noise: Physical Randomness from Carrier Multiplication

Avalanche noise diodes generate truly random electrical signals from a chaotic cascade process. When a specially biased semiconductor junction operates in avalanche breakdown, charge carriers exhibit unpredictable behavior driven by microscopic scattering events. This is physical randomness from a chaotic cascade: the microscopic triggers involve quantum-mechanical scattering, and the macroscopic output is classical chaotic noise.

The avalanche process works as follows: a strong electric field across the semiconductor junction accelerates electrons to high energies. These energetic electrons collide with atoms in the crystal lattice, liberating additional electrons through impact ionization. Each liberated electron can trigger further ionization, creating a cascade or “avalanche” of charge carriers.

The trigger events at the microscopic level involve quantum scattering processes whose individual outcomes are indeterminate. The cascade amplification classicalizes the output, but the fundamental unpredictability of when and where ionization events occur propagates through the cascade as statistically independent noise. The result is a high-entropy chaotic signal whose long-run statistics are stable and whose individual samples are not predictable from prior state.

This distinguishes avalanche noise from thermal noise (statistically predictable in aggregate) and from oscillator jitter (chaotic but deterministic dynamics). Avalanche breakdown provides a chaotic signal whose unpredictability is rooted in microscopic processes that include quantum-mechanical scattering, classically amplified into measurable currents an ADC can sample.

For applications where randomness quality matters (cryptographic keys, secure protocol initialization, unpredictable tokens), physically-derived entropy holds up against algorithmic and environmental analysis because its randomness is rooted in physics. NIST SP 800-90B sets the relevant bar at demonstrable entropy, which a four-channel avalanche source biased into a broad breakdown regime clears with margin to spare.

The Engineering Challenge

The challenge is converting this analog chaotic phenomenon into digital bytes that software can use. An analog-to-digital converter samples the noise voltage, producing numeric values. But these raw samples carry imperfections that an attacker could exploit: bias (more ones than zeros, or vice versa), correlation between successive samples, or other statistical artifacts.

Traditional approaches address these imperfections through algorithmic post-processing:

Von Neumann debiasing reads pairs of samples and discards pairs where both samples are the same. When they differ, one ordering produces a zero, the other produces a one. This guarantees unbiased output but discards roughly 75% of samples, severely reducing throughput.

Cryptographic whitening applies hash functions like SHA-256 to spread entropy across output bits. This adds computational overhead and introduces algorithmic complexity that may itself harbor vulnerabilities.

Both approaches treat the hardware as a black box that produces imperfect data requiring software correction. But what if we could design the hardware and software together, using mathematics to guide both?

XOR as Entropy Combiner

The exclusive-or (XOR) operation has a property that makes it ideal for combining entropy sources:

If either input to XOR is random, the output is random.

This is provable from the algebra. Consider two bit sources A and B, where A is perfectly random (50% ones, 50% zeros) and B is heavily biased (90% ones). The XOR of A and B remains perfectly random:

When A equals zero (50% of the time), the output equals B
When A equals one (50% of the time), the output equals NOT B

Since A acts as a random selector between B and its complement, any bias in B is masked. The randomness of A “protects” the output from B’s imperfections.

This property means XOR never degrades entropy. It can only preserve or improve it.

Quantifying Bias Reduction

Let epsilon ( $\varepsilon$ ) represent deviation from perfect balance. A source with bias $\varepsilon$ produces ones with probability $0.5 + \varepsilon$ and zeros with probability $0.5 - \varepsilon$ . For example, $\varepsilon = 0.05$ means 55% ones and 45% zeros.

When two independent sources with bias $\varepsilon$ are XOR’d, the output bias can be calculated:

\begin{aligned} P(A \oplus B = 1) &= P(A{=}1) \times P(B{=}0) + P(A{=}0) \times P(B{=}1) \\[0.5em] &= (0.5 + \varepsilon)(0.5 - \varepsilon) + (0.5 - \varepsilon)(0.5 + \varepsilon) \\[0.5em] &= 2 \times (0.25 - \varepsilon^2) \\[0.5em] &= 0.5 - 2\varepsilon^2 \end{aligned}

The output bias is $2\varepsilon^2$ , significantly smaller than $\varepsilon$ for any reasonable bias level. A 5% bias ( $\varepsilon = 0.05$ ) becomes 0.5% after XOR’ing two sources.

This mathematical relationship guides the hardware design: instead of one entropy source requiring algorithmic correction, use multiple independent sources and let XOR’s mathematical properties provide the correction.

The Four-Channel Architecture

Extending from two channels to four provides exponential bias reduction through a parallel tree structure:

%%{init: {'theme': 'neutral'}}%%
graph TD
subgraph "Hardware Entropy Layer"
Z1[Avalanche Diode 1] --> S1[Sample CH0]
Z2[Avalanche Diode 2] --> S2[Sample CH1]
Z3[Avalanche Diode 3] --> S3[Sample CH2]
Z4[Avalanche Diode 4] --> S4[Sample CH3]
end
subgraph "Parallel XOR Tree"
S1 --> XOR1[XOR Level 1]
S2 --> XOR1
S3 --> XOR2[XOR Level 1]
S4 --> XOR2
XOR1 --> XOR3[XOR Level 2]
XOR2 --> XOR3
end
XOR3 --> OUT[Output Byte]

The tree structure matters. Level 1 performs two independent XOR operations in parallel:

$\text{CH0} \oplus \text{CH1}$ produces output with bias $2\varepsilon^2$
$\text{CH2} \oplus \text{CH3}$ produces output with bias $2\varepsilon^2$

Level 2 combines these results:

$(\text{CH0} \oplus \text{CH1}) \oplus (\text{CH2} \oplus \text{CH3})$ has bias $2 \times (2\varepsilon^2)^2 = 8\varepsilon^4$

Starting with 5% bias per channel, the four-channel tree reduces it to 0.005%: a factor of 1000 improvement. But this understates what’s actually possible.

The Co-Design Advantage

The $8\varepsilon^4$ formula has a notable implication: the starting bias $\varepsilon$ is raised to the fourth power. This means small reductions in per-channel bias yield enormous improvements in output quality. Reducing $\varepsilon$ from 5% to 2% doesn’t sound dramatic, but the final bias drops from 0.005% to 0.00001%. That represents a 400× improvement from a 2.5× change in input quality.

This is where hardware/software co-design proves its value. The mathematics tells us exactly where hardware optimization matters most. Each channel can be individually tuned:

Bias current adjustment: Resistor selection affects the avalanche operating point, shifting the noise distribution toward balance
Capacitor filtering: Per-channel decoupling shapes the frequency response, emphasizing the most random components
ADC timing: Sample rate tuning ensures each read captures independent avalanche events

With careful tuning, each channel can achieve 1-2% natural bias (compared to 5% from an unoptimized circuit). The XOR tree then reduces this to bias levels below $10^{-7}$ , exceeding cryptographic requirements without any algorithmic post-processing. The hardware provides quality; the mathematics provides certainty.

Why Independence Matters

The mathematical analysis assumes the four channels are statistically independent. If two channels were correlated (producing similar values at similar times), their XOR wouldn’t achieve the full bias reduction.

This requirement guides hardware design. Each channel needs:

Its own avalanche diode (not a shared noise source)
Isolated bias circuitry (no common current paths)
Independent power filtering (decoupling capacitors per channel)

When mathematics dictates the hardware architecture, hardware tuning multiplies the guarantees.

The Parallel Computation Model

The XOR tree maps naturally to parallel computation. The two Level 1 XOR operations have no data dependency and can execute simultaneously. Only Level 2 must wait for both Level 1 results.

This parallelism extends to the sampling itself. Reading four ADC channels is logically parallel: four independent physical processes producing four independent values. Even if hardware constraints serialize the actual reads, expressing the logical parallelism enables compilers to optimize scheduling and prepares for hardware that supports true parallel sampling.

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant CH0 as Channel 0
participant CH1 as Channel 1
participant CH2 as Channel 2
participant CH3 as Channel 3
participant XOR as XOR Tree
participant OUT as Output
par Parallel Sampling
CH0->>XOR: Sample (10-bit)
CH1->>XOR: Sample (10-bit)
CH2->>XOR: Sample (10-bit)
CH3->>XOR: Sample (10-bit)
end
par Level 1 XOR
XOR->>XOR: CH0 ⊕ CH1
XOR->>XOR: CH2 ⊕ CH3
end
XOR->>OUT: Level 2 XOR → 8-bit byte

MLIR’s scf.parallel Dialect

Our Fidelity framework will express this parallelism through MLIR’s Structured Control Flow (SCF) dialect. The scf.parallel operation declares logically parallel iterations with a reduction operation:

func.func @generateEntropyByte() -> i8 {
 %c0 = arith.constant 0 : index
 %c4 = arith.constant 4 : index
 %c1 = arith.constant 1 : index
 %init = arith.constant 0 : i8

 %result = scf.parallel (%ch) = (%c0) to (%c4) step (%c1)
 init(%init) -> i8 {

 // Read ADC channel (logically parallel)
 %sample = func.call @adc_read(%ch) : (index) -> i32

 // Extract lower 8 bits
 %mask = arith.constant 255 : i32
 %masked = arith.andi %sample, %mask : i32
 %byte = arith.trunci %masked : i32 to i8

 // XOR reduction
 scf.reduce(%byte : i8) {
 ^bb0(%a: i8, %b: i8):
 %xor = arith.xori %a, %b : i8
 scf.reduce.return %xor : i8
 }
 }

 return %result : i8
}

The scf.reduce with arith.xori expresses that XOR is the combining operation. Because XOR is associative and commutative, the compiler can structure the reduction as a parallel tree automatically.

MLIR’s lowering passes act on this specification. They transform the high-level parallel form into optimal code for the target architecture: interleaved reads, pipelined execution, or truly parallel operations depending on hardware capabilities.

Standard Dialects as Stepping Stone

The Fidelity framework’s broader vision involves custom MLIR transforms through the Alex compiler component, expressing computation through the INet/DCont duality that emerges from referential transparency. These custom transforms will enable direct expression of parallel and sequential computation patterns, with the compiler automatically selecting optimal strategies based on semantic properties.

For this initial entropy generation application, standard MLIR dialects like scf.parallel and arith provide sufficient runway. They express the essential parallelism and reduction semantics without requiring custom infrastructure. This pragmatic approach proves the hardware/software co-design methodology while establishing patterns that will transition naturally to custom transforms as the compiler matures. The XOR reduction expressed here in scf.reduce will eventually map to INet’s native parallel combination semantics; the sequential channel reads will map to DCont’s continuation-based sequencing. Standard dialects today; purpose-built custom transforms tomorrow.

Why Managed Runtimes Fall Short

Developers comfortable with Python, .NET, or Node.js might wonder why this matters. Can’t you just read four values and XOR them in any language?

The answer involves understanding what happens beneath the API surface.

The Python Reality

Python’s spidev library provides ADC access, but each read involves:

Python interpreter overhead for function dispatch
C extension boundary crossing
Kernel system call for SPI transaction
Return through the same layers
Python object allocation for the result

Benchmarking reveals Python achieves roughly 3,000 samples per second for single-channel reads. For 4,096 bytes requiring 16,384 samples (four channels × 4,096 cycles), this translates to approximately 5.5 seconds.

But the overhead runs deeper than raw speed. Python’s Global Interpreter Lock prevents true parallel execution. The garbage collector may pause execution unpredictably. The interpreted nature means every operation carries fixed overhead regardless of how simple the underlying computation.

The .NET Reality

.NET improves on Python through JIT compilation and value types, but introduces its own overhead:

// Typical .NET approach
var random = new byte[4096];
using var rng = RandomNumberGenerator.Create();
rng.GetBytes(random);

This delegates to operating system entropy sources (CryptGenRandom on Windows, /dev/urandom on Linux), which are designed for security but not for the specific performance characteristics of custom hardware. The managed runtime adds:

Object allocation for byte arrays
Virtual dispatch for the RNG implementation
Potential garbage collection pauses
No ability to express hardware-specific parallelism

But the deeper issue is structural: .NET copies data everywhere. The managed heap model requires marshaling data between managed and unmanaged contexts. Each crossing copies bytes. Reading from hardware? Copy into a managed array. Passing to a cryptographic function? Copy to ensure the GC doesn’t relocate the buffer mid-operation. Returning results? Another copy.

For cryptographic materials, the copying is a security liability. Every copy creates another location in memory where sensitive data resides. Each copy expands the attack surface: more places for memory dumps to capture, more opportunities for side-channel observation, more locations that must be explicitly zeroed. Developers must maintain constant vigilance, carefully marshaling data and manually clearing buffers, hoping they haven’t missed a copy the runtime created implicitly.

Our Fidelity zero-copy architecture eliminates this entire class of concerns. Data flows from hardware registers through computation to output without intermediate copies. There’s no managed heap to marshal across, no GC relocating buffers, no implicit copies hiding sensitive material in unexpected memory locations. The attack surface shrinks to the essential minimum: the data exists in exactly the locations the code specifies, and nowhere else.

Managed runtimes abstract the hardware away. Their abstract machine stops above the device registers, so the instruction “read these four ADC channels in parallel” lives in native code, where the channels are addressable.

The Native Compilation Advantage

Fidelity’s native compilation approach eliminates these layers:

// Clef with Fidelity - compiles to native code
let generateEntropyByte () : byte =
 let s0 = Platform.Bindings.ADC.readChannel 0
 let s1 = Platform.Bindings.ADC.readChannel 1
 let s2 = Platform.Bindings.ADC.readChannel 2
 let s3 = Platform.Bindings.ADC.readChannel 3

 let b0 = byte (s0 &&& 0xFFus)
 let b1 = byte (s1 &&& 0xFFus)
 let b2 = byte (s2 &&& 0xFFus)
 let b3 = byte (s3 &&& 0xFFus)

 (b0 ^^^ b1) ^^^ (b2 ^^^ b3)

For readers unfamiliar with Clef’s bitwise operators: ^^^ is XOR, &&& is AND, ||| is OR, and ~~~ is NOT. The triple-character syntax distinguishes bitwise operations from boolean operators (&&, ||). These operators are less commonly seen in typical Clef application code but essential for systems programming.

This compiles through MLIR to native ARM64 instructions. No interpreter. No garbage collector. No virtual dispatch. The XOR operations become single eor instructions. The ADC reads become direct register manipulation.

The parallelism expressed in the source maps to hardware capabilities. On a processor with multiple execution units, the two Level 1 XORs can execute in the same cycle. On hardware with DMA-capable ADC interfaces, all four channel reads could occur simultaneously.

Performance Projection

The mathematical model projects performance based on hardware capabilities. These figures represent targets for the prototype implementation:

Approach	Samples/sec	Time for 4096 bytes	Overhead Source
Python + Von Neumann	~200	15+ seconds	Interpreter + 75% discard
Python + LSB extraction	~3,000	~5.5 seconds	Interpreter
.NET managed	~10,000	~1.6 seconds	Runtime + GC
Native (conservative)	~40,000	~410 ms	Hardware only
Native (optimized)	~100,000	~160 ms	Pipelined

The native compilation advantage is measured in orders of magnitude. And unlike algorithmic optimizations that eventually hit diminishing returns, hardware-aware compilation scales with hardware capability.

Mathematics as Design Guide

This case study illustrates a methodology that extends beyond random number generation:

Start with mathematical properties: XOR’s entropy-preserving behavior is proven from first principles. Understanding the mathematics reveals what’s possible.
Let mathematics guide hardware: The requirement for independent channels follows directly from the statistical analysis. The parallel tree structure emerges from XOR’s associativity. The hardware architecture implements mathematical requirements.
Express parallelism explicitly: Even when hardware serializes operations, expressing logical parallelism preserves optimization opportunities. Tomorrow’s hardware may parallelize what today’s serializes.
Compile down to the hardware: Managed runtimes abstract hardware away. Native compilation embraces the realities in the substrate. The difference compounds across millions of operations.

%%{init: {'theme': 'neutral'}}%%
flowchart LR
subgraph RT["What Executes at Runtime"]
subgraph MANAGED["Managed Runtime"]
direction TB
M1[Interpreter/JIT] ~~~ M2[GC Pauses]
M2 ~~~ M3[Object Allocation]
M3 ~~~ M4[Virtual Dispatch]
M4 ~~~ M5[Bounds Checking]
end
subgraph NATIVE["Native Binary"]
direction TB
N1[Machine Instructions]
end
end

The contrast is stark: managed runtimes execute layers of abstraction on every operation. Native compilation resolves those abstractions once, at compile time, leaving only the essential computation at runtime.

Entropy Integrity Through the Pipeline

A critical consideration in cryptographic applications is maintaining entropy integrity: ensuring that the randomness present in physical processes survives through to the final output without degradation or predictable transformation.

The XOR approach provides mathematical guarantees about entropy preservation. Each bit of output depends on all four input channels. No single channel failure compromises output randomness (the other three still provide entropy). No algorithmic transformation introduces predictable patterns.

Compare this with cryptographic whitening, which applies deterministic functions to input data. While whitening spreads entropy effectively, it also transforms it through known algorithms. An attacker who knows the whitening function and can observe outputs has more information than one observing pure XOR combinations.

Our Fidelity compilation model extends this integrity guarantee through the software layer. The Clef source expresses the mathematical operation directly. MLIR preserves the operation’s semantics through lowering. Native code executes exactly what was specified. No interpreter reordering, no JIT optimization surprises, no garbage collector interference.

Fault Tolerance Through Independence

The four-channel architecture provides defense in depth against hardware failures:

One channel fails: Three remaining channels still XOR to produce random output with $\varepsilon^3$ bias
Two channels fail: Two remaining channels produce $\varepsilon^2$ bias
Three channels fail: Single channel provides original $\varepsilon$ bias

Complete entropy failure requires all four independent avalanche processes to fail simultaneously. For properly isolated circuits, this probability is negligible.

Defense Against Active Attacks

Beyond passive failures, the four-channel architecture provides defense against active electromagnetic attacks. Single-channel entropy generators, whether avalanche-based or modular noise designs, share a common vulnerability: an attacker with a remote EM source can potentially bias the output.

For avalanche generators, the attack vector is the sensitive diode output operating at micro-ampere levels. Stray currents induced by external EM fields can superimpose a repetitive or fixed waveform on the noise signal. If the system uses a simple 1-bit discriminator, this bias can force the generator into a “stuck” state, producing predictable output.

Modular noise generators face a similar threat through a different mechanism. While the noise amplifier indiscriminately amplifies all noise sources (shot noise, thermal noise, EM interference, crosstalk), an attacker who can correlate EM timing to the generator’s sampling period may influence the comparator’s voltage reference. Repetitive stray currents on this node can bias the comparator’s decisions, correlating output entropy to the external EM field until the generator enters a stuck state.

The four-channel XOR architecture defeats both attack modes. An attacker would need to simultaneously bias all four independent channels with coherent EM interference. The channels operate from separate diodes with separate bias networks; influencing one provides no leverage over the others. The XOR combination means that even if an attacker succeeds in biasing three channels completely, the fourth channel’s chaotic randomness still propagates to the output. The defense holds without relying on any quantum property of the source: four independent classical chaotic sources cannot be simultaneously coherently biased by an external field.

Physical mitigations remain straightforward for defense in depth: shielding, filtering, and physical separation. But the mathematical structure of four-channel XOR provides a layer of protection beyond what single-channel designs reach.

Synthesis

Generating 4,096 bytes of cryptographic entropy might seem like a narrow problem, but it illuminates principles that apply broadly:

Mathematical foundations matter. The XOR operation’s entropy-preserving property is a theorem. Building on proven mathematics provides guarantees beyond the reach of testing.

Hardware and software co-design multiplies effectiveness. Treating hardware as an abstract resource to be accessed through layers of abstraction leaves performance on the table. Designing hardware and software together, guided by shared mathematical understanding, achieves results neither could alone.

Software-only solutions carry hidden costs. Von Neumann debiasing, the classical algorithmic approach to entropy correction, discards at least 50% of sampled data and typically 75% or more. This might seem like a minor inefficiency, but it betrays a deeper problem: the software-only perspective accepts waste as inevitable, never questioning the system design that created it. The four-channel XOR architecture uses every sample, discarding nothing, because the hardware was designed with the mathematics in mind. In the coming era where cryptographically certified post-quantum security will emerge as a mandatory requirement, the integrity of the entire entropy generation process matters. Cryptographic assurance cannot rest solely on algorithmic innovation; it requires end-to-end verification from physical entropy source through native code to final output. Hardware/software co-design is essential for systems where security guarantees must be provable.

Parallelism is semantic. Expressing that four channels are logically independent documents system architecture, enables compiler optimization, and prepares for hardware evolution. Serial execution is an implementation detail.

Native compilation enables hardware-awareness. Managed runtimes provide convenience and safety, but they abstract away the hardware characteristics that determine real-world performance. For applications where hardware interaction dominates, native compilation provides access to capabilities that higher-level approaches cannot reach.

The XOR entropy combiner demonstrates these principles in a compact, verifiable system. The same methodology will apply to signal processing, sensor fusion, real-time control, and anywhere that physical processes meet digital computation. We will keep building our co-design approach toward those domains as the compiler and the surrounding tooling mature.

Application	Title
US 63/780,027	Air-Gapped Dual Network Architecture for QRNG Cryptographic Certificate Distribution via QR Code and Infrared Transfer in WireGuard Overlay Networks
US 63/780,055	Quantum-Resistant Hardware Security Module with Decentralized Identity Capabilities

A Unified Actor Architecture for Clef

Thu, 04 Dec 2025 00:00:00 +0000

The actor model presents a useful abstraction for concurrent and distributed systems. When building two frameworks that target different runtimes, a natural question arises: can we maintain a consistent developer experience across native compilation (Fidelity) and edge deployment (Conclave)? The answer lies in recognizing that actors are defined by their semantics, not their implementation details.

This entry explores the architectural decisions behind a unified actor abstraction designed to compile to our Olivier actors in Fidelity and to Durable Object-backed actors in Conclave. By designing at the right level of abstraction, developers would write actor behaviors once and deploy them to either target without modification.

The Core Tension

The two frameworks target radically different runtimes:

Concern	Fidelity/Olivier	Conclave/SPEC Stack
State Persistence	Memory arenas + RAII	Durable Object storage API
Supervision	Prospero on elevated thread	Worker Loader capabilities
Message Passing	BAREWire IPC	WebSocket connections
Lifecycle	Process spawn/terminate	DO instantiation/hibernation
Concurrency	OS threads + continuations	V8 isolates + event loop

Despite these differences, the developer mental model should remain consistent: actors receive messages, maintain state, spawn children, and participate in supervision hierarchies. The challenge is defining an abstraction that captures these semantics without sacrificing target-specific optimizations.

Protocol and Transport Separation

A critical architectural decision is separating protocol from transport. BAREWire serves as the shared protocol across both frameworks: the same binary message encoding, the same framing, the same schema definitions. What differs is the transport layer:

Concern	Fidelity	Conclave
Protocol	BAREWire	BAREWire
Transport	IPC channels, shared memory	WebSocket connections
Connection	OS-level primitives	Durable Object holds socket
Persistence	Process lifetime	Platform-managed

Both transports are persistent and bidirectional, which means the BAREWire protocol maps cleanly to either. HTTP’s request/response model would have introduced impedance; WebSockets preserve the channel semantics that BAREWire expects.

This separation yields a concrete benefit: a Fidelity actor and a Conclave actor can exchange messages directly if bridged, because they speak the same protocol. Only the transport adapter differs.

The Shared Abstraction

The abstraction is intrinsic to the compiler. CCS defines the actor types and interfaces that actor code depends upon, the same way it owns the rest of the Native Type Universe, so there is no shared library for the targets to link against. The compiler carries no execution policy with those types; each target provides its own runtime that interprets them, the native Olivier runtime on one side and our Conclave edge platform on Cloudflare on the other.

The core types form a vocabulary for describing actor behavior without committing to execution details. An ActorRef is an opaque handle for sending messages; how the message reaches its destination is the runtime’s concern. An ActorBehavior specifies how an actor responds to messages; how that specification executes depends on the target. This separation enables the “write once, deploy anywhere” property.

Readers familiar with Erlang/OTP, Akka, or Orleans will recognize these patterns. The discriminated union ActorEffect resembles the behavior return types in those systems. The SupervisionStrategy cases mirror OTP’s supervision tree options, explored in depth in Erlang Lessons in Fidelity. The novelty lies not in the patterns themselves but in their application across compilation targets as different as native code and JavaScript on edge infrastructure.

// Olivier - actor types intrinsic to CCS, shared across targets
namespace Olivier

/// Marker for actor message types
type IActorMessage = interface end

/// Actor reference - opaque handle for message sending
[<Struct>]
type ActorRef<'Message when 'Message :> IActorMessage> = {
 Id: ActorId
 // Target-specific routing info is internal
}

/// Core actor behavior definition
type ActorBehavior<'State, 'Message when 'Message :> IActorMessage> = {
 /// Initial state factory
 Init: unit -> 'State

 /// Message handler - returns new state
 Receive: 'State -> 'Message -> ActorEffect<'State>

 /// Optional: handle actor lifecycle events
 Lifecycle: 'State -> LifecycleEvent -> ActorEffect<'State>
}

/// Effects an actor can produce (interpreted by runtime)
and ActorEffect<'State> =
 | Continue of 'State
 | ContinueWith of 'State * SideEffect list
 | Stop of reason: string
 | Restart of 'State
 | Escalate of exn

and SideEffect =
 | Tell of target: ActorId * message: obj
 | Spawn of spec: ActorSpec * replyTo: ActorId option
 | Schedule of delay: TimeSpan * message: obj
 | Log of level: LogLevel * message: string

/// Supervision strategy (same semantics, different implementations)
type SupervisionStrategy =
 | OneForOne of maxRestarts: int * window: TimeSpan
 | AllForOne of maxRestarts: int * window: TimeSpan
 | RestForOne of maxRestarts: int * window: TimeSpan

/// Actor specification for spawning
type ActorSpec = {
 Behavior: obj // Type-erased ActorBehavior
 Supervision: SupervisionStrategy option
 Capabilities: Set<Capability>
 Name: string option
}

The point of emphasis is that ActorBehavior can be considered a pure function from state and message to effects. No I/O, no platform dependencies. The ActorEffect type describes what should happen; the runtime interprets these effects according to target-specific semantics.

This effect-based design enables testability. An actor behavior can be tested by supplying state and messages, then inspecting the returned effects. No network, no storage, no platform; pure data in, pure data out. The runtime that eventually executes these effects is tested separately against the effect contract. This purity also opens the door to formal verification, where SMT proofs can validate actor behavior properties at compile time.

Computation Expression Layer

While the types above fully specify actor behavior, writing handlers by manually constructing ActorEffect values is verbose. Clef’s computation expressions provide syntactic sugar that makes actor code read naturally. The actor { } builder transforms familiar constructs like return, let!, and custom operations into the underlying effect types.

Readers unfamiliar with Clef computation expressions can think of them as programmable syntax. The builder type defines how expressions within the { } block translate to method calls. This mechanism powers Clef’s async, seq, and query expressions; we apply the same technique to actor behaviors.

module Olivier.Builder

type ActorBuilder() =
 member _.Yield(_) = Continue Unchecked.defaultof<_>

 member _.Bind(effect: ActorEffect<'a>, f: 'a -> ActorEffect<'b>) =
 match effect with
 | Continue state -> f state
 | ContinueWith(state, effects) ->
 match f state with
 | Continue s' -> ContinueWith(s', effects)
 | ContinueWith(s', effects') -> ContinueWith(s', effects @ effects')
 | other -> other
 | Stop reason -> Stop reason
 | Restart state -> Restart state
 | Escalate exn -> Escalate exn

 member _.Return(state) = Continue state

 member _.ReturnFrom(effect) = effect

 [<CustomOperation("tell")>]
 member _.Tell(state, target: ActorRef<'M>, message: 'M) =
 ContinueWith(state, [Tell(target.Id, box message)])

 [<CustomOperation("spawn")>]
 member _.Spawn(state, spec: ActorSpec) =
 ContinueWith(state, [Spawn(spec, None)])

 [<CustomOperation("scheduleOnce")>]
 member _.ScheduleOnce(state, delay: TimeSpan, message: 'M) =
 ContinueWith(state, [Schedule(delay, box message)])

 [<CustomOperation("stop")>]
 member _.Stop(_, reason: string) = Stop reason

let actor = ActorBuilder()

The custom operations (tell, spawn, scheduleOnce, stop) extend the base computation expression with actor-specific vocabulary. When a developer writes tell target message inside an actor { } block, the builder transforms this into a ContinueWith effect containing a Tell side effect. The syntactic lightness hides the underlying effect machinery.

Identical Usage Across Targets

With the abstraction and computation expression in place, the developer experience strives to be consistent regardless of compilation target. A counter actor looks the same whether it will run as native code in Fidelity or as JavaScript in a Cloudflare Durable Object. The types, the pattern matching, the effect operations: should all be identical.

The following example defines a simple counter with three message types. Increment and Decrement modify state. GetCount queries state and sends a response to the provided reply address. The lifecycle handler acknowledges start, stop, and restart events without performing special actions. This behavior definition is complete and target-agnostic.

// Works identically in Fidelity (native) and Conclave (Cloudflare)
type CounterMessage =
 | Increment
 | Decrement
 | GetCount of replyTo: ActorRef<CountResponse>
 interface IActorMessage

type CountResponse =
 | Count of int
 interface IActorMessage

let counterBehavior = {
 Init = fun () -> 0

 Receive = fun count msg -> actor {
 match msg with
 | Increment ->
 return count + 1

 | Decrement ->
 return count - 1

 | GetCount replyTo ->
 tell replyTo (Count count)
 return count
 }

 Lifecycle = fun count event -> actor {
 match event with
 | PreStart ->
 return count
 | PostStop ->
 return count
 | PreRestart reason ->
 return count
 }
}

This behavior definition compiles to either target without modification. The business logic remains pure; platform concerns are handled by the runtime.

Fidelity/Olivier Runtime

In the Fidelity framework, actors execute as native code compiled through MLIR. The runtime builds on Clef’s MailboxProcessor, a primitive that serializes message delivery to a single logical thread. This serialization eliminates data races within an actor without requiring explicit locks.

The implementation pairs each actor with a memory arena, a pre-allocated region from which the actor draws its allocations. When the actor terminates or restarts, the entire arena releases as a unit. This approach aligns with Fidelity’s RAII model, detailed in RAII in Olivier and Prospero, where resource lifetimes bind to continuation boundaries.

// Olivier.Runtime - Native compilation target
namespace Olivier.Runtime

open Olivier

/// Native actor instance with memory arena
type OlivierActor<'State, 'Message when 'Message :> IActorMessage>
 (behavior: ActorBehavior<'State, 'Message>, arena: MemoryArena) =

 let mutable state = behavior.Init()
 let mailbox = MailboxProcessor<'Message>.Start(fun inbox ->
 let rec loop () = async {
 let! msg = inbox.Receive()
 let effect = behavior.Receive state msg
 state <- interpretEffect effect
 return! loop()
 }
 loop()
 )

 interface IDisposable with
 member _.Dispose() =
 arena.Release()

 member _.Tell(msg: 'Message) = mailbox.Post(msg)
 member _.State = state

/// Prospero supervisor for native actors
type Prospero(strategy: SupervisionStrategy) =
 let children = ConcurrentDictionary<ActorId, IDisposable>()

 member _.Supervise(id: ActorId, actor: IDisposable) =
 children.TryAdd(id, actor) |> ignore

 member _.HandleFailure(id: ActorId, exn: exn) =
 match strategy with
 | OneForOne(max, window) ->
 restartActor id
 | AllForOne(max, window) ->
 children.Keys |> Seq.iter restartActor
 | RestForOne(max, window) ->
 restartFrom id

RAII cleanup happens at continuation boundaries. When an actor terminates or restarts, its memory arena releases deterministically. Our Prospero supervisor, named for the orchestrating figure in Shakespeare’s The Tempest, runs on an elevated thread and monitors child actors for failures. Its role is to apply supervision policies: restart the failed actor alone, restart all siblings, or restart actors spawned after the failed one. This actor-oriented architecture provides process-level protection without runtime overhead.

Conclave Runtime with WebSockets

Conclave targets Cloudflare’s edge infrastructure, where Durable Objects provide the persistence and isolation guarantees. Unlike native actors that live in OS processes, Conclave actors exist as JavaScript objects within V8 isolates. The platform may evict these objects from memory during idle periods, a process called hibernation. When a message arrives for a hibernated actor, the platform instantiates a fresh object and rehydrates its state from durable storage.

WebSocket connections serve as the transport layer. Each actor-to-actor relationship manifests as a persistent bidirectional channel. The Durable Object “accepts” the server side of a WebSocket pair, allowing it to receive messages and send responses without the overhead of HTTP request/response cycles.

The implementation below shows how a Conclave actor handles WebSocket upgrades, processes incoming messages, and manages connections to peer actors:

// Conclave.Runtime - WebSocket-based actor
namespace Conclave.Runtime

open Olivier
open Fable.Core

[<AttachMembers>]
type ConclaveActor<'State, 'Message when 'Message :> IActorMessage>
 (behavior: ActorBehavior<'State, 'Message>) =
 inherit DurableObject()

 let mutable state: 'State option = None
 let connections = Dictionary<ActorId, WebSocket>()

 /// Accept WebSocket upgrade for actor-to-actor communication
 member this.fetch(request: Request) = promise {
 let upgradeHeader = request.headers.get("Upgrade")

 if upgradeHeader = Some "websocket" then
 let pair = WebSocketPair.Create()
 let client = pair.Client
 let server = pair.Server

 this.ctx.acceptWebSocket(server)

 return Response.create(
 null,
 {| status = 101; webSocket = client |}
 )
 else
 return Response.json({| status = "healthy" |})
 }

 /// Handle incoming WebSocket messages
 member this.webSocketMessage(ws: WebSocket, message: string) = promise {
 // Hydrate state if needed (after hibernation)
 if state.IsNone then
 let! stored = this.ctx.storage.get<'State>("state")
 state <- stored |> Option.orElseWith (fun () -> Some (behavior.Init()))

 let actorMessage = deserialize<'Message> message

 let effect = behavior.Receive state.Value actorMessage
 let (newState, sideEffects) = interpretEffect effect

 do! this.ctx.storage.put("state", newState)
 state <- Some newState

 for effect in sideEffects do
 do! this.executeSideEffect(effect)
 }

 /// Handle WebSocket close
 member this.webSocketClose(ws: WebSocket, code: int, reason: string) =
 let actorId = findActorBySocket ws
 connections.Remove(actorId) |> ignore

 /// Execute side effects over WebSocket connections
 member private this.executeSideEffect(effect: SideEffect) = promise {
 match effect with
 | Tell(targetId, msg) ->
 match connections.TryGetValue(targetId) with
 | true, ws ->
 ws.send(serialize msg)
 | false, _ ->
 let! ws = this.connectToActor(targetId)
 connections.[targetId] <- ws
 ws.send(serialize msg)

 | Spawn(spec, replyTo) ->
 let! actorRef = this.spawnChild(spec)
 match replyTo with
 | Some ref ->
 do! this.executeSideEffect(Tell(ref.Id, box (Spawned actorRef)))
 | None -> ()

 | Schedule(delay, msg) ->
 do! this.ctx.storage.setAlarm(DateTime.UtcNow + delay)
 do! this.ctx.storage.put("scheduled", msg)

 | Log(level, message) ->
 console.log($"[{level}] {message}")
 }

 /// Establish WebSocket connection to another actor
 member private this.connectToActor(targetId: ActorId) = promise {
 let targetStub = this.env.ACTORS.[targetId]
 let targetUrl = $"wss://{targetId}/ws"

 let! response = targetStub.fetch(
 Request.create(targetUrl, {|
 headers = {| Upgrade = "websocket" |}
 |})
 )

 return response.webSocket
 }

 /// Handle scheduled alarms
 member this.alarm() = promise {
 let! scheduled = this.ctx.storage.get<obj>("scheduled")
 match scheduled with
 | Some msg ->
 do! this.ctx.storage.delete("scheduled")
 do! this.webSocketMessage(null, serialize msg)
 | None -> ()
 }

Durable Object hibernation is handled transparently by the Cloudflare platform. When no messages arrive for a configured period, the platform evicts the object from memory but preserves WebSocket connections. Upon receiving a new message, the platform reinstantiates the object and delivers the message to the appropriate handler. State rehydrates lazily from storage when the handler first accesses it.

The alarm API enables scheduled message delivery. When an actor calls setAlarm, the platform guarantees invocation of the alarm() method at approximately the specified time, even if the object hibernates in the interim. This mechanism supports time-based behaviors such as heartbeat generation, retry scheduling, and session timeouts.

Channel Abstraction

Both runtimes need a way to send and receive byte streams. The channel abstraction captures this requirement without prescribing the underlying mechanism. A channel is bidirectional, persistent, and operates on raw bytes. The BAREWire encoding happens above this layer; the channel moves frames without interpreting their contents.

This separation allows Fidelity actors to communicate over IPC endpoints (OS-level primitives optimized for local communication) while Conclave actors use WebSockets (the only persistent connection mechanism available in the Cloudflare environment). Both present the same interface to actor code:

// Olivier.Channels - Transport abstraction
namespace Olivier.Channels

/// Bidirectional persistent channel between actors
/// Carries BAREWire-encoded frames over transport
type IActorChannel =
 abstract member Send: byte[] -> Async<unit>
 abstract member Receive: unit -> Async<byte[]>
 abstract member Close: unit -> Async<unit>
 abstract member IsOpen: bool

// Fidelity implementation: IPC transport
type IPCChannel(endpoint: IPC.Endpoint) =
 interface IActorChannel with
 member _.Send(data) = async {
 do! endpoint.WriteAsync(data)
 }
 member _.Receive() = async {
 return! endpoint.ReadAsync()
 }
 member _.Close() = async {
 endpoint.Dispose()
 }
 member _.IsOpen = endpoint.IsConnected

// Conclave implementation: WebSocket transport
type WebSocketChannel(ws: WebSocket) =
 let receiveQueue = Queue<byte[]>()
 let mutable pendingReceive: AsyncReplyChannel<byte[]> option = None

 do ws.addEventListener("message", fun e ->
 let data = e.data :?> ArrayBuffer |> toByteArray
 match pendingReceive with
 | Some reply ->
 reply.Reply(data)
 pendingReceive <- None
 | None ->
 receiveQueue.Enqueue(data)
 )

 interface IActorChannel with
 member _.Send(data) = async {
 ws.send(ofByteArray data)
 }
 member _.Receive() = async {
 if receiveQueue.Count > 0 then
 return receiveQueue.Dequeue()
 else
 return! Async.FromContinuations(fun (resolve, _, _) ->
 pendingReceive <- Some { new AsyncReplyChannel<_> with
 member _.Reply(v) = resolve v
 }
 )
 }
 member _.Close() = async {
 ws.close()
 }
 member _.IsOpen = ws.readyState = WebSocketState.Open

The byte[] passed through these channels is always BAREWire-encoded. The channel implementation neither knows nor cares about message structure; its responsibility is frame delivery with ordering guarantees.

The WebSocket implementation deserves attention for its handling of asynchronous receives. JavaScript WebSockets deliver messages through event callbacks, but Clef async workflows expect a pull-based model. The pendingReceive mechanism bridges this gap: if a receive request arrives before data, it stores a continuation; if data arrives before a receive request, it queues in the receive buffer. This pattern recurs in any system that bridges event-driven and continuation-based concurrency models.

Ask Pattern Over Persistent Connections

The tell-and-forget model suffices for many actor interactions, but some scenarios require request-response semantics. A counter actor that receives GetCount needs to send the current value back to the requester. The ask pattern addresses this need.

With bidirectional persistent connections already in place, implementing ask is straightforward. The sender generates a correlation ID, includes it in the outbound message, and awaits a response with matching correlation. The channel’s persistent nature means no connection setup overhead per request; messages flow over the existing pipe:

type ActorRef<'Message when 'Message :> IActorMessage> with

 member this.Ask<'Response>(message: 'Message, timeout: TimeSpan) : Async<'Response> =
 async {
 let correlationId = Guid.NewGuid()
 let! channel = ChannelFactory.Connect(this.Id)

 let envelope = {|
 correlationId = correlationId
 payload = message
 expectsReply = true
 |}
 do! channel.Send(serialize envelope)

 let! response =
 channel.Receive()
 |> Async.withTimeout timeout

 let responseEnvelope = deserialize<ResponseEnvelope<'Response>> response

 if responseEnvelope.correlationId = correlationId then
 return responseEnvelope.payload
 else
 return! failwith "Correlation mismatch"
 }

The implementation differs between targets (BAREWire framing over IPC vs. WebSocket binary messages), but the developer API is identical. This uniformity means code that uses the ask pattern works without modification across Fidelity and Conclave deployments.

Timeout handling deserves consideration. In a distributed system, a response might never arrive due to network partition, target actor failure, or processing delays. The withTimeout combinator transforms a potentially unbounded wait into a bounded operation that either succeeds with a value or fails with a timeout exception. Callers must decide how to handle timeout failures: retry, escalate, or proceed with a default value.

Supervision in Conclave

Supervision is the mechanism by which parent actors monitor and respond to child actor failures. In Erlang/OTP, this pattern forms the foundation of fault-tolerant systems: let components fail, then recover according to policy. The same concept applies to Conclave, with WebSocket connections serving as the monitoring channel.

When a supervisor spawns a child actor, it establishes a WebSocket connection to that child. The supervisor attaches event listeners for close and error events. If the child’s Durable Object crashes or becomes unreachable, the platform closes the WebSocket, triggering the supervisor’s failure handler. The handler then applies the configured strategy: restart only the failed child (one-for-one), restart all children (all-for-one), or restart the failed child and those spawned after it (rest-for-one).

type ConclaveSupervisor(strategy: SupervisionStrategy) =
 inherit DurableObject()

 let children = Dictionary<ActorId, WebSocket * ActorSpec>()

 member this.spawnChild(spec: ActorSpec) = promise {
 let! actor = this.env.loader.loadWorker {
 script = spec.implementation
 bindings = {|
 SUPERVISOR = this.env.SELF
 |}
 props = spec.Capabilities
 }

 let! ws = this.connectToActor(actor.id)
 children.[actor.id] <- (ws, spec)

 ws.addEventListener("close", fun _ ->
 this.handleChildDisconnect(actor.id)
 )
 ws.addEventListener("error", fun e ->
 this.handleChildError(actor.id, e)
 )

 return ActorRef(actor.id)
 }

 member private this.handleChildDisconnect(childId: ActorId) =
 match strategy with
 | OneForOne(maxRestarts, window) ->
 let (_, spec) = children.[childId]
 this.spawnChild(spec) |> ignore

 | AllForOne(maxRestarts, window) ->
 let specs = children.Values |> Seq.map snd |> List.ofSeq
 children.Clear()
 for spec in specs do
 this.spawnChild(spec) |> ignore

 | RestForOne(maxRestarts, window) ->
 ()

WebSocket close events trigger supervision logic. The platform handles the mechanics of connection monitoring; the supervisor handles policy decisions. This separation of concerns mirrors the broader architecture: platform capabilities provide the foundation, while application code defines behavior.

The restart limits (maxRestarts within a time window) prevent infinite restart loops. If a child fails repeatedly, the supervisor eventually gives up and escalates the failure to its own supervisor, or terminates if it has no parent. This backoff mechanism prevents a single buggy component from consuming unbounded resources through continuous respawning.

BAREWire as the Universal Protocol

Regardless of transport mechanism, all actor messages use BAREWire encoding. BAREWire is a binary serialization protocol derived from the BARE specification, designed for minimal overhead and zero-copy potential. A BAREWire message is compact, deterministic, and self-describing through schema.

The message envelope wraps every actor-to-actor communication. It contains routing information (source and target actor IDs), optional correlation ID for request-response matching, message type discriminator, the payload itself, and a timestamp for debugging and ordering purposes.

// Olivier.Protocol - BAREWire message framing
namespace Olivier.Protocol

/// Actor message envelope (BAREWire-encoded on all targets)
type MessageEnvelope = {
 sourceId: ActorId
 targetId: ActorId
 correlationId: Guid option
 messageType: string
 payload: byte[]
 timestamp: int64
}

module ActorProtocol =
 let encode (envelope: MessageEnvelope) : byte[] =
 BAREWire.Codec.encode envelope

 let decode (data: byte[]) : MessageEnvelope =
 BAREWire.Codec.decode<MessageEnvelope> data

The transport is abstracted behind the channel interface. Fidelity writes BAREWire frames to IPC channels. Conclave writes the same BAREWire frames as WebSocket binary messages. The protocol is identical; only the delivery mechanism changes.

This uniformity has practical benefits for debugging and cross-system communication. A BAREWire message captured from a Fidelity IPC channel has the same structure as one captured from a Conclave WebSocket. Diagnostic tools work across both environments. If a future requirement involves bridging Fidelity and Conclave systems, the bridge only needs to handle transport translation; protocol translation is unnecessary.

Connection Topology

Actor systems naturally form graph structures as actors spawn children and establish peer relationships. The tell-first architecture, combined with persistent WebSocket connections in Conclave, produces topologies where connections remain open for the lifetime of the relationship.

%%{init: {'theme': 'neutral'}}%%
graph TD
subgraph "Conclave Actor System"
SUP[Supervisor<br/>Prospero]
SUP ---|WebSocket| A[Worker A<br/>Olivier]
SUP ---|WebSocket| B[Worker B<br/>Olivier]
SUP ---|WebSocket| D[Worker D<br/>Olivier]
A ---|WebSocket| C[Worker C<br/>Olivier]
B ---|WebSocket| C
end

All connections are persistent WebSockets carrying BAREWire-encoded frames. The supervisor monitors all child connections through event listeners. Workers can establish direct connections for peer-to-peer communication when the interaction pattern benefits from bypassing the supervisor.

This topology emerges organically from actor relationships. When Actor A sends its first message to Actor B, a connection forms. Subsequent messages reuse that connection. The connection closes when either actor terminates or explicitly closes the channel. The platform (Cloudflare for Conclave, OS for Fidelity) handles the low-level details of connection maintenance, keepalives, and cleanup.

State Persistence Abstraction

State management represents the most significant divergence between targets. Fidelity actors hold state in memory, potentially backed by memory-mapped files for crash recovery. Conclave actors must persist state to Durable Object storage because the platform may evict them from memory at any time. Despite this fundamental difference, the abstraction presents a uniform interface.

The IActorState interface defines three operations: load the current state, save a new state, and execute a transactional update. The transactional operation is particularly important for Conclave, where state mutations must be atomic to maintain consistency across hibernation cycles.

// Olivier.State - Abstraction over state persistence
namespace Olivier.State

type IActorState<'State> =
 abstract member Load: unit -> Async<'State option>
 abstract member Save: 'State -> Async<unit>
 abstract member Transaction: ('State -> 'State * 'Result) -> Async<'Result>

// Fidelity implementation: synchronous, in-memory
type OlivierState<'State>(arena: MemoryArena) =
 let mutable state: 'State option = None

 interface IActorState<'State> with
 member _.Load() = async { return state }
 member _.Save(s) = async { state <- Some s }
 member _.Transaction(f) = async {
 let (newState, result) = f state.Value
 state <- Some newState
 return result
 }

// Conclave implementation: async, persistent
type DurableObjectState<'State>(storage: DurableObjectStorage) =
 interface IActorState<'State> with
 member _.Load() = async {
 let! stored = storage.get<'State>("state") |> Async.AwaitPromise
 return stored
 }
 member _.Save(s) = async {
 do! storage.put("state", s) |> Async.AwaitPromise
 }
 member _.Transaction(f) = async {
 return! storage.transaction(fun txn ->
 let current = txn.get<'State>("state")
 let (newState, result) = f current
 txn.put("state", newState)
 result
 ) |> Async.AwaitPromise
 }

Shared Edges Summary

The abstraction boundaries clarify what’s shared and what’s target-specific:

Layer	Shared	Target-Specific
ActorBehavior	Definition	Interpretation
ActorRef	API	Routing implementation
Tell/Ask	Semantics	Channel type
Supervision	Strategy types	Platform integration
Protocol	BAREWire encoding	-
Transport	-	IPC vs WebSocket
State Persistence	Load/Save API	Storage backend
Channel Abstraction	Send/Receive	Transport binding

Addendum: Production-Ready .NET Actors

While the compiler-intrinsic actor types provide the unified abstraction for Fidelity and Conclave, developers working exclusively in .NET often ask: “How do we build production-ready actors with MailboxProcessor?” This addendum addresses that question with patterns that stand alone from the native compilation path.

The .NET Actor Landscape

Approach	Runtime	Transport	Use Case
Fidelity/Olivier	Native via MLIR	IPC, shared memory	High-performance systems
Conclave/SPEC	Fable → JS on Cloudflare	WebSocket	Edge deployment
.NET MailboxProcessor	Standard .NET	Configurable	On-premises, development, migration

A standalone .NET actor implementation serves multiple purposes:

On-premises deployment: Organizations that cannot use edge compute or native compilation
Development environment: Rapid prototyping before committing to a deployment target
Migration path: Existing .NET applications can adopt actor patterns incrementally
Reference patterns: Demonstrates production scaffolding without exotic compilation

The .NET Runtime

The .NET implementation demonstrates that the abstraction does not require exotic compilation targets. Standard F# on the .NET runtime provides everything needed: MailboxProcessor for message serialization, ConcurrentDictionary for thread-safe collections, and async workflows for non-blocking operations.

The DotNetActor class wraps a MailboxProcessor and adds the scaffolding that production systems require: error handling with configurable policies, resource tracking for automatic cleanup, metrics collection for observability, and supervisor integration for fault tolerance. The core message loop mirrors the Fidelity implementation but uses .NET’s managed memory model.

The code below is extensive because it addresses the complete lifecycle: message receipt, behavior application, effect interpretation, error handling, resource cleanup, and child supervision. Each concern is factored into a separate local function that the main loop coordinates:

// Standalone .NET actor implementation
namespace FSharp.Actors

open System.Collections.Concurrent

/// MailboxProcessor-backed actor with full scaffolding
type DotNetActor<'State, 'Message when 'Message :> IActorMessage>
 (behavior: ActorBehavior<'State, 'Message>,
 config: ActorConfig,
 metrics: ActorMetrics option,
 supervisor: ISupervisor option) =

 let mutable state = behavior.Init()
 let mutable restartCount = 0
 let resources = ConcurrentDictionary<ResourceId, IDisposable>()
 let children = ConcurrentDictionary<ActorId, DotNetActorRef>()

 let errorHandler =
 config.ErrorPolicy
 |> Option.defaultValue ErrorPolicy.defaultPolicy

 let mailbox = MailboxProcessor<ActorMessage<'Message>>.Start(fun inbox ->
 let rec loop () = async {
 let! envelope = inbox.Receive()

 match envelope with
 | UserMessage msg ->
 do! processMessage msg
 return! loop()

 | SystemMessage sysMsg ->
 match sysMsg with
 | Terminate reason ->
 do! cleanup reason
 | Restart ->
 do! restart()
 return! loop()
 | ChildFailed(childId, exn) ->
 do! handleChildFailure childId exn
 return! loop()
 }
 and processMessage msg = async {
 let sw = Stopwatch.StartNew()
 try
 // Pre-receive hook for metrics
 metrics |> Option.iter (fun m -> m.MessagesReceived.Increment())

 // Apply behavior
 let effect = behavior.Receive state msg

 // Interpret effect
 let! newState = interpretEffect effect
 state <- newState

 // Post-receive metrics
 metrics |> Option.iter (fun m ->
 m.MessagesProcessed.Increment()
 m.ProcessingTime.Record(sw.ElapsedMilliseconds))

 with exn ->
 metrics |> Option.iter (fun m -> m.ErrorCount.Increment())
 do! handleError exn
 }
 and interpretEffect effect = async {
 match effect with
 | Continue newState ->
 return newState

 | ContinueWith(newState, sideEffects) ->
 for effect in sideEffects do
 do! executeSideEffect effect
 return newState

 | Stop reason ->
 do! cleanup reason
 return state

 | Restart newState ->
 restartCount <- restartCount + 1
 metrics |> Option.iter (fun m -> m.RestartCount.Increment())
 return newState

 | Escalate exn ->
 match supervisor with
 | Some sup -> sup.HandleFailure(selfId, exn)
 | None -> raise exn
 return state
 }
 and executeSideEffect effect = async {
 match effect with
 | Tell(targetId, msg) ->
 // Route through actor registry
 match ActorRegistry.tryFind targetId with
 | Some target -> target.Post(msg)
 | None ->
 // Remote actor - use transport
 do! Transport.send targetId (BAREWire.encode msg)

 | Spawn(spec, replyTo) ->
 let! childRef = spawnChild spec
 children.TryAdd(childRef.Id, childRef) |> ignore
 match replyTo with
 | Some ref ->
 do! executeSideEffect (Tell(ref.Id, box (Spawned childRef)))
 | None -> ()

 | Schedule(delay, msg) ->
 do! Async.Sleep(int delay.TotalMilliseconds)
 mailbox.Post(UserMessage (msg :?> 'Message))

 | Log(level, message) ->
 Logger.log level message
 }
 and handleError exn = async {
 match errorHandler exn state with
 | Resume -> ()
 | RestartWith newState ->
 state <- newState
 restartCount <- restartCount + 1
 | Escalate ->
 match supervisor with
 | Some sup -> sup.HandleFailure(selfId, exn)
 | None -> raise exn
 | ErrorPolicy.Stop reason ->
 do! cleanup reason
 }
 and cleanup reason = async {
 // Run cleanup hooks
 for KeyValue(_, resource) in resources do
 try resource.Dispose() with _ -> ()
 resources.Clear()

 // Terminate children
 for KeyValue(id, child) in children do
 child.Post(SystemMessage (Terminate reason))
 children.Clear()
 }
 and restart () = async {
 do! cleanup "restart"
 state <- behavior.Init()
 }
 and handleChildFailure childId exn = async {
 match config.Supervision with
 | Some (OneForOne(maxRestarts, window)) ->
 match children.TryGetValue(childId) with
 | true, child -> child.Post(SystemMessage Restart)
 | false, _ -> ()

 | Some (AllForOne(maxRestarts, window)) ->
 for KeyValue(_, child) in children do
 child.Post(SystemMessage Restart)

 | Some (RestForOne(maxRestarts, window)) ->
 // Restart from failed child onward
 let childList = children |> Seq.toList
 let idx = childList |> List.findIndex (fun kv -> kv.Key = childId)
 for kv in childList |> List.skip idx do
 kv.Value.Post(SystemMessage Restart)

 | None ->
 // No supervision strategy - escalate
 match supervisor with
 | Some sup -> sup.HandleFailure(selfId, exn)
 | None -> raise exn
 }

 loop()
 )

 let selfId = ActorId.generate()

 member _.Id = selfId
 member _.Tell(msg: 'Message) = mailbox.Post(UserMessage msg)
 member _.State = state

 /// Acquire a tracked resource
 member _.AcquireResource<'R when 'R :> IDisposable>(factory: unit -> 'R) =
 let resource = factory()
 let id = ResourceId.generate()
 resources.TryAdd(id, resource :> IDisposable) |> ignore
 resource

 interface IDisposable with
 member _.Dispose() =
 mailbox.Post(SystemMessage (Terminate "disposed"))

and ActorMessage<'T> =
 | UserMessage of 'T
 | SystemMessage of SystemMessage

and SystemMessage =
 | Terminate of reason: string
 | Restart
 | ChildFailed of ActorId * exn

The message types distinguish between user-originated messages and system-originated messages. User messages carry the application-defined payload type. System messages control the actor lifecycle: terminate, restart, or notify of child failure. This separation allows the message loop to handle lifecycle concerns without polluting the application message space.

Supervision Hierarchy

Supervision in the .NET implementation follows the same patterns as Fidelity and Conclave. A supervisor tracks its children, monitors for failures, and applies restart policies. For .NET we added explicit restart counting with time windows. If a child fails too frequently, the supervisor stops attempting recovery and either terminates the child permanently or escalates the failure.

The restart window prevents a perpetually failing child from consuming resources indefinitely. If five failures occur within one minute, and the policy allows only three restarts per minute, the supervisor stops the child after the third restart. The window then slides forward, eventually allowing new restart attempts if the underlying issue resolves.

/// .NET Supervisor implementation
type DotNetSupervisor(strategy: SupervisionStrategy) =
 let children = ConcurrentDictionary<ActorId, DotNetActorRef * ActorSpec>()
 let restartCounts = ConcurrentDictionary<ActorId, int * DateTime>()

 interface ISupervisor with
 member this.HandleFailure(childId: ActorId, exn: exn) =
 let (maxRestarts, window) =
 match strategy with
 | OneForOne(m, w) -> (m, w)
 | AllForOne(m, w) -> (m, w)
 | RestForOne(m, w) -> (m, w)

 // Check restart limits
 let (count, windowStart) =
 restartCounts.GetOrAdd(childId, fun _ -> (0, DateTime.UtcNow))

 if DateTime.UtcNow - windowStart > window then
 // Reset window
 restartCounts.[childId] <- (1, DateTime.UtcNow)
 this.ApplyStrategy(childId, exn)
 elif count < maxRestarts then
 restartCounts.[childId] <- (count + 1, windowStart)
 this.ApplyStrategy(childId, exn)
 else
 // Max restarts exceeded - terminate
 match children.TryGetValue(childId) with
 | true, (actor, _) ->
 actor.Post(SystemMessage (Terminate "max restarts exceeded"))
 | false, _ -> ()

 member _.Supervise(id: ActorId, actor: DotNetActorRef, spec: ActorSpec) =
 children.TryAdd(id, (actor, spec)) |> ignore

 member private this.ApplyStrategy(failedId: ActorId, exn: exn) =
 match strategy with
 | OneForOne _ ->
 match children.TryGetValue(failedId) with
 | true, (actor, _) -> actor.Post(SystemMessage Restart)
 | false, _ -> ()

 | AllForOne _ ->
 for KeyValue(_, (actor, _)) in children do
 actor.Post(SystemMessage Restart)

 | RestForOne _ ->
 let childList = children |> Seq.toList
 let idx =
 childList
 |> List.tryFindIndex (fun kv -> kv.Key = failedId)
 |> Option.defaultValue 0
 for kv in childList |> List.skip idx do
 let (actor, _) = kv.Value
 actor.Post(SystemMessage Restart)

 /// Spawn a child actor under this supervisor
 member this.Spawn<'State, 'Message when 'Message :> IActorMessage>
 (behavior: ActorBehavior<'State, 'Message>, ?config: ActorConfig) =

 let cfg = config |> Option.defaultValue ActorConfig.defaults
 let actor = new DotNetActor<'State, 'Message>(
 behavior, cfg, None, Some (this :> ISupervisor))

 children.TryAdd(actor.Id, (actor :> DotNetActorRef, {
 Behavior = box behavior
 Supervision = cfg.Supervision
 Capabilities = Set.empty
 Name = None
 })) |> ignore

 actor

The Spawn method demonstrates how the supervisor integrates with the actor system. When spawning a child, the supervisor creates the actor with itself as the supervisor reference, adds the child to its tracking collection, and returns a reference that callers can use to send messages. The child actor, upon failure, will call back to this supervisor’s HandleFailure method.

Transport Options

The .NET implementation supports multiple transport mechanisms, selected based on deployment requirements. In-process transport uses thread-safe queues for actors within the same application domain. TCP transport enables distribution across machines. Named pipe transport offers a middle ground for inter-process communication on a single host.

Each transport implements the same ITransport interface, making the choice transparent to actor code. An actor system configured with TCP transport sends messages over the network; the same system configured with in-process transport keeps everything in memory. The actor behaviors remain unchanged.

/// Transport abstraction for .NET actors
type ITransport =
 abstract member Send: ActorId -> byte[] -> Async<unit>
 abstract member Receive: ActorId -> Async<byte[]>
 abstract member Connect: ActorId -> Async<IActorChannel>

/// In-process transport (same AppDomain)
type InProcessTransport() =
 let channels = ConcurrentDictionary<ActorId * ActorId, BlockingCollection<byte[]>>()

 interface ITransport with
 member _.Send targetId data = async {
 let selfId = ActorContext.currentId()
 let key = (selfId, targetId)
 let channel = channels.GetOrAdd(key, fun _ -> BlockingCollection<byte[]>())
 channel.Add(data)
 }
 member _.Receive sourceId = async {
 let selfId = ActorContext.currentId()
 let key = (sourceId, selfId)
 let channel = channels.GetOrAdd(key, fun _ -> BlockingCollection<byte[]>())
 return channel.Take()
 }
 member _.Connect targetId = async {
 return InProcessChannel(targetId) :> IActorChannel
 }

/// TCP transport for distributed .NET actors
type TcpTransport(bindAddress: IPEndPoint) =
 let listener = TcpListener(bindAddress)
 let connections = ConcurrentDictionary<ActorId, TcpClient>()

 interface ITransport with
 member _.Send targetId data = async {
 let! client = getOrConnect targetId
 let stream = client.GetStream()
 // BAREWire length-prefixed frame
 let length = BitConverter.GetBytes(data.Length)
 do! stream.AsyncWrite(length, 0, 4)
 do! stream.AsyncWrite(data, 0, data.Length)
 }
 member _.Receive sourceId = async {
 let! client = getOrConnect sourceId
 let stream = client.GetStream()
 let lengthBuf = Array.zeroCreate 4
 let! _ = stream.AsyncRead(lengthBuf, 0, 4)
 let length = BitConverter.ToInt32(lengthBuf, 0)
 let data = Array.zeroCreate length
 let! _ = stream.AsyncRead(data, 0, length)
 return data
 }
 member _.Connect targetId = async {
 let! client = getOrConnect targetId
 return TcpChannel(client) :> IActorChannel
 }

/// Named pipe transport for local IPC
type NamedPipeTransport(pipeName: string) =
 interface ITransport with
 // Similar pattern using NamedPipeServerStream/NamedPipeClientStream
 member _.Send targetId data = async { (* ... *) }
 member _.Receive sourceId = async { (* ... *) }
 member _.Connect targetId = async { (* ... *) }

The TCP transport uses length-prefixed framing for BAREWire messages. Each frame begins with a 4-byte integer indicating the payload length, followed by the payload itself. This simple framing protocol allows the receiver to read exactly one message at a time from the TCP stream, avoiding the complexity of delimiter-based parsing.

Named pipes serve scenarios where multiple processes on the same machine need to communicate. They offer better performance than TCP loopback (avoiding network stack overhead) while providing the same stream semantics. Windows and Unix implement named pipes differently, but the .NET abstraction presents a uniform interface.

System Builder

Constructing an actor system involves multiple configuration decisions: which transport to use, whether to collect metrics, what default supervision strategy to apply. A fluent builder API consolidates these decisions into a readable chain of method calls. The builder pattern also provides sensible defaults, reducing boilerplate for simple cases while allowing full customization when needed.

/// Build complete actor systems with scaffolding
type ActorSystemBuilder() =
 let mutable transport: ITransport = InProcessTransport() :> ITransport
 let mutable metrics: IMetricsProvider option = None
 let mutable defaultSupervision = OneForOne(3, TimeSpan.FromMinutes(1.0))

 member this.WithTransport(t: ITransport) =
 transport <- t
 this

 member this.WithMetrics(m: IMetricsProvider) =
 metrics <- Some m
 this

 member this.WithDefaultSupervision(s: SupervisionStrategy) =
 defaultSupervision <- s
 this

 member this.Build() =
 ActorSystem(transport, metrics, defaultSupervision)

and ActorSystem(transport, metrics, defaultSupervision) =
 let registry = ConcurrentDictionary<ActorId, obj>()
 let rootSupervisor = DotNetSupervisor(defaultSupervision)

 /// Spawn a top-level actor
 member this.Spawn<'State, 'Message when 'Message :> IActorMessage>
 (behavior: ActorBehavior<'State, 'Message>, ?config: ActorConfig) =

 let cfg = config |> Option.defaultValue ActorConfig.defaults
 let actorMetrics =
 metrics |> Option.map (fun m -> m.CreateActorMetrics(typeof<'Message>.Name))

 let actor = new DotNetActor<'State, 'Message>(
 behavior, cfg, actorMetrics, Some (rootSupervisor :> ISupervisor))

 registry.TryAdd(actor.Id, actor) |> ignore
 rootSupervisor.Supervise(actor.Id, actor, cfg)

 ActorRef<'Message>(actor.Id)

 /// Create a supervisor for a subtree
 member this.CreateSupervisor(?strategy: SupervisionStrategy) =
 let strat = strategy |> Option.defaultValue defaultSupervision
 DotNetSupervisor(strat)

/// Convenience functions
module ActorSystem =
 let create() = ActorSystemBuilder()

 let spawn behavior (system: ActorSystem) =
 system.Spawn(behavior)

 let spawnWith behavior config (system: ActorSystem) =
 system.Spawn(behavior, config)

The companion module provides functional-style wrappers around the object-oriented builder. These wrappers enable pipeline composition with the |> operator, a common Clef idiom that reads left-to-right.

Usage Example

The following example directly addresses questions about building production-ready actors. Developers ask for patterns that handle errors, manage resources, and provide observability without scattering boilerplate throughout handler code.

The answer combines a behavior definition with the ActorSystem builder. The behavior contains business logic; the system provides operational scaffolding. Error policies, backpressure limits, and supervision strategies are declarative configuration, not imperative code mixed into handlers.

open FSharp.Actors

// Define messages
type CounterMsg =
 | Increment
 | Decrement
 | GetCount of AsyncReplyChannel<int>

// Define behavior
let counterBehavior = {
 Init = fun () -> 0
 Receive = fun count msg -> actor {
 match msg with
 | Increment -> return count + 1
 | Decrement -> return count - 1
 | GetCount reply ->
 reply.Reply(count)
 return count
 }
 Lifecycle = fun count _ -> actor { return count }
}

// Build system with full scaffolding
let system =
 ActorSystem.create()
 .WithTransport(InProcessTransport())
 .WithMetrics(PrometheusMetrics())
 .WithDefaultSupervision(OneForOne(5, TimeSpan.FromMinutes(1.0)))
 .Build()

// Spawn actors
let counter1 = system.Spawn(counterBehavior)
let counter2 = system.Spawn(counterBehavior, {
 ActorConfig.defaults with
 ErrorPolicy = Some (RestartWith 0)
 BackpressurePolicy = Some (DropOldest 1000)
})

// Use actors
counter1.Tell(Increment)
counter1.Tell(Increment)

// Ask pattern
let! count = counter1.Ask(GetCount, TimeSpan.FromSeconds(5.0))
printfn "Count: %d" count

// Supervision example
let supervisor = system.CreateSupervisor(AllForOne(3, TimeSpan.FromMinutes(1.0)))

let worker1 = supervisor.Spawn(workerBehavior)
let worker2 = supervisor.Spawn(workerBehavior)
// If worker1 fails, both restart

Migration Path

For existing .NET applications using raw MailboxProcessor:

// Before: Raw MailboxProcessor with manual error handling
let oldActor = MailboxProcessor.Start(fun inbox ->
 let rec loop state = async {
 try
 let! msg = inbox.Receive()
 match msg with
 | Increment -> return! loop (state + 1)
 | Decrement -> return! loop (state - 1)
 with exn ->
 // Manual error handling scattered everywhere
 printfn "Error: %A" exn
 return! loop state
 }
 loop 0
)

// After: Scaffolded actor with declarative policies
let newActor =
 system.Spawn(counterBehavior, {
 ActorConfig.defaults with
 ErrorPolicy = Some Resume
 BackpressurePolicy = Some (DropOldest 1000)
 })

Existing .NET code migrates to the scaffolded model, gaining supervision, error policies, and observability without rewriting business logic. While these patterns are inspired by the intrinsic actor abstraction, they stand alone for .NET-only deployments.

Transport Evolution: Media over QUIC

WebSockets provide the foundation for Conclave actor communication, but they have inherent limitations for high-throughput, cross-system messaging. Cloudflare’s Media over QUIC (MoQ) relay infrastructure addresses these limitations and opens significant opportunities for actor system interconnection.

Why MoQ Matters Beyond Media

Despite the name, MoQ is not limited to audio/video. The protocol provides capabilities that benefit any dense messaging stream:

Capability	WebSocket Limitation	MoQ Advantage
Head-of-line blocking	Single ordered stream	Multiplexed independent streams
Connection migration	Drops on network change	QUIC handles seamlessly
Priority/ordering	Application must manage	Protocol-level support
Pub/sub topology	Point-to-point	Native track-based fanout
Congestion control	TCP-based	QUIC adaptive

For actor systems with dense message flows, these differences compound. A supervision hierarchy with dozens of actors exchanging frequent state updates suffers under WebSocket’s single-stream model. MoQ’s multiplexed streams eliminate the head-of-line blocking that creates latency spikes.

Cross-System Actor Topology

MoQ relay becomes the backbone for connecting actor systems across deployment boundaries:

%%{init: {'theme': 'neutral'}}%%
graph TB
subgraph "Fidelity System"
F_SUP[Prospero<br/>Supervisor]
F_A[Olivier<br/>Actor A]
F_B[Olivier<br/>Actor B]
F_SUP --- F_A
F_SUP --- F_B
end
subgraph "Cloudflare MoQ Relay"
RELAY[MoQ Relay<br/>320+ Locations]
end
subgraph "Conclave System"
C_SUP[Supervisor<br/>Durable Object]
C_A[Worker<br/>Actor A]
C_B[Worker<br/>Actor B]
C_SUP --- C_A
C_SUP --- C_B
end
subgraph ".NET System"
N_SUP[DotNetSupervisor]
N_A[DotNetActor A]
N_B[DotNetActor B]
N_SUP --- N_A
N_SUP --- N_B
end
F_SUP ===|MoQ Track| RELAY
RELAY ===|MoQ Track| C_SUP
RELAY ===|MoQ Track| N_SUP

Each system maintains its internal transport (IPC for Fidelity, WebSocket for Conclave, TCP/pipes for .NET), while MoQ relay handles cross-system communication. BAREWire remains the protocol; MoQ becomes another transport option.

Track-Based Actor Addressing

MoQ’s track model maps naturally to actor addressing:

// Olivier.Transport.MoQ
namespace Olivier.Transport

/// MoQ track corresponds to an actor's message stream
type MoQActorTrack = {
 Namespace: string // System identifier: "fidelity", "conclave", "dotnet"
 ActorId: ActorId // Target actor
 TrackName: string // Derived from actor address
}

/// MoQ transport implementation
type MoQTransport(relayUrl: string) =
 let session = MoQSession(relayUrl)
 let subscriptions = ConcurrentDictionary<ActorId, MoQSubscription>()

 interface ITransport with
 member _.Send(targetId: ActorId) (data: byte[]) = async {
 let track = deriveTrack targetId

 // Publish BAREWire frame to actor's track
 let! publisher = session.GetOrCreatePublisher(track)
 do! publisher.PublishObject {
 GroupId = nextGroupId()
 ObjectId = nextObjectId()
 Payload = data // BAREWire-encoded message
 Priority = Priority.Normal
 }
 }

 member _.Subscribe(sourceId: ActorId) (handler: byte[] -> Async<unit>) = async {
 let track = deriveTrack sourceId

 let! subscription = session.Subscribe(track)
 subscriptions.TryAdd(sourceId, subscription) |> ignore

 // Receive loop
 subscription.OnObject.Add(fun obj ->
 handler obj.Payload |> Async.Start
 )
 }

 member _.Connect(targetId: ActorId) = async {
 let track = deriveTrack targetId
 return MoQChannel(session, track) :> IActorChannel
 }

/// Derive MoQ track from actor address
let deriveTrack (actorId: ActorId) : MoQActorTrack =
 let parts = actorId.ToString().Split('/')
 {
 Namespace = parts.[0] // System: fidelity/conclave/dotnet
 ActorId = actorId
 TrackName = $"actor/{parts.[1]}/{parts.[2]}" // actor/supervisor/instance
 }

The track derivation function maps actor addresses to MoQ track identifiers. The namespace component identifies which system originated the actor (Fidelity, Conclave, or .NET), enabling cross-system routing. The track name encodes the actor hierarchy, allowing MoQ relays to optimize routing decisions.

Multiplexed Actor Streams

Head-of-line blocking is a limitation of single-stream protocols. When multiple logical channels share one TCP connection (as with HTTP/2 or WebSocket), a delay in processing one message blocks all subsequent messages regardless of their destination. MoQ eliminates this problem by mapping each logical channel to an independent QUIC stream.

For actor systems, this translates directly: each actor-to-actor relationship gets its own stream. A supervisor monitoring 50 child actors receives heartbeats on 50 independent streams. A slow heartbeat from one child does not delay processing of heartbeats from other children. This independence is particularly valuable for supervision, where timely failure detection prevents cascading outages.

/// Supervisor using MoQ for high-throughput child monitoring
type MoQSupervisor(strategy: SupervisionStrategy, transport: MoQTransport) =
 let children = ConcurrentDictionary<ActorId, MoQSubscription>()

 member this.SuperviseChild(childId: ActorId) = async {
 // Each child gets its own MoQ track subscription
 // No head-of-line blocking between children
 do! transport.Subscribe childId (fun heartbeat ->
 async {
 let msg = BAREWire.decode<HeartbeatMessage> heartbeat
 match msg with
 | Alive timestamp ->
 this.RecordHeartbeat(childId, timestamp)
 | Failed exn ->
 this.HandleFailure(childId, exn)
 }
 )
 }

 member this.BroadcastToChildren(msg: byte[]) = async {
 // Parallel publish to all child tracks
 // MoQ relay handles fanout efficiently
 let! _ =
 children.Keys
 |> Seq.map (fun childId -> transport.Send childId msg)
 |> Async.Parallel
 ()
 }

The broadcast operation demonstrates another MoQ advantage: efficient fanout. When a supervisor sends a configuration update to all children, it publishes once to its track, and the MoQ relay handles distribution. Children subscribed to that track receive the update without the supervisor maintaining individual connections to each child. This pub/sub model reduces connection count and simplifies topology management.

Priority and Ordering

Traditional actor systems handle priority at the application level, typically through multiple mailboxes or priority queues within a single mailbox. MoQ provides priority at the transport level, allowing urgent messages to preempt less critical traffic during congestion.

Consider a system under load: normal business messages accumulate, but a supervisor needs to send a shutdown command immediately. With application-level priority, the command enters the same queue as business messages and waits its turn. With MoQ priority, the transport layer ensures the command transmits ahead of queued traffic. The difference matters in failure scenarios where timely intervention prevents data loss or cascade failures.

type ActorPriority =
 | Supervision // Highest: supervisor commands, failure notifications
 | Control // High: lifecycle events, configuration
 | Normal // Standard: business messages
 | Bulk // Low: batch data, logs
 | Background // Lowest: metrics, telemetry

/// Priority-aware message sending
let sendWithPriority (transport: MoQTransport) priority targetId msg = async {
 let moqPriority =
 match priority with
 | Supervision -> Priority.Urgent
 | Control -> Priority.High
 | Normal -> Priority.Normal
 | Bulk -> Priority.Low
 | Background -> Priority.Lowest

 do! transport.SendWithPriority targetId msg moqPriority
}

// Supervision messages preempt normal traffic
do! sendWithPriority transport Supervision childId (Terminate "shutdown")

The priority mapping is explicit and deterministic. Supervision messages always receive urgent priority. Control messages (lifecycle events, configuration changes) receive high priority. Normal business traffic flows at standard priority. Bulk operations (batch data transfers, log shipping) and background tasks (metrics collection, health checks) receive progressively lower priority. This hierarchy ensures that operational concerns never starve due to application traffic.

Transport Selection Strategy

Not every message needs MoQ. Intra-system communication within a single Fidelity deployment uses IPC, which offers lower latency than any network protocol. Simple cross-system interactions with low message rates work fine over WebSocket. MoQ adds value for dense streams, real-time requirements, and scenarios where multiplexing benefits outweigh protocol overhead.

The transport selector encapsulates this decision logic. Given a message pattern (intra-system, cross-system light, cross-system heavy), it returns the appropriate transport. The selection happens once when establishing a communication path; subsequent messages flow over the selected transport without repeated decision overhead.

type TransportSelector = {
 /// Within same system
 IntraSystem: ITransport
 /// Cross-system, low throughput
 InterSystemLight: ITransport
 /// Cross-system, high throughput
 InterSystemHeavy: ITransport
}

let defaultSelector = {
 IntraSystem =
 match currentSystem with
 | Fidelity -> IPCTransport()
 | Conclave -> WebSocketTransport()
 | DotNet -> InProcessTransport()

 InterSystemLight = WebSocketTransport() // Simple cases
 InterSystemHeavy = MoQTransport(moqRelayUrl) // Dense streams
}

/// Route based on message pattern
let selectTransport (pattern: MessagePattern) (selector: TransportSelector) =
 match pattern with
 | IntraSystem -> selector.IntraSystem
 | CrossSystem Low -> selector.InterSystemLight
 | CrossSystem High -> selector.InterSystemHeavy
 | CrossSystem (Streaming _) -> selector.InterSystemHeavy

The default selector illustrates practical choices. Within a system, use the fastest available option. For lightweight cross-system traffic (occasional control messages, infrequent state synchronization), WebSocket suffices. For heavy cross-system traffic (continuous telemetry, real-time data feeds, dense actor interactions), engage MoQ.

Preparing for MoQ Availability

Cloudflare’s MoQ relay infrastructure is maturing. The transport abstraction allows systems to prepare now, using WebSocket as a fallback while MoQ capabilities emerge. The compatibility transport wraps both options and selects based on availability and message characteristics.

// Current: WebSocket fallback with MoQ-like interface
type MoQCompatTransport(wsUrl: string, moqRelayUrl: string option) =
 let ws = WebSocketTransport(wsUrl)
 let moq = moqRelayUrl |> Option.map MoQTransport

 interface ITransport with
 member _.Send targetId data = async {
 match moq with
 | Some m when isHighThroughput targetId ->
 // Use MoQ when available and beneficial
 do! m.Send targetId data
 | _ ->
 // Fall back to WebSocket
 do! ws.Send targetId data
 }
 // ...

When Cloudflare publishes MoQ relay APIs, the transport implementation updates without changing actor code. The abstraction boundary is clean: actors speak BAREWire, transports move bytes, and the selection logic determines which transport handles each communication path.

Real-Time AI Audio Pipeline

The motivation for tracking MoQ development extends beyond general messaging efficiency. Real-time AI audio services present particularly demanding requirements: continuous bidirectional streams, strict latency budgets, and multiple concurrent processing stages.

Consider a voice agent pipeline where audio flows from client through multiple processing stages and returns as synthesized speech. Each stage is an actor: ingestion normalizes incoming audio, ASR converts speech to text, the language model generates a response, and TTS synthesizes the reply. Audio frames arrive at approximately 50 packets per second (20ms frame intervals). Any delay compounds across stages, degrading perceived responsiveness.

WebSocket’s head-of-line blocking creates jitter in this scenario. If the ASR stage delays processing one frame, subsequent frames queue behind it even though they’re destined for independent processing paths. MoQ’s multiplexed streams eliminate this interference: each stage-to-stage connection operates independently, maintaining consistent latency under varying load.

type VoiceAgentPipeline() =
 let ingestActor = spawn ingestBehavior
 let asrActor = spawn asrBehavior
 let llmActor = spawn llmBehavior
 let ttsActor = spawn ttsBehavior

 // Audio streams use MoQ for low-latency delivery
 let audioTransport = MoQTransport(moqRelayUrl)

 // Control messages can use WebSocket
 let controlTransport = WebSocketTransport(wsUrl)

 member _.ProcessAudioFrame(frame: AudioFrame) = async {
 // MoQ stream: low latency, independent of control traffic
 do! audioTransport.Send ingestActor.Id (BAREWire.encode frame)
 }

 member _.UpdateConfiguration(config: AgentConfig) = async {
 // WebSocket: lower throughput, shared with other control
 do! controlTransport.Send llmActor.Id (BAREWire.encode config)
 }

This architecture handles dense audio streams alongside sparse control messages, each on the appropriate transport, all encoding with BAREWire protocol. The audio path uses MoQ for its multiplexing and priority capabilities. The control path uses WebSocket because control messages are infrequent and latency-tolerant. Both paths remain compatible because the protocol layer is uniform; only transport selection differs.

The pipeline also illustrates actor model benefits for AI workloads, a theme explored in Actors Take Center Stage. Each stage scales independently. If ASR becomes a bottleneck, spawn additional ASR actors and distribute incoming frames. If the LLM stage requires GPU acceleration, deploy that actor to hardware-equipped nodes while keeping other stages on standard compute. The actor boundaries create natural scaling points without requiring monolithic system redesign.

Conclusion

Considering protocol separately from transport enables a clear actor architecture across different runtimes. BAREWire provides the common language; the transport adapts to each platform’s strengths. By defining actor behaviors as pure functions that produce effects, and letting target-specific runtimes interpret those effects, we achieve a consistent developer experience without sacrificing platform-appropriate optimizations.

Developers write actor logic once. The same actor { } computation expression is designed to compile to our Olivier actors communicating over IPC in Fidelity, to Durable Object actors communicating over WebSocket in Conclave, or to MailboxProcessor actors communicating over TCP in standard .NET. All speak BAREWire. Business logic remains portable; transport concerns stay in the runtime layer.

This architectural direction is meant to carry several practical benefits: shared testing infrastructure for actor behaviors, gradual migration paths between deployment targets, reduced cognitive overhead for teams working across both frameworks, and cross-target actor communication through transport bridges. As MoQ matures, the transport selection layer can incorporate it without disturbing actor code or protocol definitions. The work continues toward that separation of protocol and transport, and I expect the actor abstraction to keep earning its place as more of the constellation comes online.

Getting the Signal with BAREWire

Wed, 03 Dec 2025 00:00:00 +0000

Reactive programming has become essential infrastructure for modern applications. From browser interfaces responding to user input to distributed systems coordinating state across nodes, the ability to propagate changes through a dependency graph underpins countless software architectures. Yet the dominant patterns for implementing reactivity carry significant cognitive and runtime overhead. The subscription model that pervades .NET, RxJS, and similar frameworks demands explicit lifecycle management that clutters application code and creates entire categories of resource leaks.

Our Fidelity Framework takes a different approach. By combining BAREWire’s zero-copy memory architecture with a signal-based reactive model, we aim for reactive semantics without the subscription ceremony. This is an architectural decision that aligns with our actor-oriented design philosophy and carries consistent reactive patterns across native compilation, browser deployment, and managed runtime targets.

The Subscription Problem

Consider a typical reactive scenario in .NET using the IObservable<T> pattern:

type SensorMonitor() =
 let temperatureSubject = new BehaviorSubject<float>(20.0)
 let pressureSubject = new BehaviorSubject<float>(1013.25)
 let subscriptions = new CompositeDisposable()

 let derivedAlert =
 temperatureSubject
 .CombineLatest(pressureSubject, fun temp pressure ->
 temp > 100.0 && pressure > 1050.0)
 .DistinctUntilChanged()

 member this.OnTemperatureReading(value: float) =
 temperatureSubject.OnNext(value)

 member this.OnPressureReading(value: float) =
 pressureSubject.OnNext(value)

 member this.SubscribeToAlerts(handler: bool -> unit) =
 let subscription = derivedAlert.Subscribe(handler)
 subscriptions.Add(subscription)
 subscription

 interface IDisposable with
 member this.Dispose() =
 subscriptions.Dispose()
 temperatureSubject.Dispose()
 pressureSubject.Dispose()

This code exhibits several characteristics that compound into significant maintenance burden:

Explicit subscription handles: Every Subscribe call returns an IDisposable that the caller must retain and eventually dispose. Forgetting to dispose leaks resources. Disposing twice throws exceptions. The subscription handle becomes a first-class concern that pollutes API boundaries.

Composite disposal management: Tracking multiple subscriptions requires container types like CompositeDisposable. These containers themselves require lifecycle management, creating nested disposal hierarchies.

Subject ownership ambiguity: Who owns the BehaviorSubject instances? When should they complete? The disposal pattern forces explicit answers to questions that often have no clean solution.

Cold versus hot confusion: Does subscribing to derivedAlert trigger computation, or is it already running? The answer depends on implementation details hidden behind the IObservable interface.

These aren’t incidental complexities; they’re intrinsic to the subscription model. The pattern treats reactive relationships as runtime entities that must be explicitly created and destroyed, imposing a resource management discipline on what should be declarative data flow.

What “Subscription-Free” Actually Means

SolidJS popularized the term “subscription-free” reactivity, but the phrase requires careful interpretation. SolidJS absolutely maintains dependency relationships internally. Signals know their dependents; computations know their sources. The graph exists.

What SolidJS eliminates is the explicit subscription API. You simply read a signal’s value. The act of reading, when performed within a reactive context, establishes the dependency. When the signal changes, the computation re-executes. No subscription handle. No disposal requirement. No ceremony.

The mechanism relies on tracking context. When a reactive computation begins executing, the runtime establishes a context that records signal reads. Each signal checks for this context when accessed. If present, the signal registers the current computation as a dependent. When the computation completes, the context captures the full dependency set.

// SolidJS pseudocode illustrating the mechanism
let currentObserver = null;

function createSignal(initialValue) {
 let value = initialValue;
 const dependents = new Set();

 const read = () => {
 if (currentObserver) {
 dependents.add(currentObserver);
 currentObserver.sources.add({ signal: read, dependents });
 }
 return value;
 };

 const write = (newValue) => {
 value = newValue;
 for (const dep of dependents) {
 dep.execute();
 }
 };

 return [read, write];
}

function createEffect(fn) {
 const effect = {
 sources: new Set(),
 execute() {
 // Clear previous dependencies
 for (const source of this.sources) {
 source.dependents.delete(this);
 }
 this.sources.clear();

 // Execute with tracking
 const prevObserver = currentObserver;
 currentObserver = this;
 try {
 fn();
 } finally {
 currentObserver = prevObserver;
 }
 }
 };

 effect.execute();
 return effect;
}

The currentObserver global variable serves as the tracking context. Signal reads check it; effect execution sets it. This works because JavaScript is single-threaded. The pattern cannot race.

Cleanup happens automatically through re-execution. When an effect runs again, it first clears its previous dependencies, then re-establishes them based on which signals it actually reads during the new execution. If a conditional branch changes which signals matter, the dependency graph updates accordingly. No explicit unsubscription required.

%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph Execution["Effect Execution"]
A[Effect starts] --> B[Clear previous dependencies]
B --> C[Set currentObserver = this]
C --> D[Execute effect function]
D --> E[Signal read checks currentObserver]
E --> F[Register dependency if context exists]
F --> G[Restore previous observer]
end
subgraph Propagation["Change Propagation"]
H[Signal write] --> I[Notify all dependents]
I --> J[Each dependent re-executes]
J --> K[Dependencies re-established]
end
G --> H

Departing from .NET Conventions

A tension emerges when bringing signal-based reactivity to our Clef language: the .NET heritage Clef descends from carries conventions that assume explicit resource management. The IDisposable pattern, use bindings, and finalizer semantics pervade idiomatic .NET code. These patterns exist because the CLR’s garbage collector cannot guarantee deterministic cleanup; developers must explicitly mark resource boundaries.

Our Fidelity Framework takes deliberate leave from these conventions. As explored in our discussion of RAII in Olivier and Prospero, our Composer compiler is designed to perform scope analysis during IR lowering and insert cleanup code automatically. Developers write signal operations; the compiler determines where cleanup belongs based on actor lifecycle boundaries and continuation points.

This reflects a position we hold: resource management is a compiler concern, not a developer burden. The same coeffect analysis that tracks async boundaries and memory access patterns also tracks resource lifetimes. When the compiler knows that a signal exists within an actor’s arena, it knows when that arena will be released. No Dispose call needed; no IDisposable interface required.

// .NET convention: explicit disposal
type SensorProcessor() =
 let subscription = sensor.Subscribe(handler)

 interface IDisposable with
 member this.Dispose() = subscription.Dispose()

// Fidelity approach: compiler-managed cleanup
let processSensor (sensor: Signal<SensorReading>) =
 let derived = memo (fun () ->
 let reading = Signal.get sensor
 transformReading reading)

 effect (fun () ->
 let value = Signal.get derived
 updateDisplay value)
 // cleanup inserted by Composer during IR lowering, scoped to the enclosing actor

The Fable target for Fidelity.CloudEdge follows a similar philosophy, delegating to SolidJS’s internal cleanup mechanisms. The .NET interop layer necessarily bridges back to IDisposable conventions when interfacing with existing .NET code, but pure Fidelity code intends to remain free of explicit disposal ceremony.

This departure may initially feel unfamiliar to developers accustomed to .NET patterns. The use binding becomes less prevalent. The IDisposable interface appears only at system boundaries. Resource cleanup would be handled by the same compilation infrastructure that manages arena allocation and actor termination. The code that developers write focuses on the reactive logic; the compiler handles the rest.

Designing Signals for Clef

Bringing signal-based reactivity to Clef requires addressing the multi-threaded reality of native execution. The global currentObserver pattern assumes single-threaded semantics that JavaScript provides but native code does not. Three architectural approaches present themselves.

Thread-Local Tracking Context

The most direct translation uses thread-local storage to maintain per-thread tracking contexts:

module Fidelity.Reactive

[<ThreadStatic>]
let mutable private currentScope: ReactiveScope voption = ValueNone

type ReactiveScope = {
 mutable Dependencies: ResizeArray<ISignalBase>
 Computation: IComputation
}

and ISignalBase =
 abstract AddDependent: IComputation -> unit
 abstract RemoveDependent: IComputation -> unit

and IComputation =
 abstract Invalidate: unit -> unit
 abstract Execute: unit -> unit

Signal reads check the thread-local context:

type Signal<'T> = {
 mutable Value: 'T
 Dependents: ResizeArray<WeakReference<IComputation>>
 Lock: SpinLock
}

module Signal =
 let inline get (signal: Signal<'T>) : 'T =
 match currentScope with
 | ValueSome scope ->
 scope.Dependencies.Add(signal)
 let lockTaken = ref false
 signal.Lock.Enter(lockTaken)
 try
 signal.Dependents.Add(WeakReference(scope.Computation))
 finally
 if lockTaken.Value then signal.Lock.Exit()
 | ValueNone -> ()
 signal.Value

 let inline set (signal: Signal<'T>) (value: 'T) : unit =
 signal.Value <- value
 let lockTaken = ref false
 signal.Lock.Enter(lockTaken)
 let deps =
 try
 signal.Dependents |> Seq.choose (fun wr ->
 match wr.TryGetTarget() with
 | true, comp -> Some comp
 | false, _ -> None)
 |> Seq.toArray
 finally
 if lockTaken.Value then signal.Lock.Exit()
 for dep in deps do
 dep.Invalidate()

This approach mirrors SolidJS closely while handling multi-threaded access. The WeakReference wrapper prevents memory leaks when computations are abandoned; dead references are cleaned up lazily during iteration.

Explicit Context via Computation Expressions

An alternative approach makes the tracking context explicit through Clef’s computation expression syntax:

type Reactive<'T> = ReactiveScope -> 'T

type ReactiveBuilder() =
 member _.Bind(signal: Signal<'T>, f: 'T -> Reactive<'U>) : Reactive<'U> =
 fun scope ->
 scope.Dependencies.Add(signal)
 signal.Dependents.Add(WeakReference(scope.Computation))
 let value = signal.Value
 f value scope

 member _.Return(x: 'T) : Reactive<'T> =
 fun _ -> x

 member _.ReturnFrom(r: Reactive<'T>) : Reactive<'T> = r

 member _.Zero() : Reactive<unit> =
 fun _ -> ()

let reactive = ReactiveBuilder()

Usage requires let! bindings for tracked reads:

let temperature = Signal.create 20.0
let pressure = Signal.create 1013.25

let alertCondition = reactive {
 let! temp = temperature
 let! press = pressure
 return temp > 100.0 && press > 1050.0
}

This approach is more explicit than thread-local tracking. Dependencies are visible in the code structure. However, it requires restructuring existing code to use the computation expression syntax, which may feel heavy for simple reactive bindings.

Hybrid Scope Markers

Our current approach leans toward thread-local context with explicit scope boundaries:

module Fidelity.Reactive

[<ThreadStatic>]
let mutable private currentScope: ReactiveScope voption = ValueNone

/// Enter a reactive scope where signal reads are tracked
let track (f: unit -> 'T) : TrackedResult<'T> =
 let scope = {
 Dependencies = ResizeArray()
 Computation = Unchecked.defaultof<_>
 }
 let prevScope = currentScope
 currentScope <- ValueSome scope
 try
 let result = f()
 { Result = result; Dependencies = scope.Dependencies |> Seq.toList }
 finally
 currentScope <- prevScope

/// Create a memoized computation that re-executes when dependencies change
let memo (f: unit -> 'T) : Signal<'T> =
 let mutable cached = Unchecked.defaultof<'T>
 let mutable valid = false
 let resultSignal = Signal.create cached

 let computation = {
 new IComputation with
 member _.Invalidate() =
 valid <- false
 computation.Execute()

 member this.Execute() =
 let tracked = track f
 cached <- tracked.Result
 valid <- true
 Signal.set resultSignal cached
 }

 computation.Execute()
 resultSignal

/// Create an effect that runs when dependencies change
let effect (f: unit -> unit) : unit =
 let mutable currentDeps: ISignalBase list = []

 let computation = {
 new IComputation with
 member _.Invalidate() =
 computation.Execute()

 member this.Execute() =
 for dep in currentDeps do
 dep.RemoveDependent(this)

 let tracked = track f
 currentDeps <- tracked.Dependencies |> List.map (fun s -> s :> ISignalBase)

 for dep in currentDeps do
 dep.AddDependent(this)
 }

 computation.Execute()
 // Cleanup handled by Composer's scope analysis during IR lowering

This hybrid provides the ergonomics of implicit tracking (signal reads within a tracked scope are automatically captured) while making scope boundaries explicit. On a semantic level, the effect function in Fidelity returns unit, not IDisposable. The Composer compiler’s scope analysis identifies where cleanup should occur and inserts it during IR lowering, eliminating the developer’s burden of managing subscription lifecycles.

Integration with BAREWire: Solving the Byref Problem

The signal model integrates with BAREWire’s zero-copy memory architecture. Traditional reactive systems copy values through the dependency graph. Each signal holds its own copy of the data; each propagation involves allocation and copying. For small values this overhead is negligible. For large buffers, frequently updated data streams, or resource-constrained environments, the overhead dominates.

Our design for BAREWire takes a different, patent-pending approach: signals that reference data in shared buffers without copying. This rests on BAREWire’s solution to the byref problem that has constrained .NET developers for decades. Where .NET’s byref restrictions force defensive copying because references cannot outlive their stack frame, BAREWire separates buffer lifetime from access permissions through capability-based memory management.

Capability-Based Memory Access

The core insight from our byref resolution work applies directly to reactive signals. Traditional .NET code faces a fundamental constraint:

// .NET limitation: byrefs cannot escape their stack frame
let getBiggestElementRef (arr: int[]) =
 let mutable maxIndex = 0
 for i = 1 to arr.Length - 1 do
 if arr.[i] > arr.[maxIndex] then
 maxIndex <- i
 &arr.[maxIndex] // ERROR: Cannot return a byref from this function

BAREWire solves this by creating capabilities that can be passed around safely while the underlying buffer lifetime is managed separately:

// BAREWire approach: capabilities separate lifetime from access
let processLargeData() =
 let buffer = BAREWire.createBuffer<LargeStruct> 1
 let writeCapability = buffer.GetWriteAccess()

 // This capability can be passed to async functions and reactive computations
 let processAsync (capability: WriteCapability<LargeStruct>) = async {
 do! Async.Sleep(100) // State machine on heap is fine
 let struct = capability.GetDirectAccess()
 struct.UpdateInPlace(newValue) // Zero-copy modification
 return capability
 }

This same capability pattern enables reactive signals that hold references into shared memory without the copying that would otherwise be required.

Buffer Signals with Zero-Copy Semantics

Building on BAREWire’s capability system, we define signals whose values live in shared buffers:

module BAREWire.Reactive

/// A signal whose value lives in a BAREWire buffer
[<Struct>]
type BufferSignal<'T> = {
 Buffer: BAREBuffer
 Offset: int
 Layout: BARELayout<'T>
 mutable Version: int64
 Dependents: ResizeArray<WeakReference<IComputation>>
}

/// Reference into a BAREWire buffer (zero-copy read)
[<Struct>]
type BARERef<'T> = {
 Buffer: BAREBuffer
 Offset: int
 Layout: BARELayout<'T>
}

module BufferSignal =
 /// Read the current value (returns a reference, not a copy)
 let inline get (signal: BufferSignal<'T>) : BARERef<'T> =
 match currentScope with
 | ValueSome scope ->
 scope.Dependencies.Add(signal :> obj)
 signal.Dependents.Add(WeakReference(scope.Computation))
 | ValueNone -> ()
 { Buffer = signal.Buffer; Offset = signal.Offset; Layout = signal.Layout }

 /// Update the buffer reference (typically called when new data arrives)
 let inline updateBuffer (signal: BufferSignal<'T>) (newBuffer: BAREBuffer) : unit =
 let oldBuffer = signal.Buffer
 signal.Buffer <- newBuffer
 Interlocked.Increment(&signal.Version) |> ignore

 for depRef in signal.Dependents do
 match depRef.TryGetTarget() with
 | true, comp -> comp.Invalidate()
 | false, _ -> ()

 BufferPool.release oldBuffer

The BARERef<'T> type doesn’t contain the value; it contains coordinates for locating the value within a buffer. Accessing the actual data requires dereferencing through the layout:

module BARERef =
 /// Dereference to get the actual value (performs the read)
 let inline deref (ref: BARERef<'T>) : 'T =
 BARELayout.read ref.Layout ref.Buffer ref.Offset

 /// Get a span view of the underlying memory (for large values)
 let inline asSpan (ref: BARERef<'T>) : ReadOnlySpan<byte> =
 let size = BARELayout.size ref.Layout
 BAREBuffer.slice ref.Buffer ref.Offset size

This design intends to separate the reactive relationship (tracking, dependency propagation) from the actual data access. The signal machinery operates on lightweight reference types. The heavy data stays in BAREWire buffers, accessed only when needed. As discussed in Cache-Conscious Memory Management, BAREWire’s deterministic layouts make this integration possible by ensuring that memory positions are statically known and guaranteed.

Hardware-Enforced Safety

Unlike .NET’s runtime-based memory safety, BAREWire maintains safety through hardware memory protection:

// Buffer ownership is explicit and transferable
let buffer = BAREWire.createBuffer<float> 1024

// Create multiple access capabilities with different permissions
let readOnlyAccess = buffer.GetReadAccess()
let writeAccess = buffer.GetWriteAccess()

// Share with another process via memory mapping
let sharedAccess = buffer.ShareWithProcess(targetProcessId)

// The compiler automatically handles cleanup at scope boundaries
// through delimited continuations - no manual disposal needed

This approach provides the memory safety developers depend on, but through hardware enforcement and compile-time analysis as opposed to design-time burdens and runtime restrictions. The signal model inherits these guarantees: reactive computations can safely hold references into shared memory because the capability system ensures those references remain valid for the duration of the computation.

Buffer Lifecycle and Actor Boundaries

BAREWire buffer design has an explicit lifetime tied to arena allocation. In the Fidelity Framework, arenas are associated with actors. This creates a natural integration point: buffer signals live within actor arenas, and buffer lifecycle follows actor lifecycle. As explained in RAII in Olivier and Prospero, when an actor terminates, its entire memory arena is immediately reclaimed without any scanning or collection pauses.

module Olivier.Reactive

/// Actor-scoped reactive state
type ReactiveActor<'State, 'Message>(initialState: 'State) =
 let arena = Arena.current()
 let stateBuffer = arena.Allocate<'State>(BARELayout.of'<'State>())
 let stateSignal = BufferSignal.create stateBuffer

 /// Read current state (zero-copy reference)
 member _.State : BARERef<'State> =
 BufferSignal.get stateSignal

 /// Update state (triggers dependents)
 member _.UpdateState(f: 'State -> 'State) =
 let current = BufferSignal.get stateSignal |> BARERef.deref
 let newValue = f current
 BARELayout.write stateSignal.Layout stateSignal.Buffer stateSignal.Offset newValue
 Interlocked.Increment(&stateSignal.Version) |> ignore

 /// Create a derived computation scoped to this actor
 member this.Derived<'T>(f: unit -> 'T) : Signal<'T> =
 let derivedBuffer = arena.Allocate<'T>(BARELayout.of'<'T>())
 let derived = BufferSignal.create derivedBuffer

 let _ = memo (fun () ->
 let result = f()
 BARELayout.write derived.Layout derived.Buffer derived.Offset result
 result)

 // No explicit cleanup registration; actor termination handles it
 derived

Dependent computations in other actors receive invalidation notifications and must handle the terminated state appropriately. From a design perspective, this would seem to align with Erlang’s “let it crash” philosophy: explicit lifecycle boundaries as opposed to distributed garbage collection for cross-actor references. We expect to give this area considerable attention as the implementation work proceeds.

%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph Actor1["Actor A (Arena A)"]
S1[State Signal]
D1[Derived Signal]
S1 --> D1
end
subgraph Actor2["Actor B (Arena B)"]
S2[State Signal]
D2[Derived Signal]
S2 --> D2
end
subgraph CrossActor["Cross-Actor Dependency"]
D1 -.->|"WeakRef"| E1[Effect in Actor B]
E1 --> D2
end
subgraph Termination["Actor A Terminates"]
T1[Arena A Released]
T1 --> T2[S1, D1 Invalid]
T2 --> T3[E1 Receives Invalidation]
T3 --> T4[E1 Handles Terminated State]
end

The Cross-Target Challenge

Our Fidelity Framework targets three distinct compilation paths: native execution via MLIR (and in many cases LLVM by extension), browser deployment via Fable to JavaScript, and .NET managed runtime interop. Each target has different characteristics that affect reactive implementation.

Native Target (Fidelity/MLIR)

The native target has full control over memory layout and threading. Buffer signals can use raw pointers into arena-allocated memory. Thread-local storage provides tracking context. RAII semantics ensure cleanup when scopes exit, with the Composer compiler inserting cleanup code during IR lowering based on coeffect analysis.

// Native implementation sketch
module Fidelity.Reactive.Native

[<Struct>]
type NativeSignal<'T> = {
 Ptr: nativeptr<'T>
 mutable Version: int64
 Dependents: nativeptr<DependentList>
}

module NativeSignal =
 let inline get (signal: NativeSignal<'T>) : 'T =
 match NativeReactive.currentScope() with
 | ValueSome scope ->
 NativeReactive.trackDependency scope signal.Ptr
 | ValueNone -> ()
 NativePtr.read signal.Ptr

 let inline set (signal: NativeSignal<'T>) (value: 'T) : unit =
 NativePtr.write signal.Ptr value
 Interlocked.Increment(&signal.Version) |> ignore
 NativeDependents.notifyAll signal.Dependents

The native implementation can leverage MLIR’s optimization passes. Simple signal reads may inline completely, leaving only the memory access. The tracking check can use branch prediction hints since the common case (reading outside a tracked scope) is predictable.

Browser Target (Fable/Fidelity.CloudEdge)

The browser target trades the native target’s memory control for JavaScript’s runtime semantics. Clef compiles through Fable to JavaScript, where we leverage the SPEC stack¹ (SolidJS, Partas.Solid, Elmish, and Fidelity.CloudEdge) for deployment. SolidJS provides the reactive runtime, and its subscription-free signal model directly inspired our approach. Rather than reimplementing signal mechanics, we delegate to SolidJS’s primitives through Partas.Solid’s F# bindings:

// Fable implementation using SolidJS
module Fidelity.Reactive.Fable

open Fable.Core
open Fable.Core.JsInterop

[<Import("createSignal", "solid-js")>]
let private createSolidSignal<'T> (initial: 'T) : (unit -> 'T) * ('T -> unit) = jsNative

[<Import("createMemo", "solid-js")>]
let private createSolidMemo<'T> (fn: unit -> 'T) : (unit -> 'T) = jsNative

[<Import("createEffect", "solid-js")>]
let private createSolidEffect (fn: unit -> unit) : unit = jsNative

type Signal<'T> = {
 Get: unit -> 'T
 Set: 'T -> unit
}

module Signal =
 let create (initial: 'T) : Signal<'T> =
 let get, set = createSolidSignal initial
 { Get = get; Set = set }

 let inline get (signal: Signal<'T>) : 'T = signal.Get()
 let inline set (signal: Signal<'T>) (value: 'T) : unit = signal.Set(value)

let memo (f: unit -> 'T) : unit -> 'T =
 createSolidMemo f

let effect (f: unit -> unit) : unit =
 createSolidEffect f

This approach leverages SolidJS’s reactive implementation while presenting an idiomatic Clef API. The Fable compiler erases the types to direct SolidJS calls, incurring no additional overhead at runtime. SolidJS handles its own cleanup internally; no IDisposable bridge required.

For BAREWire integration in the browser, buffers become Uint8Array instances:

module BAREWire.Fable

type BAREBuffer = Uint8Array

type BufferSignal<'T> = {
 Buffer: Signal<BAREBuffer>
 Layout: BARELayout<'T>
}

module BufferSignal =
 let get (signal: BufferSignal<'T>) : BARERef<'T> =
 let currentBuffer = Signal.get signal.Buffer
 { Buffer = currentBuffer; Offset = 0; Layout = signal.Layout }

 let updateBuffer (signal: BufferSignal<'T>) (newBuffer: BAREBuffer) : unit =
 Signal.set signal.Buffer newBuffer

The SolidJS signal wraps the buffer reference. When a new buffer arrives (via WebSocket, for example), setting the signal triggers SolidJS’s reactive propagation. Dependent computations re-execute, reading from the new buffer. The old buffer becomes eligible for garbage collection.

Edge Target (Fidelity.CloudEdge)

The same Fable compilation pipeline that targets browsers also deploys to Cloudflare Workers, the “C” in the SPEC stack. Our Fidelity.CloudEdge library² provides type-safe bindings to Cloudflare’s edge infrastructure, including Durable Objects for persistent state and coordinated WebSocket connections. Where SolidJS manages reactive state within a browser tab, Durable Objects extend this coordination across distributed clients through what we term the “cloud edge backplane.”

Our Fidelity.CloudEdge library exposes this infrastructure through idiomatic Clef types:

// Fidelity.CloudEdge Durable Object state management
type DurableObjectState<'Props> =
 abstract member acceptWebSocket: ws: WebSocket * ?tags: ResizeArray<string> -> unit
 abstract member getWebSockets: ?tag: string -> ResizeArray<WebSocket>
 abstract member storage: DurableObjectStorage

type DurableObjectStorage =
 abstract member get<'T> : key: string -> JS.Promise<'T option>
 abstract member put<'T> : key: string * value: 'T -> JS.Promise<unit>
 abstract member transaction<'T> : closure -> JS.Promise<'T>

The persistent WebSocket capability matters here. Unlike traditional WebSocket servers that lose connection state on restart, Durable Objects maintain WebSocket connections across the Cloudflare network with automatic failover. This persistence aligns with BAREWire’s zero-copy philosophy: binary messages flow directly from edge to client without intermediate serialization layers.

// Signal bridge between Durable Object and connected clients
module EdgeSignal =
 let broadcast (state: DurableObjectState<obj>) (signal: BufferSignal<'T>) =
 effect (fun () ->
 let ref = BufferSignal.get signal
 let buffer = BARERef.asArrayBuffer ref
 let sockets = state.getWebSockets()
 for socket in sockets do
 socket.send(buffer))

 let fromWebSocket (state: DurableObjectState<obj>) (layout: BARELayout<'T>) =
 let signal = Signal.create Unchecked.defaultof<'T>
 // WebSocket message handler updates signal
 // Connected clients receive updates via reactive propagation
 signal

We envision the architecture enabling scenarios where a signal update on one client propagates through the edge backplane to all connected clients, with BAREWire carrying the binary payload across the network without transformation. The Durable Object becomes a coordination point in the reactive graph, extending signal semantics beyond a single runtime.

This is a feature area that is still under development at Cloudflare, and we will follow their updates closely.

.NET Interop Target

For scenarios requiring .NET interop, the signal model can bridge to IObservable<T>. This bridge exists at system boundaries where Fidelity code must interface with existing .NET libraries:

module Fidelity.Reactive.DotNet

open System
open System.Reactive.Subjects

/// Wrap a Fidelity signal as an IObservable (for .NET interop)
let toObservable (signal: Signal<'T>) : IObservable<'T> =
 let subject = new BehaviorSubject<'T>(Signal.get signal)

 effect (fun () ->
 let value = Signal.get signal
 subject.OnNext(value))

 subject :> IObservable<'T>

/// Create a Fidelity signal from an IObservable (for .NET interop)
let fromObservable (observable: IObservable<'T>) (initial: 'T) : Signal<'T> =
 let signal = Signal.create initial
 // This subscription handle is managed at the interop boundary
 let _ = observable.Subscribe(fun value -> Signal.set signal value)
 signal

This bridge necessarily reintroduces subscription handles at the boundary between Fidelity and .NET code. This complexity appears only at system edges. Again, the details of this still have to be sorted out, but our sense is that this is a manageable complexity that can achieve some consistency across the various platforms.

The BAREWire Driver Architecture

To support all three targets from a unified codebase, we introduce an abstraction layer that captures essential operations while allowing target-specific implementations:

// In BAREWire.Core (target-agnostic)
[<AbstractClass>]
type BareDriver<'TBuffer>() =
 /// Allocate a buffer of the specified size
 abstract Allocate: size:int -> 'TBuffer

 /// Release a buffer back to the pool
 abstract Release: buffer:'TBuffer -> unit

 /// Get a span view of buffer contents
 abstract GetSpan: buffer:'TBuffer -> BARESpan

 /// Create a signal holding a buffer reference
 abstract CreateBufferSignal<'T> : layout:BARELayout<'T> -> initial:'TBuffer -> BufferSignal<'T>

 /// Create a standard value signal
 abstract CreateSignal<'T> : initial:'T -> Signal<'T>

 /// Create a memoized computation
 abstract CreateMemo<'T> : compute:(unit -> 'T) -> Signal<'T>

 /// Create an effect (cleanup handled by driver)
 abstract CreateEffect: action:(unit -> unit) -> unit

/// Transport abstraction for message delivery
[<AbstractClass>]
type BareTransport<'TBuffer>() =
 /// Send a buffer
 abstract Send: buffer:'TBuffer -> Async<unit>

 /// Receive the next buffer
 abstract Receive: unit -> Async<'TBuffer>

 /// Connection state as a signal
 abstract State: Signal<ConnectionState>

 /// Register message handler (cleanup handled by transport)
 abstract OnMessage: ('TBuffer -> unit) -> unit

and ConnectionState =
 | Connecting
 | Connected
 | Disconnected of reason:string
 | Reconnecting of attempt:int

/// Edge transport leveraging Cloudflare's persistent WebSockets
[<AbstractClass>]
type EdgeTransport<'TBuffer>() =
 inherit BareTransport<'TBuffer>()

 /// Durable Object coordination for multi-client scenarios
 abstract DurableObjectId: string option

 /// Subscribe to tagged broadcast groups
 abstract JoinGroup: tag:string -> Async<unit>

 /// Leave a broadcast group
 abstract LeaveGroup: tag:string -> Async<unit>

 /// Broadcast to all clients in a group
 abstract Broadcast: tag:string -> buffer:'TBuffer -> Async<unit>

Note that CreateEffect and OnMessage return unit, not IDisposable. The driver implementation manages cleanup internally, aligned with Fidelity’s philosophy that resource management is infrastructure, not application code.

Application code programs against these abstractions:

module Application

let createReactiveChannel<'TOut, 'TIn>
 (driver: BareDriver<'TBuffer>)
 (transport: BareTransport<'TBuffer>)
 (outCodec: BARECodec<'TOut>)
 (inCodec: BARECodec<'TIn>) =

 let lastMessage = driver.CreateSignal<'TIn option>(None)

 transport.OnMessage(fun buffer ->
 let decoded = inCodec.Decode(driver.GetSpan(buffer))
 Signal.set lastMessage (Some decoded)
 driver.Release(buffer))

 let send (message: 'TOut) =
 async {
 let size = outCodec.EncodedSize(message)
 let buffer = driver.Allocate(size)
 outCodec.Encode(message, driver.GetSpan(buffer))
 do! transport.Send(buffer)
 }

 {| Send = send
 Messages = lastMessage
 State = transport.State |}

The driver and transport are injected, allowing the same application code to run against different targets. For Fidelity.CloudEdge browser deployment, the driver wraps SolidJS signals and Uint8Array buffers; the transport wraps WebSocket. For Fidelity.CloudEdge edge deployment, the transport extends to leverage Durable Objects for persistent connections and coordinated state. For Fidelity native, the driver uses native signals and arena-allocated buffers; the transport might use Unix sockets or shared memory.

%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph Application["Application Code"]
A[Reactive Channel]
A --> B[BARECodec]
A --> C[Signal Operations]
end
subgraph Abstraction["Driver Abstraction"]
D[BareDriver]
E[BareTransport]
E2[EdgeTransport]
end
subgraph Browser["Browser Target"]
F1[SolidJS Signals]
F2[Uint8Array Buffers]
F3[WebSocket Transport]
end
subgraph Edge["Edge Target (Cloudflare)"]
CF1[Durable Object State]
CF2[Persistent WebSockets]
CF3[Tagged Broadcast Groups]
end
subgraph Native["Native Target"]
N1[Native Signals]
N2[Arena Buffers]
N3[Unix Socket / Shared Memory]
end
subgraph DotNet[".NET Interop"]
DN1[Bridge to IObservable]
DN2[Managed Buffers]
DN3[TCP / Named Pipes]
end
Application --> Abstraction
D --> F1
D --> CF1
D --> N1
D --> DN1
E --> F3
E2 --> CF2
E --> N3
E --> DN3

Actor Integration: Signals and Message Boundaries

The Olivier actor model provides natural boundaries for reactive scope. An actor processes messages sequentially; between messages, no actor code is executing. This creates clean points for reactive operations:

Message arrival: When a message arrives, it may carry data that should update signals. The actor’s receive function reads from the message buffer and updates local state signals.

Message processing: During processing, the actor may read signals (triggering dependency tracking if within a reactive scope) and compute derived values.

Message completion: When processing completes, any pending reactive effects execute. The actor is then ready for the next message.

module Olivier.ReactiveActor

type ReactiveMailbox<'State, 'Msg> = {
 State: Signal<'State>
 Derived: (unit -> 'T) -> Signal<'T>
}

let reactiveActor<'State, 'Msg>
 (initialState: 'State)
 (behavior: ReactiveMailbox<'State, 'Msg> -> 'Msg -> 'State) =

 let stateSignal = Signal.create initialState

 let mailbox = {
 State = stateSignal
 Derived = fun f -> memo f
 }

 let processMessage (msg: 'Msg) =
 let newState = behavior mailbox msg
 Signal.set stateSignal newState
 // Effects run here, after state update

 { new IActor<'Msg> with
 member _.Tell(msg) = processMessage msg
 member _.State = stateSignal }

Reactive propagation happens within message processing, not asynchronously. When Signal.set stateSignal newState executes, any effects depending on stateSignal run immediately, before the message handler returns. This provides deterministic execution order: the actor sees a consistent state throughout message processing.

For effects that should run after message processing (such as sending messages to other actors), we introduce a deferred effect queue:

module Olivier.DeferredEffects

let mutable private deferredQueue: (unit -> unit) list = []

let defer (action: unit -> unit) =
 deferredQueue <- action :: deferredQueue

let flushDeferred () =
 let actions = deferredQueue
 deferredQueue <- []
 for action in List.rev actions do
 action()

let processMessage (msg: 'Msg) =
 let newState = behavior mailbox msg
 Signal.set stateSignal newState
 // Immediate effects run during set
 flushDeferred() // Deferred effects run after

This pattern prevents cascading message sends during state updates, which could lead to unbounded recursion or reentrancy issues.

The Cloud Edge Backplane

Having established how our signal model operates within individual runtimes (native via MLIR, browser via the SPEC stack, and .NET interop), we now turn to coordination across distributed clients. This is where our Fidelity.CloudEdge library and Durable Objects would carry the signal model from a single-runtime pattern into distributed coordination. Cloudflare’s platform provides three capabilities that extend reactive semantics across the network.

Durable Objects as Signal Coordinators

Durable Objects provide single-threaded, strongly consistent execution at the edge. Each Durable Object instance runs in exactly one location globally, processes requests sequentially, and maintains persistent state across invocations. This model maps directly to our actor-based signal architecture:

// Durable Object as a distributed signal coordinator
type SignalCoordinator(state: DurableObjectState<obj>) =
 let storage = state.storage

 member _.fetch(request: Request) = promise {
 match Request.path request with
 | "/ws" ->
 // Upgrade to WebSocket and accept with tagged group
 let pair = WebSocketPair.create()
 state.acceptWebSocket(pair.server, ResizeArray(["signals"]))
 return Response.webSocket(pair.client)

 | "/signal" ->
 // Retrieve current signal value from persistent storage
 let! value = storage.get<SignalState>("current")
 return Response.json(value)

 | _ -> return Response.notFound()
 }

 member _.webSocketMessage(ws: WebSocket, message: string) = promise {
 // Parse incoming signal update
 let update = JSON.parse<SignalUpdate>(message)

 // Persist to Durable Object storage
 do! storage.put("current", update.value)

 // Broadcast to all connected clients
 let sockets = state.getWebSockets("signals")
 for socket in sockets do
 if socket <> ws then socket.send(message)
 }

The Durable Object becomes the single source of truth for a signal’s value, with connected clients maintaining local reactive mirrors. When any client updates the signal, the change persists in Durable Object storage and propagates to all other clients through the persistent WebSocket connections. The sequential execution guarantee ensures no race conditions, while the tagged broadcast groups enable efficient fan-out without iterating all connections.

Persistent WebSockets and Connection Resilience

Traditional WebSocket architectures lose connection state when servers restart or connections migrate. Cloudflare’s persistent WebSockets, managed through Durable Objects, maintain connection metadata across these events. For the signal model, this means reactive subscriptions survive infrastructure changes:

module EdgeReactive =
 /// Create a signal synchronized across edge and clients
 let distributedSignal<'T>
 (durableObject: DurableObjectStub)
 (initial: 'T)
 (codec: BARECodec<'T>) =

 let localSignal = Signal.create initial
 let ws = WebSocket.connect(durableObject.url + "/ws")

 // Incoming updates from edge coordinator
 ws.onMessage(fun buffer ->
 let value = codec.Decode(buffer)
 Signal.set localSignal value)

 // Outgoing updates to edge coordinator
 let sendToEdge = effect (fun () ->
 let value = Signal.get localSignal
 let buffer = codec.Encode(value)
 ws.send(buffer))

 localSignal

The resulting signal behaves identically to a local signal from the application’s perspective. Reads and writes use the same API. The edge synchronization is infrastructure, not application code. When the WebSocket reconnects after a network interruption, the Durable Object replays the current state, and the local signal updates accordingly.

Media-over-QUIC: The Streaming Horizon

Looking ahead, Cloudflare’s Media-over-QUIC relay infrastructure opens possibilities for real-time media streaming integrated with the signal model. Where WebSockets excel at reliable, ordered message delivery, QUIC provides the low-latency, loss-tolerant transport that live media requires. The combination enables scenarios where:

Signal updates coordinate playback state (play, pause, seek position)
Media frames flow through QUIC streams without head-of-line blocking
BAREWire encodes both control signals and media metadata in a unified format

While this integration remains forward-looking, the architecture anticipates it. The BareTransport abstraction can accommodate QUIC-based implementations alongside WebSocket transports. The signal model’s separation of reactive semantics from transport mechanics means applications written today can adopt QUIC transport tomorrow without architectural changes.

%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph Clients["Connected Clients"]
C1[Browser A]
C2[Browser B]
C3[Native App]
end
subgraph Edge["Cloudflare Edge"]
DO[Durable Object]
WS1[Persistent WebSocket]
WS2[Persistent WebSocket]
WS3[Persistent WebSocket]
Storage[(Durable Storage)]
end
subgraph Future["Future: Media Streaming"]
QUIC[Media-over-QUIC Relay]
Media[Live Media Streams]
end
C1 <-->|BAREWire| WS1
C2 <-->|BAREWire| WS2
C3 <-->|BAREWire| WS3
WS1 --> DO
WS2 --> DO
WS3 --> DO
DO <--> Storage
DO -.->|Control Signals| QUIC
QUIC -.->|Media Frames| Media

This edge backplane is meant to carry the signal model from a single-runtime pattern into distributed coordination. The same Clef code, compiled through Fable to JavaScript, would run in browsers, Cloudflare Workers, and Durable Objects. Signals flow across these boundaries with BAREWire carrying the binary encoding. The platform provides the persistence, fan-out, and resilience; the application code remains focused on reactive logic.

Connecting to Alloy.Rx

The signal model presented here complements the Alloy.Rx reactive framework by providing a different abstraction for different use cases. Where Alloy.Rx distinguishes between multicast (broadcast) and unicast (isolated) observables based on push semantics, the signal model focuses on pull semantics with automatic dependency tracking.

The distinction matters. Our Alloy.Rx multicast observables suit event streams where producers push updates to multiple observers, achieving zero allocation for broadcast scenarios. This makes them a natural fit for sensor data, UI events, and system notifications. Unicast observables provide per-subscriber isolation with arena-based allocation, which becomes appropriate when each subscriber needs independent processing state.

The signal model takes a different approach: pull-based semantics with implicit dependency tracking. This makes signals a natural fit for derived state, computed properties, and reactive UI bindings where the consumer controls when values are read rather than reacting to pushed updates.

These models can interoperate. A signal can be updated from an Alloy.Rx observable:

let bridgeToSignal (observable: Observable<'T, Multicast>) : Signal<'T> =
 let signal = Signal.create (Multicast.current observable)
 Multicast.subscribe (fun (_, newValue) ->
 Signal.set signal newValue) observable
 signal

And a signal’s changes can drive an observable:

let bridgeToObservable (signal: Signal<'T>) : Observable<'T, Multicast> =
 let observable = Multicast.create (Signal.get signal)
 effect (fun () ->
 let value = Signal.get signal
 Multicast.next value observable)
 observable

The choice between models depends on the communication pattern. Event streams flow through Alloy.Rx; derived state lives in signals.

Schema Evolution and Versioning

BAREWire uses position-based encoding, which raises questions about schema evolution. Our approach follows an append-only discipline with explicit versioning:

[<BAREMessage>]
type SensorReading_V1 = {
 [<BAREField(0)>] Timestamp: uint64
 [<BAREField(1)>] Value: float32
}

[<BAREMessage>]
type SensorReading_V2 = {
 [<BAREField(0)>] Timestamp: uint64
 [<BAREField(1)>] Value: float32
 [<BAREField(2)>] SensorId: string
 [<BAREField(3)>] Confidence: float32
}

The wire format includes a version byte. Decoders read the version and parse accordingly:

module BAREWire.Versioned

let decode<'T> (layout: VersionedLayout<'T>) (span: BARESpan) : 'T =
 let version = span.[0]
 match Map.tryFind version layout.Versions with
 | Some decoder -> decoder span
 | None ->
 if version > layout.LatestVersion then
 // Newer version than we know; decode what we can
 layout.Versions.[layout.LatestVersion] span
 else
 failwith $"Unknown schema version: {version}"

Older readers encountering newer versions decode the fields they understand and ignore the rest. Newer readers handle older versions by applying default values for missing fields. This approach is simpler than protobuf’s sparse field numbering but requires discipline: once a field occupies position N, it remains there permanently.

Performance Characteristics

The signal model’s performance depends heavily on the target environment and usage patterns. Key characteristics:

Read overhead: In the common case (reading outside a tracked scope), signal reads add only a branch to check the thread-local context. With branch prediction, this approaches zero overhead. Within tracked scopes, reads additionally register dependencies.

Write overhead: Signal writes iterate through dependents and invoke invalidation callbacks. The cost scales with the number of dependents. For signals with many dependents, batching updates can amortize overhead.

Memory overhead: Each signal maintains a list of dependents. Using weak references prevents leaks but adds indirection. For native targets, custom allocators can pool dependency records.

Zero-copy benefit: For buffer signals, the reactive machinery operates on lightweight references while heavy data remains stationary. This is particularly beneficial for large messages and streaming data. As detailed in our byref resolution, eliminating the “copy tax” transforms what would be performance bottlenecks into competitive advantages.

Aspect	Signal Model	Rx/IObservable
Subscription creation	Implicit via read	Explicit Subscribe call
Subscription disposal	Compiler-managed	Manual Dispose required
Dependency tracking	Runtime during execution	Compile-time via operators
Cold/hot distinction	N/A (signals are always hot)	Complex semantics
Threading model	Thread-local context	Schedulers and synchronization
Memory model	Zero-copy capable	Values copied through pipeline
Edge distribution	Native via Durable Objects	Requires custom infrastructure

Reactive Architecture Without Ceremony

The signal model outlined here departs from subscription-based reactive programming. By making dependency tracking implicit in signal reads, we remove a category of resource management concerns. By integrating signals with BAREWire’s capability-based memory architecture, we aim to handle large data volumes without allocation overhead. By aligning signal lifecycle with actor boundaries through RAII principles, we provide deterministic cleanup without garbage collection.

By having our Composer compiler manage cleanup through scope analysis and coeffect tracking, we free developers from the IDisposable ceremony that pervades .NET code. This reflects a principle of our framework: infrastructure concerns belong in the compiler, not in application code.

The architectural implications carry through application design:

Derived signals compose freely without subscription handles to manage. Reactive graphs emerge from function composition rather than orchestration of observable chains.

Testing simplifies as well. Without subscription lifecycle concerns, reactive code tests like pure functions: set inputs, check outputs. There’s no need to mock subscription mechanics or carefully time disposal.

The same reactive patterns work consistently across native, browser, edge, and managed targets. Application logic remains unchanged; only the driver implementation varies by platform.

Signals and actors reinforce each other. Actor boundaries provide cleanup semantics while signals provide reactive state within actors. Durable Objects extend this to the edge, providing persistent coordination across distributed clients.

Our framework’s approach to reactivity follows a position we hold across the design: provide capabilities through composable primitives. Developers should not need to become experts in subscription lifecycle management to build reactive applications. They should be able to read values, derive computations, and rely on the system to handle the plumbing.

BAREWire provides the memory substrate where signals and buffers coexist. The BARE protocol’s binary encoding maps directly to memory layout, which aligns with our zero-copy goals. Our solution to the byref problem removes the defensive copying that would otherwise negate the benefits of reactive optimization. The SPEC stack carries these patterns to the browser through SolidJS’s signal implementation, while our Fidelity.CloudEdge library extends the reach to the edge, where Durable Objects and persistent WebSockets carry local reactive patterns into distributed coordination.

This is the direction we will keep building toward: a reactive system where data flows through dependency graphs without copying, where cleanup is deterministic, and where signals propagate from native code through edge infrastructure to browser UIs. There is implementation work ahead on every target named here, and we will report on it as the design takes shape.

This article is part of our ongoing series exploring the Fidelity Framework’s designs for systems programming with Clef. Related entries include Coeffects and Codata in Composer, RAII in Olivier and Prospero, ByRef Resolved, and Alloy.Rx: Native Reactivity in Fidelity.

The SPEC stack (SolidJS, Partas.Solid, Elmish, and Fidelity.CloudEdge) is our design for unified web development in Clef. See The SPEC Stack: A Proposal for the full architectural overview. ↩︎
Our Fidelity.CloudEdge library enables Clef deployment to Cloudflare’s edge network through Fable compilation. For a deeper exploration of this integration, see Leaner, Smarter AI Cloud Systems. ↩︎

A Vision For Unified Cognitive Architecture

Wed, 15 Oct 2025 00:00:00 -0400

AI’s Berlin Wall

In our exploration of neuromorphic computing, we examined how specialized hardware might finally deliver on AI’s efficiency promises. Hardware alone does not address a structural limitation in current AI systems: the wall between how systems learn and how they operate.

🔄 Updated October 22, 2025

This article now includes cross-references to related blog entries, connecting broader concepts presented here to detailed technical explorations elsewhere. These inline links serve as entry points for readers seeking deeper dives into various topics, while this blog entry illuminates our broader vision.

Every “AI” company today lives with a divide we’ve accepted as natural law: models are trained (expensive, slow, centralized) then deployed for inference (cheap, fast, distributed). This binary worldview has created a technological imbalance where accumulating and disseminating information exists in separate universes. Yes, modern models can incorporate corrections through in-context learning, but it’s a workaround, not a solution. The knowledge lives in fragile prompt engineering and vanishes when context windows reset. A GPT model absorbs your correction only until the conversation ends. A vision model adapts to your specific product only within the bounds of few-shot examples, never truly updating its understanding. The working model is still trapped in the high, thick wall between training and inference.

The industry has built considerable infrastructure around this dichotomy: training clusters that consume megawatts, inference servers that strain to learn, and an entire industry devoted to managing frozen intelligence. Much like the specialized LISP machines that seemed essential until economic forces rendered them obsolete, this barrier has come to seem permanent. We think a different architecture can move past it.

Hypergraphs as a Shared Substrate

We’ve written before about how hypergraphs naturally express complex relationships in our compiler’s Program Semantic Graph. The question we are working through is whether this same mathematical substrate can represent not just program structure, but knowledge, reasoning, and learning.

We theorize an AI system without frozen weights, a knowledge structure that grows with use, where compilation, knowledge representation, reasoning, and execution share one mathematical foundation. That is the direction hypergraph-based cognitive architectures point, extending our compiler work toward continuous intelligence.

The Universal Substrate

Traditional neural networks force everything through matrix multiplication:

y = \sigma(Wx + b)

As we explored in our entry on moving beyond transformers, this matrix-centric view is increasingly recognized as a limitation rather than a law. Reasoning isn’t matrix multiplication. Knowledge isn’t tensors. Language isn’t vectors. The hypergraph represents relationships as they actually exist:

\mathcal{H} = (V, E, \phi: E \to 2^V, \tau: E \to \text{Type}, w: E \to \mathbb{P})

Where vertices $V$ are concepts, hyperedges $E$ connect multiple concepts simultaneously, and weights $w$ use posit arithmetic for tapered precision exactly where needed.

type UniversalHypergraph =
 | Knowledge of concepts: Set<Concept> * relations: Set<Relation>
 | Reasoning of states: Set<State> * transitions: Set<Inference>
 | Compilation of ast: Set<Node> * constraints: Set<Proof>
 | Execution of ops: Set<Operation> * dataflow: Set<Dependency>

 // one traversal over four cases, one structure
 member this.Traverse(query) =
 match this with
 | Knowledge kg -> reasonThroughKnowledge kg query
 | Reasoning rg -> executeReasoning rg query
 | Compilation cg -> optimizeViaProofs cg query
 | Execution eg -> runOnHardware eg query

Fidelity’s Library of Alexandria

Knowledge as a Service, Not a Monument

The ancient Library of Alexandria didn’t try to fit all knowledge into a single scroll. It organized specialized collections that scholars could access as needed.

Instead of monolithic models, we envision specialized knowledge hypergraphs that load on demand. A system loads the domains a given query requires, rather than holding all knowledge resident at once.

type CognitiveLibrary = {
 Core: BaseKnowledge // Always loaded: language, logic, common sense
 Catalog: DomainRegistry // Available specializations
 Librarian: Alex // Builds and indexes hypergraphs during compilation
 Cards: Map<User, Capabilities> // What each user can access
}

let processQuery query library =
 let required = analyzeRequirements query
 let loaded =
 required
 |> Set.map (fun domain ->
 library.Load domain // Memory-mapped, zero-copy
 )
 traverse loaded query

The economics shift with this structure: instead of training massive models, organizations build and trade specialized hypergraphs. A Bloomberg financial relationships graph. A PubMed molecular interactions graph. A proprietary manufacturing process graph. Knowledge becomes a modular, composable asset that can be loaded and exchanged.

Fractal Reasoning

System 1 vs System 2, Naturally

In this design, reasoning depth follows from the dynamics of the hypergraph traversal. We place the interesting behavior at the boundary between stable and chaotic regimes:

\lambda = \lim_{n \to \infty} \frac{1}{n} \sum_{i=0}^{n-1} \ln|f'(x_i)|

When $\lambda < 0$ : stable, fast, System 1 thinking. When $\lambda > 0$ : chaotic, exploratory, System 2 reasoning.

let adaptiveReasoning state query =
 let lyapunov = computeLyapunov state
 match lyapunov with
 | lambda when lambda < -0.5 ->
 // Stable: direct retrieval
 { Strategy = DirectPath; Depth = 1 }
 | lambda when lambda > 0.5 ->
 // Chaotic: deep exploration
 { Strategy = MonteCarloTree; Depth = 10..20 }
 | _ ->
 // Boundary: the interesting dynamics
 { Strategy = FractalSearch; Depth = variable }

Variable Depth Across the Graph

Reasoning depth isn’t global; it’s local to the complexity encountered:

\text{depth}(v) = \begin{cases} 1 & \text{if simple lookup} \\ \log n & \text{if pattern matching} \\ \sqrt{n} & \text{if analogical reasoning} \\ n & \text{if proof construction} \end{cases}

let variableDepthTraversal hypergraph start =
 let rec traverse vertex depth =
 let complexity = lambdaLocalComplexity vertex
 let newDepth =
 if complexity > threshold then
 depth * 2 // Double at complexity spikes
 else
 max 1 (depth - 1) // Reduce in simple regions

 let neighbors = hypergraph.GetHyperedges vertex
 neighbors |> List.map (traverse newDepth)

 traverse start 1

Hardware as Fluid Architecture

CGRA: The Shape-Shifting Processor

As we’ve seen in our examination of dataflow architectures emerging from the LISP era, computation doesn’t have to be instruction-driven. Coarse-Grained Reconfigurable Arrays represent this evolution: hardware that doesn’t execute fixed instructions but reshapes itself to match the computation’s dynamics.

let reconfigureBasedOnDynamics cgra lyapunov =
 match lyapunov with
 | Stable ->
 // Configure as pipeline
 cgra.Configure {
 Topology = LinearPipeline
 Tiles = [MAC; MAC; Add; Store]
 Routing = NearestNeighbor
 }
 | Chaotic ->
 // Configure as exploration mesh
 cgra.Configure {
 Topology = FullMesh
 Tiles = [Branch; Compare; Route; Cache]
 Routing = Crossbar
 }
 | Boundary ->
 // Hybrid configuration
 cgra.Configure {
 Topology = FractalClusters
 Tiles = mixed
 Routing = Adaptive
 }

Neuromorphic Spike Patterns

Building on our work with neuromorphic architectures, hyperedges become synchronization patterns that map naturally to spike-based computation:

type NeuromorphicMapping = {
 Vertices: Map<Concept, NeuronGroup>
 Hyperedges: Map<Relation, SpikePattern>
 Reasoning: SpikeWave -> SpikeWave
}

let neuromorphicReasoning query mapping =
 let spikes = encodeAsSpikes query
 let rec propagate wave =
 let activated =
 mapping.Hyperedges
 |> Map.filter (matchesPattern wave)

 let next = fireNeurons activated
 if converged next then
 decode next
 else
 propagate next

 propagate spikes

The Twilight of Discrete Training

Every Query Teaches

In the architecture we are sketching, training and inference are not separate phases. Each interaction would strengthen the paths it used:

let continuousLearning hypergraph query =
 // traditional "inference"
 let result = traverse hypergraph query

 // strengthen used paths, traditional "training"
 let strengthened =
 result.Path
 |> List.map (fun edge ->
 { edge with Weight = edge.Weight * 1.01 })

 let newEdges =
 if result.Quality > threshold then
 createHyperedge result.Concepts
 else []

 { hypergraph with
 Edges = strengthened @ newEdges }

Blue-Green Knowledge Deployment

We intend knowledge updates to land without disrupting active queries:

let updateKnowledge current update =
 // Load new knowledge in parallel
 let green = loadHypergraph update

 // Test compatibility
 let compatible =
 testQueries
 |> List.forall (fun q ->
 similarity (query current q) (query green q) > 0.95)

 if compatible then
 // Gradual transition
 async {
 for i in 0..100 do
 let weight = float i / 100.0
 activeGraph <- blend current green weight
 do! Async.Sleep 10
 }
 else
 // Keep current until resolved
 reportIncompatibility update

Proof-Aware Reasoning

Proofs Guide Optimization

Our exploration of proof-aware compilation showed how verification properties carry information a compiler can act on. In cognitive architectures, we extend that principle: proofs establish correctness and also direct how reasoning unfolds.

The verifier’s output feeds the optimizer here, the same properties that establish correctness also open up safe transformations:

type ProofGuidedOptimization = {
 Invariants: Set<Property>
 Transforms: Set<Optimization>
 Preserve: Property -> Optimization -> bool
}

let optimizeWithProofs hypergraph proofs =
 hypergraph.Edges
 |> List.map (fun edge ->
 let applicable =
 proofs.Transforms
 |> Set.filter (fun t ->
 proofs.Invariants
 |> Set.forall (fun inv ->
 proofs.Preserve inv t))

 // Apply most aggressive safe optimization
 let optimal = selectBest applicable
 optimal edge)

From Frozen Models to Living Intelligence

The transition to cognitive architectures can be incremental. Much as we’ve designed our Fidelity Framework to bridge traditional and emerging hardware, this path starts by augmenting existing systems before moving to native implementations.

Phase 1: Hybrid Enhancement

Start with existing models, add hypergraph reasoning alongside:

Keep your GPT/Claude/Gemini APIs
Add hypergraph layers for specialized reasoning
Target measurable gains in specific domains, reported with their bounds
Learn which knowledge patterns benefit most from dynamic structures

Phase 2: Knowledge Migration

Convert frozen weights to dynamic hypergraphs:

Extract knowledge from trained models
Recondition as hypergraph structures
Deploy on neuromorphic hardware where available
Establish continuous learning pipelines

Phase 3: Native Intelligence

Pure hypergraph cognitive systems:

No more massive training runs
Continuous learning through use
Knowledge as composable, tradeable assets
Hardware that morphs with thought patterns

Intelligence That Grows

We expect the advantage in the next era of AI to favor systems that update from use over systems that ship a fixed set of weights and freeze them. A hypergraph whose edges strengthen as queries traverse them is one way to build that.

Throughout this series, from proof-aware compilation to neuromorphic hardware, from dataflow architectures to post-transformer models, we’ve been building toward one convergence. We treat the hypergraph as the mathematical substrate that lets compilation, reasoning, and learning share a single representation, rather than as one more compiler intermediate representation or neural network architecture sitting alongside the others.

The wall between training and inference is already under pressure from in-context methods and retrieval. Our interest lies in replacing the wall with shared structure: knowledge that grows, reasoning that adapts its depth, and hardware that reconfigures to the work. We will keep building toward that design as the rest of the framework comes into place.

Unexpected Fusion

Mon, 29 Sep 2025 00:00:00 -0500

The story of distributed systems in F# begins with two distinct programming traditions that converge in F# in unique ways. From OCaml came the functional programming foundation and type system rigor. From Erlang came the mailboxprocessor and with it an approach to fault-tolerant distributed systems. Don Syme’s work fused concurrency into the primitives of a high-level programming language. What emerged in F# was a language that could express actor-based concurrency with type safety, integrate with existing ecosystems, and compile to multiple target platforms.

This convergence wasn’t immediately obvious. When Don Syme first presented F# to Erlang developers at the 2010 Erlang Factory conference in London, he described it as “a pragmatic functional language which the Erlang programmer will find both familiar and foreign.” That duality, familiar yet foreign, captures something essential about F#’s approach to actors. It borrowed wisdom from both traditions while creating space for innovations that neither OCaml nor Erlang had contemplated.

The OCaml Foundation: More Than Just Syntax

F# began its life in 2002 as what was informally called “Caml for .NET”, an attempt to bring OCaml’s ML-style functional programming to Microsoft’s new runtime. But from the beginning, F# was more than a transliteration of OCaml to a new platform. The language took OCaml’s core strengths, its type inference, pattern matching, and functional-first philosophy, and enhanced them with features that would prove essential for building concurrent systems.

The divergences from OCaml were deliberate and thoughtful. Where OCaml required explicit type annotations in many contexts, F# pushed type inference further. Where OCaml used semicolons and explicit delimiters, F# adopted Python’s significant whitespace, making the code cleaner and more approachable. These might seem like surface-level changes, but they reflected a deeper philosophy: F# would be pragmatic where OCaml was purist, accessible where OCaml was opinionated.

Beyond syntax, F# introduced semantic innovations that OCaml hadn’t explored. Units of measure brought dimensional analysis to the type system, allowing developers to catch unit conversion errors at compile time. Computation expressions provided a general framework for defining domain-specific languages within F#, from async workflows to query expressions. These features would later prove essential for expressing complex behaviors.

The Critical Addition: MailboxProcessor as Language Primitive

Perhaps the most significant departure from OCaml was F#’s inclusion of the MailboxProcessor as a built-in language feature. More than a library addition, it gave message-passing concurrency first-class support. The MailboxProcessor brought Erlang’s actor model directly into F#’s type-safe world:

type Message =
 | Increment of int
 | GetValue of AsyncReplyChannel<int>

let counter = MailboxProcessor.Start(fun inbox ->
 let rec loop value = async {
 let! msg = inbox.Receive()
 match msg with
 | Increment delta ->
 return! loop (value + delta)
 | GetValue channel ->
 channel.Reply value
 return! loop value
 }
 loop 0)

This primitive opened a range of possibilities. Beyond Erlang’s dynamically typed messages, F#’s discriminated unions provided compile-time guarantees about message protocols. Inside the confines of the traditional .NET threading models, the MailboxProcessor offered isolation and safety through message passing. It was a bridge between worlds, bringing actor-model thinking to developers who had never encountered Erlang while remaining familiar to those with distributed systems experience.

The inclusion of MailboxProcessor wasn’t accidental. Don Syme’s engagement with the Erlang community, including that 2010 presentation, demonstrates a deliberate connection between OCaml and Erlang ecosystems.

The Fable Connection: OCaml’s Web Legacy

There’s another thread in F#’s relationship with OCaml: the path to the web. Alfonso Garcia-Caro created the Fable compiler, and in later interviews he explicitly acknowledged the inspiration he took from js_of_ocaml (jsoo), OCaml’s solution for compiling to JavaScript. Where jsoo operated as a library extension within OCaml’s ecosystem, Fable forged a different path.

The challenge was architectural. F# was deeply entwined with the .NET runtime, its type system integrated with the Common Language Infrastructure. Translating F# to JavaScript the way jsoo translated OCaml wouldn’t work. Fable needed to become a distinct compilation path, one that could understand the necessary superset of F# semantics at a deep level and regenerate them in type-safe way within the JavaScript ecosystem. An early “slug line” for Fable was “JavaScript you could be proud of” which pointed to its ability to place type ‘hints’ in JavaScript that increased its runtime safety over standard JS code.

Our Fidelity.CloudEdge toolkit builds directly on this foundation. By leveraging Fable’s JavaScript targeting, we intend a library to compile and deploy F# actors as lightweight JavaScript functions. The MailboxProcessor abstractions that developers write would compile down to Workers and Queues, maintaining the actor model’s semantics while taking up the platform’s distribution capabilities.

This synthesis draws OCaml’s web compilation legacy through jsoo, Fable’s reimagining for F#, and our Fidelity.CloudEdge platform integration into one convergence of functional programming traditions adapted for modern computing. At first glimpse it may seem strange to “re-converge” OCaml’s and Erlang’s influences in our Fidelity framework and CloudEdge toolkit, but we hope the interested reader will follow us along a path we consider rewarding.

Extending the Actor Vision

Our Fidelity Framework extends F#’s actor model when unconstrained by the .NET SDK and runtime. By building on the MailboxProcessor primitive, the framework introduces an actor system with supervision hierarchies, distributed message passing, and deterministic resource management.

Where F#’s standard MailboxProcessor operates within the CLR’s managed environment, our Olivier actor model is designed to compile directly to native code. By operating outside a managed runtime, the framework can provide guarantees about memory usage, execution timing, and resource consumption that would be difficult to reach in managed runtime environments.

graph TD
subgraph ".NET F# MailboxProcessor"
MP[MailboxProcessor] --> CLR[CLR Thread Pool]
CLR --> GC[Garbage Collector]
end
subgraph "Fidelity Olivier Model"
Actor[Olivier Actor] --> Arena[Arena Allocator]
Supervisor[Prospero Supervisor] --> Actor
Supervisor -.-> Native[Zero-Copy, IPC<br>& Sentinels]
Arena --> Native
end

Our Prospero supervision layer, detailed in our exploration of RAII in Olivier and Prospero, brings Erlang-style supervision trees to F#. Where Erlang’s process-per-actor model gives each actor an isolated heap, Prospero uses arena allocation within shared process memory. This design choice reflects modern hardware realities: cache coherence has advanced, memory is abundant, and the cost of message copying often exceeds the benefit of complete isolation.

module Olivier =
 type SupervisorStrategy =
 | OneForOne of maxRetries: int * withinTimeSpan: TimeSpan
 | AllForOne of maxRetries: int * withinTimeSpan: TimeSpan
 | RestForOne of maxRetries: int * withinTimeSpan: TimeSpan

 let supervise strategy children =
 // Arena allocated per supervision tree
 use arena = Arena.create (64 * 1024 * 1024) // 64MB

 let supervisor =
 Supervisor.create strategy arena
 |> Supervisor.withChildren children

 supervisor.Start()

Our framework’s design aspires to maintain compatibility with Akka.NET’s cluster communication protocols, because real-world systems need “paved” paths to interoperate. Organizations with existing Akka.NET deployments would be able to gradually include Fidelity components where they show particular advantage. The actor-oriented architecture we advocate is designed to provide process-level protection without runtime overhead, a balance between isolation and native efficiency. We expect this awareness to grow as “agentic systems” become the norm in enterprise environments.

Actors at The Cloud’s Edge

While our Fidelity framework re-imagines actors for native execution, Fidelity.CloudEdge takes a different path, one shaped by the constraints and capabilities of edge computing. In our CloudEdge design, developers write standard F# MailboxProcessor-style actors, and the Fable compiler and CloudEdge toolkit would transform these into compositions of Cloudflare’s platform services. A Worker provides the execution context, a Queue provides the mailbox, and Cloudflare’s global network provides the distribution mechanism, all handled during compilation:

// Developer writes standard F# actor code
type OrderProcessor() =
 inherit MailboxProcessor<OrderMessage>()

 override this.Receive() = async {
 let! msg = this.Receive()
 match msg with
 | ProcessOrder order ->
 // Standard F# async workflow
 let! inventory = checkInventory order.items
 let! payment = processPayment order.payment
 this.Post(UpdateState OrderConfirmed)

 | CancelOrder id ->
 do! refundPayment id
 this.Post(UpdateState OrderCancelled)
 }

// Fidelity.CloudEdge compiles this to:
// - Worker with Queue binding for message handling
// - Durable Object for state management
// - Service bindings for actor communication

This model makes different trade-offs than our native actor model. Where the native model provides fine-grained control over memory and execution, our CloudEdge design accepts platform constraints in exchange for global scale. A CloudEdge actor would handle a high volume of messages per second across hundreds of edge locations and scale horizontally using standard Cloudflare management. The platform handles distribution and scaling, and we design fault response to route through our Prospero supervision hierarchy.

The supervision model in our CloudEdge design differs from both Akka and the native Fidelity model. Where the native model provides inter-process supervision, CloudEdge would distribute fully across Cloudflare’s edge network, with Durable Objects managing hierarchy state and coordinating workers:

// Fidelity.CloudEdge Supervisor actor
module Supervisor =
 type State = {
 ChildActors: Map<string, ActorRef>
 RestartSchedule: Map<string, DateTime>
 }

 let handleMessage (state: State) = function
 | RegisterChild (name, actorRef) ->
 { state with ChildActors = Map.add name actorRef state.ChildActors }

 | ChildFailed (name, error) ->
 match Map.tryFind name state.ChildActors with
 | Some ref ->
 // Schedule restart through platform retry
 let restartTime = DateTime.UtcNow.AddSeconds(5.0)
 { state with RestartSchedule = Map.add name restartTime state.RestartSchedule }
 | None -> state

 | CheckRestarts ->
 // Platform handles actual restarts via Queue retry mechanism
 state

An edge-to-native path also runs between our Fidelity framework and CloudEdge design through Cloudflare’s Container support. As explored in our vision for distributed intelligence, our future design for Fidelity-compiled unikernels would run within Cloudflare’s Container infrastructure, bringing native performance to edge computing. This forms a hybrid: CloudEdge actors for coordination and integration, Fidelity unikernels for compute-intensive operations. A distributed global deployment that runs without Kubernetes is a direction we intend to explore with our customers.

The F* Connection: Proofs Through Shared Heritage

The story of F#’s OCaml influence in our Fidelity framework extends to formal verification. Here, the shared OCaml heritage between F# and F* matters. F* (pronounced F-star) is a proof-oriented language with an extensive pedigree in critical systems verification. Its syntax and semantics are close enough to F# that verification can be integrated into the development process, and the artifacts it produces can support verification with global certification labs.

Because F# and F* share their OCaml lineage, this verification capability integrates through our proof-aware compilation pipeline.

Verification properties guide optimization, not only correctness checks.

When the compiler knows that certain message orderings are impossible, it can eliminate defensive code. When it proves that an actor never exceeds certain memory bounds, it can adjust memory strategies to improve speed. Many such optimizations can improve execution efficiency and memory safety while supporting the developer experience.

Erlang Lessons, F# Innovations

The influence of Erlang on our actor implementations beyond the mailboxprocessor runs deep, and we have weighed it critically. As examined in our analysis of Erlang lessons in Fidelity, we have taken up several of Erlang’s positions while adapting our technical approach to modern technology.

Erlang’s “let it crash” philosophy appears in both Fidelity and Fidelity.CloudEdge but with type-safe refinements. Where Erlang relies on dynamic pattern matching to handle failures, F# uses exhaustive pattern matching with compile-time verification:

type FailureDirective =
 | Restart
 | Stop
 | Escalate

let decideFailureAction (error: exn) : FailureDirective =
 match error with
 | :? TransientException -> Restart
 | :? ConfigurationException -> Stop
 | _ -> Escalate
 // Compiler ensures all cases handled

The supervision hierarchy concept transfers, with architectural adaptations. Erlang’s isolated process model made sense for 1980s hardware where memory protection was expensive. Modern systems offer different trade-offs. Our Olivier actor model, with Prospero’s sentinels and arena allocation, is designed to provide similar fault isolation with better cache utilization. Our CloudEdge design uses platform-managed distribution to reach global scale without direct process marshaling.

Perhaps most significantly, our actor implementations benefit from decades of evolution in type theory and compiler technology. Where Erlang must check message types at runtime, F# verifies them at compile time. Where Erlang’s hot code reloading requires careful coordination, F#’s immutable actors enable blue-green deployments. Where Erlang’s distribution requires EPMD and careful network configuration, Fidelity.CloudEdge leverages Cloudflare’s global infrastructure resiliency.

The Agentic Future: Actors for AI

This principled adaptation carries over to AI agents. As explored in our piece on actors taking center stage, the patterns that Erlang pioneered for telecom systems map onto what modern AI systems need.

A well designed AI agent works as an actor: it maintains state, processes messages (prompts), and interacts with other agents (tools, knowledge bases, other inference sources). The supervision hierarchies that Erlang pioneered to manage telecom switches can orchestrate multi-agent AI systems. The fault tolerance that kept phone networks running for decades can serve AI services that need to operate with high performance and reliability.

type AIAgentMessage =
 | Query of prompt: string * reply: AsyncReplyChannel<Response>
 | UpdateKnowledge of facts: KnowledgeGraph
 | Collaborate of agent: AgentRef * task: Task

type AIAgent() =
 inherit Actor<AIAgentMessage>()

 let knowledgeBase = KnowledgeGraph.create()
 let collaborators = ResizeArray<AgentRef>()

 override this.Receive(msg) = async {
 match msg with
 | Query(prompt, reply) ->
 let! response = this.Reason prompt knowledgeBase
 reply.Reply response

 | UpdateKnowledge facts ->
 knowledgeBase.Merge facts

 | Collaborate(agent, task) ->
 collaborators.Add agent
 let! result = this.CollaborateOn task agent
 return result
 }

These are emerging practices built on proven technologies. The patterns we are designing into our Fidelity framework and CloudEdge toolkit are meant to support agentic architectures. The type safety holds agents to a correct communication protocol. The supervision hierarchies manage agent lifecycles. The distribution mechanisms are designed to carry globally distributed systems with millisecond timing.

Practical Convergence: Theory Meets Implementation

Actor models matter when they translate to practical benefits. Through our designs and conversations with customers we are seeing several convergence patterns.

First, a hybrid architecture pairs our CloudEdge coordination layer with a native framework that handles compute-intensive operations. A CloudEdge supervisor actor might orchestrate dozens of Fidelity-compiled workers running in containers. The supervisor handles routing, load balancing, and fault recovery while the workers perform specialized computations.

Second, the ability to verify actor interactions through F* reshapes how we think about distributed system correctness. Rather than hoping our message protocols are correct, we aim to prove the memory patterns are safe. Rather than testing for race conditions, we aim to prove they cannot occur. The intent is practical engineering that reduces production incidents.

Third, actor-based architectures open up a new economic model. Our CloudEdge actors would scale “to zero” and extend with load. Cloudflare charges for actual message processing through scaling controls it has honed over nearly a decade of global production use. Fidelity unikernel actors can run with minimal resource overhead, allowing dense deployment. Together, they point to a cost model that recasts previously uneconomical applications in terms of both speed and operational integrity.

The Synthesis of Traditions

The path from OCaml’s functional design and Erlang’s actor pragmatism, through F#’s implementations, into our Fidelity framework and CloudEdge toolkit traces a line of convergence across programming traditions and modern technologies.

F# adopted the MailboxProcessor and reimagined it with type safety. Our Fidelity framework is designed to compile actors to native code with deterministic resource management. Our CloudEdge design aims at a development model that carries global deployment. Each step builds on previous insights while adding new capabilities.

As the field takes on agentic challenges in AI orchestration, the patterns established by OCaml and Erlang, refined through F#, and carried into our Fidelity framework and CloudEdge toolkit, give us a foundation to build on.

Erlang pioneered the actor model for telephone switches in the 1980s. OCaml championed type safety and functional design patterns in the same era. Don Syme synthesized those precepts through F#’s pragmatic design in the 2000s. We intend to carry that lineage forward into actor systems design for 2030 and beyond, and we will keep building toward it as the rest of the framework comes into place.

Breaking the P vs NP Mystique

Fri, 26 Sep 2025 00:00:00 -0400

The technology industry has developed an unfortunate habit of wrapping ordinary engineering advances in mystical language. When sites online boast of claims to “blur the lines between P and NP,” they’re usually describing something far more mundane: dealing with technology problems more efficiently. The mathematical complexity remains unchanged, but it shouldn’t be used as a barrier to understanding the practicalities. The recognition that matters here is that most real-world performance barriers come from architectural mismatches, not algorithmic limits.

Our Fidelity framework takes a transparent approach to this challenge. Rather than claiming to solve millennium problems, we show how intelligent compilation can make previously intractable computations practical. The work here is recognizing when we’re using the wrong computational model for the problem at hand.

The Control Flow Trap

To understand why so many problems seem harder than they should be, consider how modern computers execute programs. Everything flows through what we call the von Neumann bottleneck:

\text{Fetch} \to \text{Decode} \to \text{Execute} \to \text{Memory} \to \text{Repeat}

This sequential pipeline forces every computation, no matter how naturally parallel, through a narrow channel of step-by-step execution. When we encounter an NP-complete problem like the traveling salesman, we dutifully encode it as nested loops and recursive functions, then wonder why it takes exponential time. The bottleneck is compilation strategy, not computational complexity.

As we explored in our hypergraph architecture, there’s a duality in how we can represent computation. Control flow treats programs as sequences of instructions, while data flow treats them as networks of dependencies. Many problems that appear intractable under control flow become manageable under data flow, not because the complexity changed, but because we stopped forcing parallel operations through a sequential bottleneck.

The Satisfiability Example

Consider Boolean satisfiability (SAT), the canonical NP-complete problem. Given a logical formula, can we find variable assignments that make it true? Traditional approaches compile this to control flow:

\text{Time} = O(2^n) \text{ where } n = \text{number of variables}

This exponential growth seems inevitable when you’re checking possibilities sequentially. But what if we could check multiple possibilities simultaneously? This is where our interaction nets approach changes the picture. Instead of sequential evaluation, we would compile SAT problems to parallel graph reductions where thousands of possibilities evaluate concurrently.

The mathematical complexity hasn’t changed, since SAT is still NP-complete. But the wall-clock time drops because we’re no longer limited by sequential execution. On an FPGA with spatial computation, what took hours on a CPU might complete in seconds. The problem didn’t become easier; we just stopped making it artificially harder.

Breaking Down the Mystique

When companies claim breakthrough performance on NP-complete problems, they’re typically doing one of three things:

1. Hardware Specialization: Building circuits that naturally express the problem structure. This is what we do with our design for ternary models on FPGAs, where the hardware literally reshapes itself to match the computation.

2. Approximate Solutions: Finding “good enough” answers in polynomial time. Many real-world applications don’t need optimal solutions, just acceptable ones. This doesn’t solve the NP-complete problem; it recognizes that the practical problem is often easier than its theoretical formulation. When considering how LLMs take exactly this approach to generative language constructs, it makes apparent in real terms what approximation can do, within limits.

3. Exploiting Problem Structure: Real-world instances of NP-complete problems often have special properties that make them easier. A delivery routing problem might be NP-complete in theory, but actual road networks have structure that enables efficient solutions.

Our Fidelity framework takes up all three approaches transparently. Through our coeffect analysis, we would identify which strategy applies to each part of your program and compile accordingly.

The Constraint Satisfaction Pattern

Many “hard” problems share a common pattern: they involve finding assignments that satisfy multiple constraints. Whether it’s scheduling employees, routing deliveries, or optimizing portfolios, the underlying structure is similar. Traditional compilation forces us to check constraints sequentially:

\text{Check}_1 \to \text{Check}_2 \to ... \to \text{Check}_n

But constraints don’t inherently require sequential checking. Our Program Hypergraph is designed to recognize when constraints can be evaluated in parallel and to compile them to data flow architectures where all checks happen simultaneously. The speedup follows from matching the execution model to the problem structure.

Real-World Impact

Let’s be concrete about what this means for actual applications:

Scheduling Systems: A hospital scheduling system with 100 nurses and complex constraints might take hours to find valid schedules using traditional approaches. Compiled to interaction nets on an FPGA, the same problem solves in seconds. The constraints haven’t changed, but they’re evaluated in parallel rather than sequentially.

Financial Optimization: Portfolio optimization with thousands of assets and risk constraints appears computationally prohibitive. But as we detailed in our unified vision for heterogeneous computing, compiling to specialized accelerators makes real-time optimization practical.

Route Planning: Delivery routing for hundreds of packages seems to require checking exponentially many possibilities. Yet when compiled to spatial computation architectures, the problem decomposes into parallel subproblems that solve efficiently.

In each case, we’re not defeating mathematics. We’re recognizing that the way we’ve been compiling these problems, forcing them through sequential control flow, creates artificial bottlenecks that make them seem more intractable than they actually are.

The Transparency Principle

Unlike companies that hide behind mystical claims, our Fidelity framework stays transparent about what we’re actually doing. Where we expect dramatic speedups on “hard” problems, the reasons would be these:

We compile to the right execution model data flow vs. control flow based on problem structure
We target appropriate hardware: CPUs for control, FPGAs for data flow, GPUs for parallel, with an eye toward CGRA, neuromorphic and even quantum compute where they contribute in a meaningful way
We preserve mathematical properties that enable optimization that align with the hardware targets where the workloads are deployed at scale
We recognize problem-specific structure that exists in real-world scenarios that preserve alignment to the material factors of the solution space

This doesn’t transcend computational complexity; it engineers through it. As we explored in categorical deep learning, the same mathematical structures that make problems hard in one representation might make them tractable in another.

Democratizing Performance

The real tragedy of the P vs NP mystique is that it convinces engineers their problems are fundamentally hard when they’re often just dealing with solutions that are poorly designed. A small business trying to optimize delivery routes shouldn’t need quantum computers or mysterious “P-NP blurring” technology. They need their existing routing algorithm to run on appropriate hardware.

This is what our Fidelity framework is designed to provide: transparent compilation strategies that make “hard” problems practical. Your code doesn’t change. The mathematical problem doesn’t change. But by compiling to interaction nets, delimited continuations, or spatial architectures as appropriate, we would remove the artificial barriers that make those problem classes seem so daunting.

Moving Beyond the Mystique

The path forward isn’t about solving P vs NP or achieving quantum supremacy on classical hardware. It’s about recognizing that most performance barriers are architectural, not algorithmic. We stop forcing naturally parallel problems through sequential pipelines, we compile to hardware that matches problem structure, and we preserve mathematical properties through compilation. That is when “intractable” problems become practical.

The companies claiming esoteric mathematical acumen are often doing exactly what we’re doing with Fidelity, except they’re not being transparent about it. There’s no shame in good engineering. The shame is in mystifying it to the point where practical solutions seem like magic rather than the natural result of matching compilation strategy to problem structure.

As we continue developing our Fidelity framework, we mean to stay transparent about it. Where we expect dramatic speedups, we will explain exactly how. Where problems remain hard, we will acknowledge why. The work we keep building toward is the work of keeping artificial compilation barriers from making problems harder than they need to be, so that the only complexities we invite are the ones essential to the problem at hand.

How Fidelity Solves The Abstract Machine Model Paradox

Fri, 05 Sep 2025 00:00:00 +0000

The blog post “Abstract Machine Models - Also: what Rust got particularly right” makes a compelling case for Abstract Machine Models (AMMs) as a missing conceptual layer between computer science and hardware. The author, reflecting on a failed microprocessor project, discovers that programmers don’t reason about either programming theory or raw hardware, but rather about intermediate mental models that predict extra-functional behavior: execution time, memory usage, concurrency patterns, energy consumption. These AMMs, the author argues, exist independently of both languages and hardware, explaining how a C programmer can transfer skills to Python despite their semantic differences.

The article presents three design philosophies: machine-first (where new hardware drives new AMMs), second-language (reusing existing AMMs), and AMM-first (defining the mental model to control programmer thinking). This taxonomy seems thorough. Our Fidelity framework and its Composer compiler raise a question the author did not anticipate: what happens when a language naturally embodies the AMM of its compilation target, and the supposed dichotomy between high-level abstraction and machine control no longer holds?

The author’s categorization, influenced by their celebration of Rust as a “second-language” success story (reusing C’s AMM with added safety), leaves room for a fourth possibility: AMM-transcendent design, where a language maintains multiple AMMs simultaneously and selects among them based on semantic analysis and the targeted processor architecture(s). This approach treats the choice as recognizing that different parts of a program align with different execution strategies, rather than committing to a single mental model.

The Composer Compiler’s Opening Gambit

Our Composer compiler does not begin with traditional lexing and parsing into an abstract syntax tree. Instead, it takes a longer path to what was originally dubbed the Program Semantic Graph (PSG), though we now name it the Program Hypergraph (PHG). The PHG is more than an intermediate representation. It is a unified structure that carries semantic, symbolic, and proof information throughout compilation, so each transformation preserves meaning, context, and intent alongside correctness.

The hypergraph supports nanopass compilation, where each transformation is a small, verified step. Because the PHG carries full semantic context, Composer can express multiple AMMs at once. A single Clef function might be viewed as SSA form for traditional optimization, as continuation-passing style for async operations, and as interaction nets for parallel reduction. The compiler is designed to maintain all of these views rather than choose one, selecting the representation based on semantic analysis and, in the design we are building toward, the capabilities of the device the code will run on.

The author of the AMM article would likely object that this violates the principle of AMM independence. How can one language embody multiple incompatible mental models? The answer lies in Clef’s computation expressions and what they reveal about computation itself.

Continuations, Already in the Language

Clef’s async workflows and computation expressions are explicit representations of continuation-passing style, not just syntactic sugar. When a Clef programmer writes an async block, they are thinking in continuations rather than in threads or callbacks. This mental model, this AMM, is the same way MLIR’s new DCont (delimited continuation) dialect represents computation.

The DCont dialect, emerging from CMU research, recognizes that continuations are central to how modern programs execute. Every async operation, every suspendable computation, every cooperative multitasking system manipulates continuations at its core. F#, the lineage Clef descends from, has carried this model in its async workflows for years, and that AMM aligns with MLIR’s execution model.

The picture extends with Martin Coll’s Inet dialect for MLIR. Interaction nets provide a model of computation where parallel reduction is both possible and deterministic. Pure functions can be parallelized automatically without race conditions or synchronization overhead. Clef’s expression-oriented, immutable-by-default style maps to this model. Our language’s AMM does not force programmers to think about parallelism; it is structured so that parallelism emerges from normal code.

The Library of Alexandria

This brings us to our “Alex” component of the Composer compiler, conceptually the Library of Alexandria for transformations. Alex does something the AMM article’s framework treats as out of reach: it uses parser combinator techniques borrowed from Haskell to generate machine-optimized code. Rather than lowering from high-level to low-level, it uses the compositional structure of functional programming to construct efficient machine code directly.

Parser combinators, particularly the XParsec framework, treat code generation as the inverse of parsing. Just as parsers compose small recognizers into larger grammars, Alex composes small code generators into larger optimizations. Each combinator preserves semantic context while producing efficient MLIR operations. The result is machine code that is both optimized and semantically transparent.

The AMM article would classify this as either machine-first (generating for specific hardware) or AMM-first (controlling programmer thinking), but it sits across both. The parser combinator approach provides high-level compositionality while generating low-level efficiency, a synthesis the article’s taxonomy does not account for.

The Rust Lens

The article’s perspective is shaped by Rust’s particular achievements. The author declares that “Rust pushed the Pareto envelope” by combining C’s hardware control with additional safety guarantees. This is true as far as it goes, but it carries a constrained view of what is possible. Rust’s contribution was showing that you could add safety to C’s AMM without losing control. The further question is whether the next step is adding safety to imperative programming, or moving past the imperative model while keeping memory safety and hardware efficiency.

The author’s Rust-centric view carries several assumptions that limit the analysis. First, the article treats Rust’s mandatory memory management as the pinnacle of systems programming, where “every line of code must consider ownership and borrowing.” In our BAREWire approach, memory management is fully present while at design time it remains optional rather than mandatory. This lets developers focus on business logic and engage with memory concerns only where it delivers performance benefits, which is intentional control with an informed default rather than an abdication of responsibility.

Second, the article assumes that the control/guarantees tradeoff is two-dimensional, with Rust having pushed the frontier as far as it can go. This leaves out a third dimension: abstraction level. Rust operates at a fixed abstraction level, forced by its direct compilation to LLVM. Our use of MLIR opens multiple abstraction levels at once, holding high-level intent while generating efficient code and several avenues to express memory management. The Pareto frontier here is not only control versus guarantees. It also covers holding semantic detail through compilation to maximize the efficiency of compute on the targeted processor.

What MLIR Changes

The emergence of MLIR challenges the AMM article’s assumption that there is a fixed set of AMMs that programmers use. MLIR’s dialect system means new AMMs can be created, composed, and transformed. The DCont and Inet dialects are not just implementation details. They are established mathematical models built for efficient expression on sympathetic hardware. The theory has existed for more than a generation (delimited continuations since the 1980s, interaction nets since 1990), and current technology can express them in a way that newer architectures now make practical.

Clef aligns with these models by mathematical correspondence rather than by deliberate design. Our language’s emphasis on expressions over statements, its explicit handling of effects through computation expressions, and its type system that preserves information through compilation make it well suited to MLIR’s compilation model.

Consider how Clef expresses what MLIR represents: async workflows map to DCont’s continuation points, pure functions map to Inet’s interaction rules, and sequential computations decompose into SSA form where each binding creates a new immutable value. The programmer writes Clef thinking in one semantic model, and that model decomposes into the multiple AMMs that MLIR supports. Clef holds semantic coherence, one language and one mental model, while that single model projects onto different platforms as needed.

Heterogeneous Hardware

The AMM article’s most telling limitation is an assumption: that a program runs on a single type of processor. This one-program-to-one-chip framing made sense when the choice was between CPU and GPU, but it does not match the current processor landscape. The author’s focus on Rust as the exemplar of modern systems programming shows why this gap exists, since Rust’s ownership model assumes von Neumann architecture with linear memory. Compile Rust to an FPGA’s dataflow architecture or a neuromorphic chip’s spike-based computation, and the impedance mismatch becomes apparent.

FPGAs operate on dataflow, neuromorphic chips compute with spikes, CGRAs reconfigure their architecture per workload, and quantum processors manipulate probability amplitudes. Each demands a different AMM. Rust’s borrow checker, for all its contributions, has no model for spatial computation, where data flows through configured circuits and “borrowing” has no referent. It has no model for spike-timing-dependent plasticity, where information rides in temporal patterns rather than memory values.

Our Program Hypergraph supports multiple AMMs, and the design we are building toward is meant to let the compiler partition a solution across heterogeneous hardware. Consider a modern AI workload: pattern discovery might run on neuromorphic processors, matrix operations on GPUs, control logic on CPUs, and stream processing on FPGAs. The hypergraph carries both control flow and data flow views, which lets the compiler identify which portions of code align with which hardware execution models.

The article misses a consideration we treat as central: abstraction is what enables hardware-aware compilation. The author’s Rust-influenced view treats abstraction as something to be fought against or carefully controlled. The hypergraph points the other way. Abstraction that carries semantic detail also exposes computational opportunities, data dependencies, and patterns. It supplies enough information for the compiler to make process-mapping decisions, rather than hiding hardware details. The lowering passes do more than translate to machine code. They analyze the hypergraph to determine which path through the hardware constellation will maximize each chip’s contribution.

Control Flow versus Data Flow

The distinction between control flow and data flow is more than an implementation detail. It is the key to reading modern computing paradigms. Traditional processors execute control flow: branching, looping, sequential execution. FPGAs, CGRAs, and neuromorphic processors operate on data flow, with values streaming through configured pathways and triggering computations as they arrive.

This is where the article’s Rust-colored view becomes most limiting. Rust’s ownership model is about control flow: who owns data, when it can be borrowed, when it is dropped. That fits CPUs with their von Neumann architecture. It does not transfer to an FPGA’s spatial computation. There is no “owner” of a signal propagating through configured logic blocks. There is no “borrowing” when data flows through a systolic array. The conceptual framework stops applying.

Our Composer compiler carries both CFG (Control Flow Graph) and DFG (Data Flow Graph) views within the same hypergraph structure. A single Clef function might compile to control flow for CPU execution, data flow for FPGA implementation, or spike trains for neuromorphic processing. The compiler holds both views and selects based on the target hardware and the computation’s inherent structure rather than forcing a choice.

This is why our approach of making design-time memory management optional rather than mandatory (as explored in Memory Management by Choice) carries weight here. In Rust, every function signature must declare its ownership semantics: taking ownership, borrowing immutably, or borrowing mutably. On an FPGA those concepts have no referent. Data has flow paths, not ownership. Memory is accessed through spatial locality, not borrowed. By making memory management an optional optimization rather than a mandatory concern, our framework can target architectures where Rust’s core abstractions fight the targeted processor’s reality.

A Heterogeneous Future in Action

Consider a theoretical financial risk calculation combining Monte Carlo simulation with pattern recognition. In the AMM article’s world, this would run on either a CPU (control flow) or GPU (data parallelism). Composer’s hypergraph analysis would surface a richer structure:

let calculateRisk portfolio market =
 // pattern matching on historical data: neuromorphic
 let patterns = detectAnomalies market.history

 // Monte Carlo paths: GPU parallel
 let paths = generateScenarios market.volatility

 // optimization as a dataflow pipeline: FPGA
 let optimal = findMinimumRisk portfolio paths

 // regulatory proofs: CPU, verifier-discharged
 let proof = verifyCompliance optimal patterns

 (optimal, proof)

The hypergraph would capture that detectAnomalies is sparse pattern matching (suited to neuromorphic), generateScenarios is embarrassingly parallel (GPU), findMinimumRisk is a dataflow pipeline (FPGA), and verifyCompliance needs sequential proof construction (CPU). This future compiler would not force the entire computation onto one processor when the targeted platform supports those devices. It would partition the workload based on each component’s structural fit.

Our framework’s design targets our patent-pending BAREWire implementation, which is meant to provide zero-copy data movement across these processors through several mechanisms. In the design we are building toward, each piece would be produced for its target and the framework would assemble them with high-speed communication.

The Paradox Resolved

The AMM article presents a useful initial framework for how programmers might choose to think about computation. It identifies that mental models matter more than either programming language theory or hardware details. It then assumes these models are fixed, independent entities held in tension with each other.

Our Fidelity framework and Composer compiler point to a different reading: AMMs are interconnected rather than independent. A language like Clef that makes effects explicit, continuations transparent, and parallelism eager is not choosing one AMM over another. It treats these models as projections of a shared computational context that compilation must preserve.

The Program Hypergraph is more than a data structure. It is a recognition that all these models (control flow graphs, dataflow graphs, continuation trees, interaction nets) are different views of the same computation. The nanopass compilation strategy is more than an implementation technique. It treats transformations between AMMs as tractable, verifiable, and bidirectional.

When the article asks why programmers would choose between functionally equivalent sorting algorithms, it asks the right question with too narrow an answer. Programmers do use AMMs to reason about performance. With the right language and compilation strategy, they would not have to choose. A well-designed compiler can carry multiple models, analyze the code’s semantic intent, and select the execution strategy that fits the targeted processor.

The Future They Didn’t See Coming

The AMM article, written in 2022, reflects a public discussion still dominated by von Neumann “reflexes”. The computational landscape is shifting. Tomorrow’s workloads will demand parallel execution and also different computational paradigms. The next AI waves will be non-quadratic and will routinely use sparse spike-based inference. Scientific computing will continue to need both deterministic precision and probabilistic approximation. Embedded systems will balance continuous pressure to produce real-time data under increasing power constraints.

These are divergent paths of computing rather than variations on a theme. The article’s framework, shaped by its Rust-influenced perspective, assumes programmers must choose one mental model and accept its constraints. Modern applications that ask developers to consider the hardware they run on do not fit neatly into a single constrained paradigm. A computer vision system might need neuromorphic processing for feature detection, FPGA dataflow for convolution, GPU parallelism for training, and CPU control for orchestration, all within the same application.

Our Composer compiler’s hypergraph approach points to a different direction. By carrying semantic, symbolic, and proof information throughout compilation, it is designed to target different computational paradigms within a single program, not only different processors. The compiler analyzes which portions of code the targeted hardware can best express.

The underlying technologies exist today. What has been missing is a compilation strategy that can use them coherently. The hypergraph is meant to supply that: a representation rich enough to capture the full range of computational models while carrying the semantic information needed to map workloads to appropriate hardware.

Abstraction Enables Optimization

The article identifies that AMMs mediate between languages and hardware, but it treats abstraction as a barrier to optimization. This perspective, influenced by Rust’s philosophy of explicit control, sets aside a point worth examining. The author celebrates Rust for exposing “access to the largest diversity of I/O interactions” while providing safety guarantees, as if maximum exposure to hardware details is the goal. That exposure may itself be limiting.

Consider what Rust achieved: it took C’s imperative, von Neumann-centric AMM and added memory safety. That was a notable step in 2015, and a conservative one, since it accepted C’s model of computation as the baseline. The author’s claim that Rust provides “intuition about hardware behavior that’s at least as good as C’s AMM” carries the limitation with it. C’s 1970s-era model of sequential memory access is a weak benchmark for hardware intuition in an era of heterogeneous, parallel, and quantum processors.

Our Fidelity framework points the other way: abstraction that carries semantic intent is what makes optimization possible. Preserve semantic intent through compilation, and the compiler can make informed decisions about hardware mapping. Carry proof obligations alongside code transformations, and it can verify correctness against the targeted hardware. Represent computations as hypergraphs rather than trees, and optimization opportunities that sequential syntax would hide become visible.

The author’s failed microprocessor project showed the importance of AMMs. It may also have shown something further: the problem was not that languages and hardware were misaligned, but that the compilation technology to bridge them was not yet in place. MLIR’s dialect system, combined with Clef’s computational model and our Composer compiler’s structural detail, supplies that bridge. Where Rust’s direct LLVM compilation commits to a single lowering path, MLIR allows progressive refinement through multiple abstraction levels.

Our framework does more than compile Clef to native code. It treats the supposed tradeoffs between abstraction and efficiency as artifacts of limited compilation technology rather than facts about computation. The hypergraph holds multiple computational views at once. MLIR’s dialect system allows progressive refinement through abstraction levels. Proof obligations flow through optimizations. These are aspects of one compilation strategy that loosens the old constraints.

The article was right that AMMs matter, and Rust did push important boundaries. The author did not account for what MLIR and hypergraph compilation would make possible. With our Fidelity framework and Composer compiler, programs would have the potential to span heterogeneous processors through principled targeting rather than ad hoc workarounds. We have found no other representative implementations of this AMM-transcendent approach in the standing literature we have reviewed. Abstraction supports optimization rather than fighting it. Multiple AMMs coexist in the same program, selected based on semantic analysis and the hardware’s capabilities.

This direction runs through our Fidelity framework and the Clef language, which descends from F# and the wider ML lineage whose OCaml branch was used to bootstrap Rust itself. The structures functional programming has carried for decades may have been waiting for the right compilation technology, and our current interest lies in building that technology out as the work continues.

The Advent of Neuromorphic AI

Thu, 21 Aug 2025 10:00:00 +0600

Transformers have delivered broad capabilities, and their energy consumption scales with that reach. The human brain operates on roughly 20 watts, processing large volumes of information through sparse, event-driven spikes, at least as we currently understand it. Current AI systems consume thousands of watts to support narrow inference capabilities, forcing dense matrix operations through every computation. That gap is the starting point for this design.

Spiking Neural Networks (SNNs) take a different path, one that neuromorphic processors have begun to realize in silicon. Despite decades of research and steady hardware progress, SNNs remain difficult to train and deploy. As with many algorithmic methods, efficient and accurate gradient calculation has been the constant challenge. For those who have worked in the field for years, the core question is how to compute gradients through discrete, non-differentiable spike events.

This post works through a convergence of three ideas: ternary number systems, forward-mode gradient computation, and spiking neural processing. Coupled with our Fidelity framework design, they bring heterogeneous architectures into a single coherent compilation target. We theorize that this combination lets the multi-state hardware already in silicon do work the current binary algorithms leave on the table.

Ternary Representation: Beyond Binary States

Modern spiking neural network algorithms, as described in the Multi-Plasticity Synergy Learning (MPSL) framework¹, operate on a binary principle despite running on far more capable hardware. The Leaky Integrate-and-Fire equation from the paper defines spike generation as:

S^{t,l} = \Theta(U^{t,l} - V_{th}) = \begin{cases} 1, & U^{t,l} \geq V_{th} \\ 0, & U^{t,l} < V_{th} \end{cases}

This binary representation, spike (1) or silent (0), has been the algorithmic convention in neuromorphic computing, not because of hardware limitations, but due to historical precedent and the mathematical challenges of training. Some neuromorphic processors like Intel’s Loihi 2 support graded spikes with up to 32-bit payloads, programmable neuron models, and thousands of states per neuron. The hardware already runs ahead of the theoretical conventions. Even the MPSL framework, which combines multiple learning mechanisms (Spatio-Temporal Backpropagation or STBP for gradient-based learning, Hebbian plasticity for correlation-based local learning, and Self-Backpropagation or SBP for local feedback without explicit gradients), constrains itself to binary representations despite the hardware’s richer capabilities.

Lessons from Biology

Biological neurons exhibit richer dynamics, and that detail contributes to the model. Between rest and firing, neurons spend time in distinct computational regimes, processing information without generating spikes. A binary SNN cannot represent that intermediate processing at all.

Consider what happens in the binary model. A neuron accumulating toward threshold carries temporal information about recent inputs, and that information vanishes the moment we sample its state. If the membrane potential is at 0.9 × threshold, the binary representation sees only “0”, identical to a neuron at rest. This discretization discards the temporal information that drives spike-timing computation.

The Computational Regime Model

Our encoding separates the continuous membrane potential dynamics from the discrete computational regimes that neurons occupy. Biological neurons carry a continuous voltage, and they also occupy distinct operational modes based on that voltage:

Silent/Resting: Near the resting potential (typically -70mV), the neuron is minimally responsive, with leak currents dominating
Active/Integrating: Depolarized but below firing threshold (between -55mV and -40mV), actively accumulating and processing inputs
Spiking/Firing: Above threshold, generating output spikes

This biological reality maps naturally to a ternary encoding that captures computational regime, not just voltage:

\text{TernaryState} = \begin{cases} 0 & \text{Silent (near resting potential)} \\ -1 & \text{Active (integrating, between thresholds)} \\ +1 & \text{Spiking (above firing threshold)} \end{cases}

The continuous membrane potential $U$ maps to discrete states via two thresholds:

$\theta_{active}$ : Transition from silent to active integration
$\theta_{fire}$ : Spike generation threshold

This preserves critical information about neurons actively integrating inputs ( $U \in [\theta_{active}, \theta_{fire})$ ) that binary representations discard.

Leveraging New Hardware

This observation leads to our core design choice: expand the algorithmic state space to match what the hardware already supports. The MPSL paper advances the field through multiple learning mechanisms, and it follows the convention of binary spike representation, which leaves multi-level hardware states unused. Our ternary encoding targets the multi-level states these processors already provide.

This requires no hardware modification. It uses the hardware as designed. Intel’s Loihi 2 can represent 4096 states per neuron, and SambaNova’s RDU can reconfigure for arbitrary word-level operations. The active state (-1) captures neurons that are integrating inputs but have not yet reached firing threshold, preserving the temporal context that binary algorithms discard.

type TernarySpikingNeuron = {
 Potential: Posit<16, 1> // continuous membrane potential
 RestingPotential: float32 // baseline (-70mV)
 ActiveThreshold: float32 // activation begins (-55mV)
 FiringThreshold: float32 // spike generation (-40mV)
 State: TernaryState // discrete computational regime
}

let computeState (potential: Posit<16,1>) (neuron: TernarySpikingNeuron) =
 match Posit.toFloat32 potential with
 | p when p >= neuron.FiringThreshold -> Spiking // +1
 | p when p >= neuron.ActiveThreshold -> Active // -1
 | _ -> Silent // 0

let updateNeuron (neuron: TernarySpikingNeuron) (input: float32) =
 // Leaky Integrate-and-Fire, continuous domain
 let leak = (neuron.Potential - neuron.RestingPotential) * leakRate
 let newPotential = neuron.Potential - leak + Posit.fromFloat32 input

 let newState = computeState newPotential neuron

 // reset only after spike
 match newState with
 | Spiking ->
 { neuron with
 State = Spiking
 Potential = Posit.fromFloat32 neuron.RestingPotential }
 | otherState ->
 { neuron with
 State = otherState
 Potential = newPotential }

stateDiagram-v2
Silent --> Active: Potential > θ_active
Active --> Spiking: Potential > θ_fire
Spiking --> Silent: Reset to resting
Active --> Silent: Leak below θ_active
Silent --> Silent: Remain near rest
Active --> Active: Integrate inputs

Breaking the Backpropagation Dependency

The Surrogate Gradient Problem

The MPSL paper, like virtually all modern SNN training approaches, relies on surrogate gradients to handle the non-differentiable spike function. As the paper states in Equation 6:

\frac{\partial S^{t,l}}{\partial U^{t,l}} \approx u'(U^{t,l}, V_{th})

This approximation replaces the undefined gradient with a smooth surrogate function. The substitution introduces instability and limits learning efficiency, because the spike function it stands in for is not smooth. Every major SNN training method depends on this workaround, including STBP, BPTT, and the MPSL approach.

The Forward Gradient Approach

Work by Baydin, Pearlmutter, Siskind and Syme² gives an alternative that removes the surrogate entirely. The forward gradient method computes unbiased gradient estimates using only forward-mode automatic differentiation:

g(\theta) = (\nabla f(\theta) \cdot v) v

Where $v$ is a random perturbation vector. This formula has direct consequences for SNNs:

No surrogate needed: The directional derivative $\nabla f(\theta) \cdot v$ can be computed exactly even for discrete spike functions
Single forward pass: Removes the entire backward propagation phase
Unbiased estimator: Its expectation equals the true gradient, so it converges in expectation rather than on any single pass
2x speedup: The paper demonstrates training neural networks up to twice as fast as backpropagation

What Forward Gradients Resolve for SNNs

The forward gradient approach addresses the problem at the center of SNN training. Where the MPSL framework resorts to rectangular surrogate functions (Equation 7 in their paper), forward gradients handle discrete transitions directly:

let computeStateGradient (potential: Posit<16,1>) (thresholds: Thresholds) =
 // directional derivative exists at transition boundaries
 let perturbation = samplePerturbation()
 let perturbedPotential = potential + perturbation

 let originalState = computeState potential thresholds
 let perturbedState = computeState perturbedPotential thresholds

 if originalState <> perturbedState then
 perturbation // sensitivity at boundary
 else
 Posit.zero // no transition

let trainTernarySNN (network: SpikingNetwork) =
 let v = samplePerturbation<Posit<16,1>>()

 // single forward pass over a discrete spike function
 let output, directional =
 Furnace.ForwardMode.evaluateWithDerivative network v

 let forwardGradient = directional * v // unbiased estimate
 updateSynapticWeights forwardGradient

Discreteness does not break the directional derivative. When a perturbation causes a state transition (Silent → Active, Active → Spiking), the derivative captures that sensitivity exactly. When it does not, the derivative is zero. The expectation over random perturbations recovers the full gradient with no surrogate term.

Biological Plausibility Through Global Signals

The forward gradient paper notes that this approach can be interpreted as “feedback of a single global scalar quantity that is identical for all computation nodes”². That maps onto biological neuromodulatory systems:

Dopamine for reward signaling
Serotonin for mood regulation
Acetylcholine for attention modulation

Combined with the MPSL framework’s multiple plasticity mechanisms¹, this gives a learning system whose global scalar feedback has a biological analog, where the weight-transport that backpropagation requires has none.

Hebbian Plasticity Through State Transitions

The forward gradient approach combines with local Hebbian rules keyed to our ternary state transitions:

\Delta w_{ij} = \eta \cdot (\nabla f \cdot v) \cdot P(\text{State}_j | \text{State}_i)

Where weight updates depend on state transition probabilities:

Silent → Active: Potentiation (strengthen connection)
Active → Spiking: Hebbian reinforcement
Spiking → Silent: Refractory adjustment

This extends the MPSL framework’s multi-plasticity approach, which already combines STBP, Hebbian, and SBP mechanisms. Our ternary states give these learning rules more transition information to work with:

let updateSynapticWeights (network: SpikingNetwork) (weight: Posit<16,1>) =
 let v = sampleGaussian<Posit<16,1>>()

 // directional derivative, no surrogate term
 let directional = computeDirectionalDerivative network v

 let gradient = directional * v // unbiased estimate

 // weight by state-transition probability
 match (preState, postState) with
 | (Silent, Active) ->
 weight + learningRate * gradient * potentiationFactor
 | (Active, Spiking) ->
 weight + learningRate * gradient * hebbianFactor
 | (Spiking, Silent) ->
 weight - learningRate * gradient * depressionFactor
 | _ -> weight

Posits: The Natural Language of Membrane Dynamics

The Leaky Integrate-and-Fire equation from the MPSL paper shows why posit arithmetic suits SNNs:

U^{t,l} = \rho_m(U^{t-1,l} - S^{t-1,l}V_{th}) + I^{t,l}

This equation involves:

Exponential decay ( $\rho_m$ )
Threshold comparisons
Accumulation of many small inputs

Posit arithmetic’s variable precision naturally matches these requirements:

High precision near threshold: Where spike/no-spike decisions are critical
Lower precision for strongly polarized states: Where exact values matter less
Exponential representation: Natural for the $\rho_m$ decay factor
Exact accumulation via quire: No rounding errors during integration

let computeMembranePotential (current: Posit<32,2>) (input: Posit<32,2>) =
 use quire = Quire<32, 512>.Zero // exact accumulation

 quire.AddProduct(current, decayRate) // decay current potential

 // weighted inputs accumulate without intermediate rounding
 for synapse in activeSynapses do
 quire.AddProduct(synapse.Weight, synapse.Input)

 quire.ToPosit() // single rounding at the end

Integration with Furnace Auto-Differentiation

The Furnace library, originally developed as ‘DiffSharp’ by the same team behind the forward gradient work (Syme, Baydin, Pearlmutter and Siskind), gives us the forward-mode AD foundation for SNNs:

module Furnace.Neuromorphic =
 let trainSpikingNetwork (network: TernarySpikingNetwork) (data: SensorData) =
 let forwardGradient = furnace {
 let! v = sampleStandardNormal network.ParameterShape

 // single forward pass yields output and directional derivative
 let! output, directional =
 ForwardMode.evaluateWithDirectional network data v

 return directional * v // E[g] = ∇f
 }

 network.UpdateWeights forwardGradient

This reuses what the forward gradient paper demonstrated: training neural networks “without backpropagation” while remaining “computationally competitive”², with up to 2x speedup over backpropagation.

Hardware Capabilities Already in Silicon

Modern neuromorphic processors and Coarse-Grained Reconfigurable Architectures (CGRAs) already carry the capabilities our approach depends on. What they lack is software that targets them.

Intel’s Loihi 2 is not limited to binary spikes. It supports:

Graded spikes with up to 32-bit integer payloads
Programmable neuron models via microcode that can implement arbitrary dynamics
Up to 4096 states per neuron, not just spike/no-spike
Ternary weight matrices already demonstrated in recent implementations

IBM’s TrueNorth, BrainChip’s Akida, and other neuromorphic processors similarly offer programmable models and multi-bit communications. The limit has been the algorithms, not the silicon.

CGRAs for Adaptive Topologies

Coarse-Grained Reconfigurable Architectures from companies like NextSilicon and SambaNova offer further flexibility:

Platform	Architecture	Key Advantage for SNNs
NextSilicon Maverick	Runtime reconfigurable dataflow	Automatically tunes to code patterns, no manual optimization needed
SambaNova RDU	Reconfigurable at each clock cycle	Can morph between neural and conventional processing dynamically
General CGRAs	Word-level reconfigurable arrays	Natural fit for ternary representations and posit arithmetic

As SambaNova describes it, their RDU is “an array of compute and memory on chip” that can be reconfigured to match the computational pattern needed. This suits CGRAs to:

Ternary state machines that map to word-level operations
Posit arithmetic implementations on the reconfigurable compute units
Dynamic network topologies that adapt during runtime
Mixed conventional/neuromorphic workloads in the same chip

Hardware Capability Summary

Feature	Neuromorphic (Loihi 2)	CGRAs (SambaNova/NextSilicon)	Fidelity Framework
State representation	Up to 4096 states/neuron	Arbitrary via reconfiguration	Ternary mapping
Arithmetic precision	8-32 bit configurable	Word-level operations	Posit arithmetic
Learning capability	Programmable plasticity	Runtime adaptable	Forward gradients
Computation model	Event-driven spikes	Dataflow reconfigurable	Both paradigms
Programming model	Microcode/assembly	High-level dataflow	Clef unified abstraction

The Reality of Hybrid Compute

In practice, CGRA and neuromorphic processors rarely operate as the sole component to a solution. They’re deployed in heterogeneous systems as accelerators:

On-die integration: Accelerators alongside conventional CPU/GPU cores
CXL coherent memory: Shared memory spaces between neuromorphic and traditional processors
PCIe accelerators: Accelerator cards working within host systems
Edge hybrids: Low-power neuromorphic/CGRA units paired with DSPs or microcontrollers

Our Fidelity framework design, and the Composer Hypergraph in particular, treats this as a “control flow to data flow” transformation, which is what these heterogeneous deployments need:

[<CompileToNeuromorphic>]
let neuromorphicCore (neurons: TernarySpikingNeuron array) =
 neuromorphic {
 let! target = detectNeuromorphicPlatform()

 match target with
 | Intel_Loihi2 config ->
 configureLoihi config
 | IBM_TrueNorth config ->
 configureTrueNorth config
 | BrainChip_Akida config ->
 configureAkida config
 | Infineon_Neuromorphic config ->
 configureInfineon config
 | FPGA_Emulation config ->
 configureFPGAEmulation config
 | CPU_Simulation fallback ->
 // Graceful degradation to CPU simulation
 configureCPUSimulation fallback

 // Common neuromorphic operations
 return! compileToDataFlow neurons
 }

Platform-Specific Implementation Strategies

As we currently conceive it, our approach maps this design across several hardware architectures:

On Neuromorphic Processors (Loihi 2):

[<CompileToLoihi>]
let ternaryNeuronLoihi (state: int32) (input: int32) =
 // Loihi 2 carries up to 4096 states; we use 3, as microcode
 match state with
 | -1 -> processActive input // state 0-1365
 | 0 -> processSilent input // state 1366-2730
 | 1 -> processSpike input // state 2731-4095

On CGRAs (SambaNova RDU, NextSilicon Maverick):

[<CompileToCGRA>]
let ternaryNeuronCGRA (neurons: TernaryNeuron array) =
 cgra {
 let! pe_array = allocatePEs (neurons.Length)

 for pe in pe_array do
 pe.ConfigureForTernary() // word-level ternary ops
 pe.SetPositPrecision(16, 1) // native posit support

 // dataflow scheduled by the platform
 return dataflowProcess neurons
 }

CGRAs fit here because they can:

Reconfigure arithmetic units for posit operations at runtime
Adapt dataflow patterns based on spike density
Move between neural and conventional processing on the same fabric
Run the forward gradient computation in parallel across PEs

Each learning mechanism operates on appropriate hardware:

STBP (Spatio-Temporal Backpropagation): Gradient-based learning that propagates errors through both space (layers) and time (timesteps)
Hebbian Plasticity: Local learning based on the principle “neurons that fire together, wire together”
SBP (Self-Backpropagation): Local feedback mechanism that approximates gradients without explicit error propagation

The ternary states provide richer information than binary for all three mechanisms. The MPSL paper’s choice to use binary states was algorithmic convention, not hardware necessity.

Each learning mechanism operates independently on parallel cores, then combines via learnable coefficients as described in the MPSL paper:

W^l = \sum_{i=1}^{3} \lambda_i W_i^l

Where $\lambda_i$ are adaptively learned mixing coefficients, optimized through local feedback using forward gradients, not global backpropagation.

Performance Projections

We project the following efficiency gains from combining ternary representations, forward gradient training, and acceleration hardware, measured against both conventional approaches and existing binary SNNs:

Metric	GPU (A100)	Binary SNN (Multi-Plasticity)¹	Ternary + Forward Gradient	Improvement vs GPU
Power (Inference)	400W	50W	1-5W	80-400x
Power (Training)	400W	100W	2-10W	40-200x
Latency (per spike)	10μs	1μs	10-100ns	100-10000x
Training passes	2 (fwd+bwd)	2 (fwd+bwd)	1 (fwd only)	2x
Gradient accuracy	N/A	Surrogate	Exact	No surrogate term
Information preserved	N/A	Binary states	Ternary states	50% more
Biological correspondence	None	Medium	High	Higher

Note: Performance varies by neuromorphic processor and deployment configuration.

The forward gradient approach demonstrated 2x speedup over backpropagation in conventional networks². For SNNs, the advantage is even greater since we eliminate the surrogate gradient approximation entirely.

Roadmap

Phase 1: Foundation

Implement ternary SNN models in our Fidelity framework
Integrate forward gradient training via Furnace
Develop a neuromorphic backend for our Composer compiler
Demonstrate MNIST/CIFAR-10 benchmarks

Phase 2: Hardware Integration

Intel Loihi 2 support with ternary neuron models
BAREWire integration for event streaming
Posit arithmetic emulation on fixed-point units
Heterogeneous CPU-neuromorphic demonstrations

Phase 3: Platform Expansion

Support for IBM TrueNorth, BrainChip Akida
FPGA-based neuromorphic emulation
Cloud deployment with neuromorphic simulation
Edge deployment on heterogeneous SoCs

Phase 4: Applications

Real-time sensor fusion for robotics
Ultra-low-power edge AI
High-throughput inference systems
Continuous learning systems

Where Our Framework Fits

The hardware is already here, and current software addresses only part of it. Current neuromorphic software treats these processors as binary spike generators, using a fraction of their states. CGRAs from NextSilicon and SambaNova are often programmed with conventional approaches that do not use their reconfigurable structure. Our framework targets this gap by:

Using existing hardware features: Ternary states map to the multi-bit spikes and programmable neurons already in silicon
Removing the surrogate approximation: Forward gradients replace the surrogate gradient that has limited SNN training
Providing one compilation target: Clef code that compiles to both neuromorphic and CGRA backends

Platform-Specific Advantages

For Neuromorphic Processors (Intel, IBM, BrainChip):

Use the full state space (4096 states, not just 2)
Encode richer information through graded spikes
Run online learning without backpropagation

For CGRAs (NextSilicon, SambaNova):

Word-level operations for ternary representations
Runtime reconfiguration for adaptive neural topologies
Neural and conventional processing on the same fabric

For Heterogeneous Systems:

Neuromorphic cores for spiking dynamics
CGRA/GPU for dense operations when needed
CPU for orchestration and control flow
All unified through BAREWire’s zero-copy communication

Why These Pieces Line Up Now

Three classes of platform now carry the capabilities advanced SNNs need: neuromorphic processors, CGRAs, and heterogeneous systems that combine them. The remaining gap is the software layer, one that can:

Train these networks without the surrogate gradient approximation
Deploy across diverse hardware without a rewrite per target
Address the full state space and precision of modern silicon

Our Fidelity framework, with forward gradient training, is the design we are building to fill that gap.

Today’s and Tomorrow’s Silicon

The gating constraint is software, not new silicon. Neuromorphic chips carry multiple states per neuron, and the field’s algorithms have used two. CGRAs can reconfigure every clock cycle, and conventional programming treats them as fixed architectures. The hardware is in place, and the algorithms have room to catch up.

We theorize that combining ternary spiking neural networks with forward gradient training, on the classical hardware that already ships, addresses the surrogate gradient problem that has held SNN training back. We have found no other representative implementation of this combination in the standing literature we have reviewed. Matching the encoding to the multi-state hardware is what recovers the efficiency that neuromorphic computing has projected for years.

The pieces this rests on:

Ternary modeling targets the multi-state capabilities already in neuromorphic processors and CGRAs, capturing distinct computational regimes of biological neurons
Forward gradients train without the surrogate approximations that have constrained the field
Posit arithmetic maps to the word-level operations of CGRAs and the programmable precision of neuromorphic chips
Existing hardware from Intel, IBM, BrainChip, NextSilicon, and SambaNova is available today

Our control-flow to data-flow compilation, forward gradient training through Furnace, and platform-agnostic backend are the layer that connects this hardware capability to a working SNN. That is where our current interest lies, and we will keep building the design toward the heterogeneous targets the rest of this post names as the work continues.

Liu, Y., Deng, X., & Yu, Q. (2024). Multi-Plasticity Synergy with Adaptive Mechanism Assignment for Training Spiking Neural Networks. arXiv preprint arXiv:2508.13673v1 ↩︎ ↩︎ ↩︎
Baydin, A. G., Pearlmutter, B. A., Syme, D., Wood, F., & Torr, P. (2022). Gradients without Backpropagation. arXiv preprint arXiv:2202.08587. ↩︎ ↩︎ ↩︎ ↩︎

Categorical Deep Learning and Universal Numbers

Sun, 10 Aug 2025 00:00:00 +0000

A Confession and a Vision

A personal note from the founder of SpeakEZ Technolgies, Houston Haynes

I must admit something upfront: when I began design of the Fidelity framework in 2020, I was driven by practical engineering frustrations, particularly with AI development. The limitations of a managed runtime, the endless battle with numeric precision, machine learning framework quirks, constant bug chasing; these weren’t just inconveniences, they felt like fundamental architectural flaws. So I started building something different, guided more by engineering intuition than mathematical theory. Then I recently encountered the position paper “Categorical Deep Learning is an Algebraic Theory of All Architectures” by Gavranović et al., and experienced that rare moment of recognition: the mathematical foundations for what I have been building already existed.

Like many other of the recent advances and discoveries I’ve made, a significant credit is owed to Paul Snively, for his polyglot perspective that led to many of the connections made as formalism has taken a greater role in the framework. You can see more about him here Paul Snively on Programming Languages, Reliable Code and Good Taste in Software Engineering. While I may have eventually connected the dots on my own, Paul’s decades of lived experience with the practicalities of functional programming, and more specifically with formal verification has been a force multiplier for the speed at which my grasp of the domain has expanded over the past few months. A great deal of credit for this synthesis rests with him, while mistakes and omissions remain my own.

As for the reading, the authors of the CDL paper go through their process (as I currently understand it) of formalizing neural networks as morphisms in a 2-category. This provides a condensed theoretical underpinning I had been assembling piece by piece in the Fidelity framework. It was both humbling and exhilarating; humbling because I had been unknowingly fumbling in the dark where category theory sheds light, but it’s also exhilarating because it validated my framework’s architectural direction in a way that was completely unexpected.

This document represents my attempt to synthesize years of practical framework development with these theoretical underpinnings. It’s forward-looking and aspirational, and shouldn’t be taken as making light of the many technical hurdles still to overcome. I believe it points toward the convergence of classical systems with High-Performance Computing, Artificial Intelligence and Quantum as a single, mathematically unified paradigm.

This would, in effect, provide a coherent framework to explore them all from a single, hardware-aware software platform. I’m sharing a summation of what I’ve learned and the future directions I see.

The Journey So Far

Over the past several years, SpeakEZ has been designing components that form the foundation for a unified vision. Each piece on its own was solving a technical problem, and each was removing a source of computational inefficiency:

The exploration of matmul-free architectures demonstrated that the industry’s obsession with matrix multiplication was more historical accident than mathematical necessity. It showed how ternary quantization and sub-quadratic models could achieve comparable performance with dramatically lower computational requirements, often 10-100x more efficient.

And work on ternary models and heterogeneous computing speculated on how AMD’s unified memory architecture could enable new paradigms for distributed AI inference, with BAREWire providing the zero-copy substrate for efficient model orchestration, eliminating the memory bandwidth bottleneck that consumes up to 90% of cycle wait times in current systems.

The investigation into discriminated unions for post-transformer AI revealed how type-safe heterogeneous representations could better capture the diverse computational patterns emerging in modern architectures, reducing the “representation overhead” that forces complex patterns through inappropriate abstractions.

Insights into hypergraph architecture showed how preserving multi-way relationships enables data-flow computation that can be orders of magnitude more efficient than control-flow paradigms.

Our patent-pending proof-aware compilation design demonstrated that verification doesn’t add overhead; it removes it by enabling aggressive optimizations impossible without formal guarantees.

And the early vision of Fidelity as an AI Refinery established the framework’s role in transforming raw computational capabilities into efficient intelligent systems.

Each of these efforts was solving a specific problem, but looking back, they were all converging on the same observation:

The artificial separation among classical, HPC and AI is holding the industry back.

It also makes each technical domain less efficient than it could be.

The Current Crisis: Divergent Paths

Modern computing faces a long-standing schism. High-Performance Computing (HPC) and Artificial Intelligence (AI) have evolved along divergent paths, each developing its own tools, techniques, and staffing constituencies:

HPC’s Challenges

Verification Burden: Safety-critical simulations require formal proofs
Numerical Precision: IEEE-754 limitations cause accumulation errors
Scalability Walls: Traditional methods hit complexity barriers
Rigid Models: Physics equations can’t adapt to real-world complexity

AI’s Challenges

Black Box Problem: No proofs about model behavior
Numerical Instability: Gradient underflow, training irreproducibility
Semantic Loss: Meaning disappears in tensor operations

These aren’t separate problems; they’re symptoms of the same underlying issue: the lack of a unified mathematical foundation for computation.

The Solution: Categorical Deep Learning

As articulated in the paper by Gavranović et al., neural networks are not just computational graphs; they are morphisms in a 2-category. This insight provides the bridge between HPC’s rigorous mathematics and AI’s adaptive learning.

In mathematical terms, they show that a neural network is a morphism $f: \mathcal{P} \to \mathcal{L}$ where $\mathcal{P}$ is the parameter space and $\mathcal{L}$ is the learner category. Backpropagation itself is the canonical 2-cell:

\text{Para}(\mathcal{P}) \xrightarrow{\text{forward}} \mathcal{L} \xrightarrow{\text{backward}} \text{Para}(\mathcal{P})

Where $\text{Para}$ is the parameterized category construction that enables gradient flow. This is the structure our Fidelity framework is designed to express through the Clef language’s type system.

Why Clef Is a Natural Choice for This Domain

Clef is designed to express these higher-order mathematical structures directly. The functional foundation comes by way of F# and OCaml; the type-safety lineage runs through languages like Rust. Clef carries capabilities aimed specifically at categorical deep learning:

Computation Expressions: Native Categorical Structures

Clef’s computation expressions are a direct encoding of monadic and categorical patterns. Where other languages require extensive type encoding to express categorical operations, Clef expresses them directly:

// A 2-categorical morphism expressed naturally in Clef
type NeuralMorphism<'Input, 'Hidden, 'Output> =
 categorical {
 // Computation expressions model the 2-category structure
 let! layer1 = Morphism<'Input, 'Hidden>
 let! layer2 = Morphism<'Hidden, 'Output>

 // Horizontal composition (functor composition): (g ∘ f)
 let! forward = compose layer1 layer2

 // Vertical composition (natural transformations): α ∙ β
 let! transform = naturalTransform forward

 // The 2-cell (modification between natural transformations)
 return! modification transform
 }

This directly implements the mathematical structure from the CDL paper where neural networks form a 2-category $\mathbf{Learn}$ with:

Objects: Parameterized types (our Clef types with measures)
1-morphisms: Learners (our typed functions)
2-morphisms: Updates/reparameterizations (our gradient transformations)

This requires extensive encoding in OCaml, and the ownership model in Rust constrains the composition that category theory relies on.

Units of Measure: Dimensional Analysis for Free

Clef goes beyond OCaml with zero-cost units of measure that naturally express the dimensional analysis inherent in physical simulations and neural architectures:

// Dimensional correctness in neural architectures
[<Measure>] type neuron
[<Measure>] type layer
[<Measure>] type activation

type CategoricalLayer<[<Measure>] 'input, [<Measure>] 'output> = {
 Weights: Matrix<float<'output/neuron>, float<'input/neuron>>
 Transform: Morphism<'input, 'output>
 Adjoint: ContravariantFunctor<'output, 'input>
}

This dimensional typing keeps our categorical structures tied to physical and mathematical meaning at compile time, which neither OCaml nor Rust expresses as directly.

Type Providers: Bridging Abstract and Concrete

Clef’s type providers generate categorical structures from external schemas, connecting the mathematics directly to real-world data:

// Type provider generates categorical structure from neural architecture
type NeuralArchitecture = JsonProvider<"model.json">

let model = NeuralArchitecture.Load("transformer.json")

// Automatically derived categorical morphisms from architecture
let categoricalModel =
 model.Layers
 |> Seq.map (fun layer ->
 Morphism.fromStructure layer.Type layer.Parameters)
 |> Morphism.compose2Category

This capability extends to emerging standards like the Hypergraph Interchange Format (HIF), which provides a unified JSON schema for higher-order network data.

Our implementation of Clef’s type providers is designed to ingest type-safe representations from HIF-compliant datasets, integrating relational data from co-authorship networks, chemical reactions, or biological interactions directly into HPC simulations and AI training pipelines. This could ease the interchange of data and concepts across academic disciplines and industry verticals.

Active Patterns: Recognizing Categorical Structures

Clef’s active patterns let us recognize and destructure categorical patterns in ways that would require verbose visitor patterns in other languages:

// Recognize categorical patterns in neural networks
let (|Functor|Monad|Adjunction|) morphism =
 match morphism with
 | HasLeftAdjoint adj -> Adjunction(morphism, adj)
 | HasBindOperation bind -> Monad(morphism, bind)
 | _ -> Functor(morphism)

// Use pattern matching to optimize based on categorical structure
let optimize = function
 | Adjunction(f, g) ->
 // Exploit adjunction for perfect backpropagation
 optimizeAdjoint f g
 | Monad(m, bind) ->
 // Use monadic structure for sequential optimization
 optimizeMonadic m bind
 | Functor f ->
 // Standard functorial optimization
 optimizeFunctor f

Quotations: Preserving Mathematical Semantics

Clef’s quotations preserve the mathematical structure of expressions, enabling us to analyze and transform categorical operations at compile time:

// Quotations preserve categorical structure for analysis
let neuralOperation =
 <@ fun (input: Tensor<'n, 'd>) (weights: Morphism<'d, 'h>) ->
 categorical {
 let! forward = weights.Apply input
 let! backward = weights.Adjoint forward
 return (forward, backward)
 } @>

// Analyze the categorical structure at compile time
let structure = analyzeCategoricalStructure neuralOperation
// Generates optimized MLIR based on mathematical properties
let optimizedMLIR = compileToMLIR structure

Beyond Functional: The Engineering Bridge

What sets Clef apart is its pragmatic bridge to software engineering reality. We carefully and selectively extend that with specific design choices in the Fidelity framework:

Shared Edges with .NET: We have gone to great lengths to preserve F# idioms in our framework. By extension this will offer many “shared edges” with .NET based solutions, allowing teams currently using F# for machine learning and HPC workloads a gradual transition path with manageable source modifications, including pathways for implementing classical compute with higher integrity and efficiency.
Mutable Optimization: When needed, Clef allows controlled mutation which Fidelity framework and Composer compiler leverages for performance-critical sections without breaking the categorical abstraction. This hybrid approach, detailed in our reactive framework design (now absorbed into CCS), presents developers with pure, immutable interfaces while allowing the compiler to selectively introduce mutation based on scope analysis in the computation graph. This “immutability at design time, verified mutation at runtime” strategy means the categorical abstractions remain pure for reasoning and composition, while achieving the same performance as hand-optimized imperative code. The compiler’s scope analysis ensures mutations only occur when mathematically equivalent to the pure version, preserving all categorical properties while eliminating allocation overhead.
True Concurrency & Parallelism: Clef’s async expressions naturally model the parallel structure of categorical compositions, but as we explored in The Full Frosty Experience, this goes far beyond traditional managed runtime implementations. Through delimited continuations, Frosty transforms async computations into explicit categorical morphisms that can be verified, traced, and compiled to platform-native code without runtime overhead. The delimited continuations make the “rest of the computation” a first-class value that can be inspected, transformed, and verified, turning what was once managed runtime magic into compile-time certainty.
Interop: The FFI in Rust is verbose and the ecosystem in OCaml is narrower, where Clef provides direct interop with C/C++ libraries, extended through our Farscape CLI tool. As detailed in Farscape’s Modular Entry Points, this goes beyond simple bindings; we can generate drop-in replacements for tools like OpenSSL that maintain API compatibility while adding type safety. The established ecosystem of HPC libraries, from PETSc for scientific computing to FFTW for signal processing, becomes available with Clef’s type safety. The categorical structures we’re implementing integrate with numerical libraries that have been optimized over decades. We have found no other functional language that pairs this combination in the standing literature we have reviewed: expressing 2-categorical morphisms while calling directly into established HPC kernels, with compile-time type safety and no additional overhead.

Clef is a strong fit for implementing categorical deep learning: it combines the expressiveness to represent 2-categories with the engineering capabilities to deploy them at scale. OCaml carries the theory with a narrower ecosystem. Rust carries the performance with friction against the abstractions. Haskell carries the categories with a harder interop story. Through close alignment to F# idioms and our Composer compiler, the Fidelity framework is designed to bridge these worlds.

Quantum Computing: The Natural Beneficiary

As we explored in our quantum optionality analysis, this 2-categorical foundation is not limited to classical computation. Quantum computing emerges as a beneficiary of the same mathematical framework, and the alignment follows from the mathematics rather than from forcing a fit.

Quantum computations are categorical. Quantum circuits are morphisms, quantum gates are natural transformations, and quantum mechanics is expressed in the language of monoidal categories. The same 2-categorical structure that unifies HPC and AI extends to quantum:

// Quantum operations ARE 2-categorical morphisms
type QuantumMorphism<'Input, 'Output> =
 | Unitary of U: UnitaryOperator<'Input, 'Output> * U†: Adjoint<'Output, 'Input>
 | Measurement of Projector<'Input, Classical<'Output>>

// The SAME categorical structure works for quantum
let quantumCategorical = categorical {
 // Prepare quantum state (functor)
 let! prepared = QuantumFunctor.prepare classicalData

 // Apply quantum circuit (morphism composition)
 let! evolved = QuantumMorphism.compose [
 Hadamard
 CNOT
 PhaseShift(π/4)
 ]

 // The adjoint is built-in (2-category structure)
 let! adjoint = evolved.Adjoint

 // Measurement (natural transformation to classical)
 return! Measurement.collapse evolved
}

This is not shoe-horning quantum into our framework; the mathematical foundations of quantum mechanics are categorical. Unitarity describes morphisms that compose with their adjoints to give identity. Entanglement describes the monoidal structure of tensor products. The language of quantum mechanics is the language of category theory.

Our categorical foundation for unifying HPC and AI is designed to encompass quantum without modification. The same Clef computation expressions that model neural network training are meant to model quantum circuit evolution. The same proof systems that verify conservation laws are meant to verify quantum unitarity. The same Universal numbers that handle HPC precision are meant to handle quantum amplitudes. Microsoft Research’s own work to create the Q# language from F# points to that alignment.

While others are building separate classical and quantum stacks, hoping to integrate them later, our categorical approach is designed as a single framework that covers all four paradigms.

Organizations using Fidelity platform won’t need to adopt new abstractions or rewrite their systems for quantum-classical hybrid workloads.

In our design, the Fidelity framework would offer that degree of freedom by adding another backend to the same software semantics.

Implementing the Core Insight

With those Clef capabilities in place, we can express the unified view directly:

// The unified view: All computation as categorical morphisms
type UnifiedComputation<'Input, 'Output> = {
 // Forward computation (HPC simulation OR AI inference)
 Forward: Functor<'Input, 'Output>

 // Backward computation (Adjoint methods OR backpropagation)
 Backward: ContravariantFunctor<'Output, 'Input>

 // the duality: Forward ⊣ Backward
 Adjunction: AdjointPair<'Input, 'Output>

 // Preserved invariants (Conservation laws OR learned constraints)
 Invariants: Set<CategoricalProperty>
}

and AdjointPair<'Input, 'Output> = {
 // unit of the adjunction: η: Id → G∘F
 Unit: 'Input -> 'Input

 // counit of the adjunction: ε: F∘G → Id
 Counit: 'Output -> 'Output

 // triangle identities, verifier-discharged
 Certificate: Z3Certificate
}

// Clef supports custom operators for the mathematical notation
let inline (⊣) forward backward =
 { Unit = fun x -> backward.Apply(forward.Apply x)
 Counit = fun y -> forward.Apply(backward.Apply y)
 Certificate = checkAdjunction forward backward } // Z3 discharges the triangle identities

let computation = {
 Forward = myForwardFunctor
 Backward = myBackwardFunctor
 Adjunction = myForwardFunctor ⊣ myBackwardFunctor // Optional operator syntax
 Invariants = Set.ofList [EnergyConservation; MomentumConservation]
}

This implements the fundamental theorem from CDL: every differentiable function $f: A \to B$ gives rise to an adjunction:

\text{Fwd}_f \dashv \text{Bwd}_f : \text{Para}(A) \rightleftarrows \text{Para}(B)

Where the forward pass $\text{Fwd}_f$ and backward pass $\text{Bwd}_f$ form an adjoint pair. In HPC, this manifests as the adjoint method for sensitivity analysis. In AI, it’s backpropagation. In quantum computing, it’s the unitary conjugate.

The mathematics are identical. Only the substrate differs.

This is a blueprint for unification that Clef is designed to express through its computation expressions, type providers, and quotation system. As we explored in our Beyond Transformers work, the shift away from matrix multiplication opens the door to other representations. Category theory provides that representation, and Clef provides the engineering vehicle.

Key CDL Principles Applied to HPC+AI

The CDL paper establishes four principles that our Fidelity framework is designed to implement:

Compositional Structure: Both simulations and neural networks compose functorially
$f: A \to B, \quad g: B \to C \quad \Rightarrow \quad g \circ f: A \to C$
Duality: Every forward computation has a dual (adjoints in HPC, gradients in AI)
$\text{Forward}: \mathcal{C} \to \mathcal{D} \quad \dashv \quad \text{Backward}: \mathcal{D} \to \mathcal{C}$
Algebraic Properties: Conservation laws and weight tying emerge from the same structures
$\text{Invariant}(f \circ g) = \text{Invariant}(f) \wedge \text{Invariant}(g)$
Equivariance: Symmetries in physics and neural architectures share mathematical roots
$\rho(g) \cdot f(x) = f(\rho(g) \cdot x) \quad \text{for all } g \in G$

These aren’t separate implementations in Fidelity; they’re different views of the same categorical structure encoded in our Clef type system.

Universal Numbers: Solving the Numerical Problem

The Universal Numbers Library provides the numerical foundation that makes both HPC and AI practical at scale. This addresses one of the core challenges we identified in our ternary models exploration: maintaining numerical fidelity across heterogeneous computing environments.

Posit Arithmetic: The Best of Both Worlds

While Fidelity is rooted in Clef, the Universal Numbers library exists as optimized C++ code that we integrate through our compilation pipeline. MLIR itself is implemented in C++, and when our Clef code lowers through the compilation stack, it interfaces with these numerical primitives:

// C++ Universal posit - perfect for both domains
template<unsigned nbits, unsigned es>
class posit {
 // Tapered accuracy: More precision near 1.0 (AI's operating point)
 // Wide dynamic range: Handles HPC's extreme scales
 // Exact accumulation: Via quire for reproducibility
};

Think of this as the numerical “engine” that drives our type-safe Clef abstractions. Just as you don’t need to understand the assembly instructions your CPU executes, you interact with posits through Clef’s type system while the C++ implementation handles the bit-level arithmetic. The MLIR lowering strategy is designed so that the C++ posit operations become direct hardware instructions rather than library calls.

Clef Type-Safe Integration

Building on our work with discriminated unions, we can create type-safe numerical representations that respect the categorical structure:

// Type-safe wrapper preserves semantics
type Posit<[<Literal>] nbits: int, [<Literal>] es: int> =
 private | PositValue of uint64

 // For HPC: Preserve conservation laws
 static member ConservationSum (values: Posit<'n,'e> array) =
 use quire = Quire<'n, 512>.Zero
 for v in values do quire.Add(v) // Exact!
 quire.ToPosit() // Single rounding

 // For AI: Stable gradients
 static member StableGradient (loss: Posit<32,2>) =
 // Tapered accuracy prevents underflow
 Gradient.compute loss

This connects to the CDL paper’s requirement for a symmetric monoidal category with biproducts. The Universal numbers provide the numerical semiring $(\mathbb{R}, +, \times)$ with the crucial property that our morphisms preserve the algebraic structure:

\text{Hom}_{\text{Posit}}(A \oplus B, C) \cong \text{Hom}_{\text{Posit}}(A, C) \times \text{Hom}_{\text{Posit}}(B, C)

This biproduct structure is essential for gradient decomposition and is automatically preserved by our type-safe implementation.

Formal Verification: Provable Numerics

The Missing Link

F* and Z3 provide the formal verification layer for the unified framework. The point often missed is that proofs don’t just ensure correctness; they inform optimization patterns that can be up to 100x more efficient. This extends the work we outlined in Transforming AI Efficiency where we show that proofs are also lowered in MLIR to execute through SMTLIB.

Proof-Carrying Code in the Hypergraph

This abstract should be considered an advanced example, something that would be opt-in, going above and beyond the MISRA-C-class proofs that would ride along with most classical Clef code in this framework. But it’s an example of how extensible the framework can be when the use case calls for this level of specialization.

// Hypergraph edges carry both algorithmic and numerical proofs
type ProofHyperedge =
 // HPC proofs
 | ConservationProof of system: Node * law: ConservationLaw * cert: Z3Certificate
 | StabilityProof of solver: Node * condition: StabilityCondition * cert: SMTProof

 // AI proofs
 | ConvergenceProof of training: Node * bound: ConvergenceBound * cert: SMTProof
 | RobustnessProof of model: Node * perturbation: Epsilon * cert: Z3Certificate

 // Unified proofs
 | NumericalExactnessProof of computation: Node * error: ErrorBound * cert: Universal
 | CompositionCorrectnessProof of components: Node list * property: Property * cert: SMTProof

Beyond Moore’s Law: The Data-Flow Advantage

Traditional Von Neumann and Modified Harvard architectures force a control-flow paradigm where computation and memory are separated, creating endless cycles of fetch-decode-execute with associated wait states and heat dissipation. As we explored in our hypergraph architecture, data-flow representations can be significantly more efficient because computation happens where the data lives.

This isn’t theoretical. I’ve witnessed the inverse cost of this firsthand in digital audio engineering, where 16-bit representation for Compact Disc authoring required heroic efforts to “dither” floating point truncation. The computational gymnastics needed to mask quantization noise consumed more engineering time than the actual audio post-production flow. We’d spend 90% of our cycles compensating for representation limitations. The parallel with current AI/HPC is stark: we waste enormous computational resources managing the impedance mismatch between our mathematical intentions and our computational substrates.

Proofs as Optimization Catalysts

As detailed in our proof-aware compilation work, mathematical proofs don’t just verify correctness; they reveal optimization opportunities invisible to traditional compilers:

// Traditional approach: defensive programming with runtime checks
let traditional_matrix_multiply A B =
 // Runtime dimension checks
 assert (A.Cols = B.Rows)
 // Bounds checking on every access
 for i in 0..A.Rows-1 do
 for j in 0..B.Cols-1 do
 for k in 0..A.Cols-1 do
 // Each access checks bounds
 result.[i,j] <- result.[i,j] + A.[i,k] * B.[k,j]

// Categorical approach with proofs
let categorical_matrix_multiply
 (A: Matrix<'n, 'm>)
 (B: Matrix<'m, 'p>)
 : Matrix<'n, 'p> =
 // dimensions checked at compile time; proofs remove the runtime bounds checks
 categorical {
 return! matmul A B
 }

The proofs give the compiler permission to optimize aggressively. This connects to the CDL paper’s observation that gradient flow is a natural transformation $\eta: \text{Id} \Rightarrow T$ where $T$ is the gradient operator. When we prove properties about $\eta$ , we’re proving properties about the available optimizations:

\text{Optimize}(f) = g \quad \text{iff} \quad \eta_f = \eta_g \text{ and } \text{Invariants}(f) = \text{Invariants}(g)

Safety and performance line up here as aspects of the same mathematical correctness. When the framework aligns with the structure of the computation, the engineering burden of the defensive patterns drops away. Those patterns turn out to be the cost of working against the underlying mathematics rather than with it.

Calendar Time: The Hidden Multiplier

The efficiency gains won’t just be found during training and inference; they will also be multiplied by shifts in development time. Current AI/HPC development follows assumptions around costly patterns:

Write initial code (teams, for weeks)
Debug tensor shape errors (teams, for weeks)
Track down numerical instabilities (separate team, for weeks)
Optimize bottlenecks (separate team, for weeks)
Re-verify after optimization (days)
Deploy and remediate edge cases (ongoing)

With Fidelity’s proposed approach:

Write type-safe categorical code (one team, for weeks)
Compiler catches all shape/dimension errors ( – )
Universal numbers prevent instabilities ( – )
Proofs guide optimizations automatically ( – )
Verification is built-in ( – )
Edge cases caught at compile time ( – )

The same functionality would ship in weeks instead of months. Smaller, more tightly aligned teams could iterate faster, with more room to adapt the code and the data it’s trained on, exploring solution spaces that were previously infeasible to reach.

The Compounding Effect

These gains compound across their domains rather than adding. A system that’s 10x more efficient at runtime, developed 6x faster, with 10x fewer bugs, running on hardware that dissipates 10x less heat, multiplies its advantage at each layer. Modern AI/HPC systems carry a great deal of “computational dithering,” cycles spent managing accidental complexity rather than solving the problem at hand:

Memory management overhead: Copying data between CPU/GPU
Synchronization waste: Barriers and locks for incorrect abstractions
Precision management: Converting between float16/32/64
Framework translation: PyTorch -> ONNX -> TensorRT -> Hardware

Each layer adds overhead, heat, and opportunity for error, all while leadership is watching sand drop through the hourglass. The categorical approach with Universal numbers eliminates these layers:

// Direct path from mathematics to hardware
let QCE molecule =
 // Mathematics expressed directly
 let hamiltonian = constructHamiltonian molecule

 // Universal numbers handle precision automatically
 let groundState = Posit<32,2>.DiagonalizeExactly hamiltonian

 // Category theory ensures correct composition
 let correlation = categorical {
 let! hartreeFock = HF.compute molecule
 let! correction = Neural.correlationEnergy molecule
 return hartreeFock + correction
 }

 // lowers straight to hardware through MLIR, no intermediate copies or conversions
 Fidelity.compile correlation |> Execute.on GPU

And rather than waiting on a new silicon process node, these gains are within reach on today’s hardware through mathematical and architectural work.

Everything Fidelity framework establishes for the next generation of hardware can also target the current generation of hardware with some in-compiler adjustments to target appropriate hardware.

Our goal is to make the design-time experience substantially the same in either case.

Fidelity Framework: Unifying Implementation

The Fidelity framework is designed to carry these concepts into working engineering, building on all our previous work:

Coeffect Analysis for Unified Optimization

As we explored in our ternary models work, different computational patterns require different execution strategies:

// Coeffects determine optimal execution strategy
type UnifiedCoeffects =
 | SimulationPattern of timesteps: int * conservation: Laws
 | LearningPattern of epochs: int * gradients: Flow
 | HybridPattern of physics: Simulation * correction: Learning

 member this.CompilationStrategy =
 match this with
 | SimulationPattern(_, laws) when laws.AreLinear ->
 InteractionNets // Parallel physical simulation
 | LearningPattern(_, flow) when flow.IsSparse ->
 DelimitedContinuations // Sequential gradient flow
 | HybridPattern(physics, learning) ->
 HeterogeneousExecution(physics.OnHPC(), learning.OnGPU())

Proof-Aware Compilation Through Hypergraphs

The hypergraph architecture is designed to optimize while preserving proofs:

Layer 1: Hypergraph Optimization with Proofs

// Optimize while maintaining verified properties
let optimizeWithProofs (graph: Hypergraph) =
 // Extract proof obligations
 let proofs = graph.Hyperedges |> filterProofs

 // For HPC: Maintain conservation laws
 let physicsProofs = proofs |> filterPhysics
 let optimizedPhysics =
 graph
 |> fuseOperations physicsProofs
 |> parallelizeTimeSteps physicsProofs

 // For AI: maintain convergence bounds
 let learningProofs = proofs |> filterLearning
 let optimizedLearning =
 graph
 |> fuseGradients learningProofs
 |> quantizeWeights learningProofs

 // Unified: Both optimized with proofs preserved
 combineOptimized optimizedPhysics optimizedLearning proofs

Layer 2: MLIR with Constraint Preservation

graph TD
A[Clef Source with Proofs] --> B[Alex AST Transform]
B --> C[High-Level MLIR]
C --> D[Proof-Preserving Lowering]
D --> E[Hardware-Specific MLIR]
E --> F[Verified Machine Code]
G[Z3 Verification] --> D
H[SMT Proofs] --> D
I[Universal Numerics] --> D

At the MLIR level, proof obligations once satisfied transform into optimization constraints using standard MLIR infrastructure. Rather than custom dialects, we leverage MLIR’s existing attribute system and transformation framework, including the SMT dialect for encoding verification conditions that can be checked by Z3 during lowering. Proof metadata travels as function and operation attributes that standard MLIR passes respect but don’t need to understand.

For example, when lowering HPC simulations, we use standard linalg and affine dialects for the computation, with satisfied proof obligations encoded as attributes that prevent unsafe transformations. The affine dialect’s polyhedral model naturally preserves loop invariants that correspond to conservation laws. The SMT dialect encodes these invariants as assertions that can be verified at compile time. Similarly, AI operations lower through tensor and linalg dialects with attributes marking gradient-critical paths that must maintain numerical stability.

MLIR’s pass infrastructure already supports preserving unknown attributes through transformations. These attributes flow all the way through to LLVM as metadata and function attributes that constrain backend optimizations. For instance, a conservation law verified by Z3 becomes both an affine constraint in MLIR and a llvm.loop.invariant metadata node in LLVM IR. A convergence bound becomes both a barrier to certain MLIR transformations and an llvm.assume intrinsic that enables safe optimizations while preventing unsafe ones. The mathematical properties guide the lowering without requiring MLIR or LLVM to understand the proofs themselves. They respect the constraints that our PHG-guided proofs impose, based on their satisfaction through MLIR and F*.

Layer 3: Hardware-Specific Verified Code

// Generate verified code for specific hardware
let generateVerifiedCode (target: HardwareTarget) (graph: OptimizedGraph) =
 match target with
 | CPU ->
 // HPC kernels with vector intrinsics
 generateCPUWithProofs graph "AVX-512" "OpenMP"
 | GPU ->
 // AI kernels with tensor cores
 generateGPUWithProofs graph "CUDA" "TensorRT"
 | FPGA ->
 // Custom datapaths for both
 generateFPGAWithProofs graph "Verilog" "HLS"
 | Heterogeneous(cpu, gpu) ->
 // Split verified computation
 let hpcPart = extractHPC graph |> generateCPU
 let aiPart = extractAI graph |> generateGPU
 combineWithProofs hpcPart aiPart

The HPC-AI Convergence

Why Convergence Follows

HPC and AI are discovering they need each other, and the CDL mathematics shows why. Both domains are working with the same underlying structure:

\mathbf{HPC}: \text{Phys} \xrightarrow{F} \text{Comp} \xrightarrow{F^*} \text{Phys}

\mathbf{AI}: \text{Data} \xrightarrow{N} \text{Latent} \xrightarrow{N^*} \text{Data}

Where $F^*$ and $N^*$ are the adjoints (sensitivity analysis and backpropagation respectively). The convergence occurs when we recognize these are the same pattern:

\mathbf{Unified}: \mathcal{A} \xrightarrow{\Phi} \mathcal{B} \xrightarrow{\Phi^{\dagger}} \mathcal{A}

New Computational Primitives

The convergence creates new primitives that transcend the HPC/AI divide:

// Differentiable simulation - gradients through physics
type DifferentiableSimulation<'State> = {
 // Forward: HPC simulation
 Simulate: 'State -> 'State

 // Backward: Automatic differentiation
 Gradient: 'State -> Gradient<'State>

 // Certificates: Z3 discharges energy conservation and gradient correctness
 ForwardCertificate: Z3Certificate
 BackwardCertificate: Z3Certificate

 // Numerics: Unified representation
 Arithmetic: Posit<32,2>
}

// Verified neural operators - learning with proofs
type VerifiedNeuralOperator<'Domain> = {
 // Learn: AI optimization
 Train: Dataset<'Domain> -> Model<'Domain>

 // Verify: Z3 discharges the safety property and returns a certificate
 Certify: Model<'Domain> -> Z3Certificate

 // Execute: HPC performance
 Run: 'Domain -> 'Domain

 // Numerics: Same substrate
 Computation: Universal.Quire<32, 1024>
}

Unified Applications

Digital Twins with Verified Learning

Building on our exploration of heterogeneous computing, we can create unified digital twins:

module JetEngineDigitalTwin =
 // Clef implementation with SMT verification attributes
 [<SMT Requires("valid_state(engine)")>]
 [<SMT Ensures("energy_conserved(result)")>]
 [<SMT Ensures("wear_monotonic(result)")>]
 [<SMT Ensures("safe_operating_envelope(result)")>]
 let implementation (engine: EngineState) (sensorData: SensorStream) =
 // HPC: Computational fluid dynamics
 let cfd = VerifiedCFD<Posit<64,3>>(
 proof = ConservationProof.Energy
 )

 // AI: Learn degradation patterns
 let degradation = LearnedDegradation<Posit<32,2>>(
 proof = MonotonicityProof.Wear
 )

 // Unified: Compose with verification
 let unified = categorical {
 let! flow = cfd.Simulate
 let! wear = degradation.Predict

 // Z3 verifies composition preserves properties
 let! composed = verifyComposition flow wear

 return composed
 }

 // Compile to efficient implementation
 unified
 |> Fidelity.BuildHypergraph
 |> Fidelity.AttachProofs [cfd.Proof; degradation.Proof]
 |> Fidelity.OptimizeWithProofs
 |> Universal.GenerateCode

Climate Modeling with Physics-Informed Learning

module ClimateModel =
 // Verification specification
 type ClimateInvariants = {
 EnergyBalance: bool // RadiationIn = RadiationOut + Storage
 MassConservation: bool // TotalMass = Constant
 ThermodynamicLaws: bool // EntropyNonDecreasing
 }

 // Hybrid implementation with SMT verification
 [<SMT Requires("valid_initial_state(atmosphere)")>]
 [<SMT Ensures("energy_balance(result)")>]
 [<SMT Ensures("mass_conserved(result)")>]
 [<SMT Ensures("entropy_nondecreasing(result)")>]
 let climatePredictor (initialState: ClimateState) =
 // HPC: Atmospheric dynamics
 [<SMT Invariant("conserves_energy(atmospheric)")>]
 [<SMT Invariant("conserves_angular_momentum(atmospheric)")>]
 let atmospheric = NavierStokesOnSphere<Posit<32,2>>()

 // AI: Sub-grid phenomena
 [<SMT Ensures("preserves_mass_balance(subgrid)")>]
 [<SMT Ensures("bounded_output(subgrid, -100.0, 100.0)")>]
 let subgrid = NeuralParameterization<Posit<16,1>>()

 // Verification through composition
 [<SMT Assert("no_overflow(atmospheric.Compute)")>]
 [<SMT Assert("no_underflow(subgrid.Compute)")>]
 [<SMT Assert("composition_preserves_invariants(atmospheric, subgrid)")>]
 let verifiedComposition =
 {
 Atmospheric = atmospheric
 SubGrid = subgrid
 Invariants = {
 EnergyBalance = true
 MassConservation = true
 ThermodynamicLaws = true
 }
 }

 // Execute with verified invariants
 VerifiedExecution(verifiedComposition)

Autonomous Systems with Certified Safety

As we explored in our post-transformer architectures, safety-critical systems benefit from unified verification:

module AutonomousVehicle =
 // Clef implementation with SMT verification annotations
 [<SMT Requires("perception.Accuracy > 0.99 && perception.Latency < 10ms")>]
 [<SMT Ensures("forall obstacle in scene.Obstacles.
 distance(cmd.Trajectory, obstacle) > safety_margin")>]
 [<SMT Ensures("cmd.Velocity <= speed_limit && cmd.Acceleration <= comfort_threshold")>]
 let safe_trajectory (perception: SceneUnderstanding)
 (planning: TrajectoryOptimization)
 : VehicleCommand =

 // AI: perception with verified detection bounds
 let verifiedPerception =
 [<SMT Invariant("forall obj. obj.IsObstacle => detected(obj)")>]
 let transformer = VerifiedVisionTransformer<Posit<16,1>>()
 transformer.Configure({
 MinDetectionConfidence = 0.99
 MaxLatency = 10<ms>
 })
 transformer

 // HPC: Real-time trajectory optimization
 [<SMT Ensures("trajectory.IsDynamicallyFeasible()")>]
 [<SMT Ensures("forall t. trajectory.MinDistance(t) > margin")>]
 let optimizeTrajectory (scene: Scene) =
 let controller = OptimalControl<Posit<32,2>>()
 controller.SetConstraints({
 VehicleDynamics = getVehicleModel()
 SafetyMargin = 2.0<m>
 ComfortLimits = getComfortProfile()
 })
 controller.Optimize(scene)

 // Unified: Compose with safety proof
 let integrated = categorical {
 // Process sensor data
 let! scene = verifiedPerception.Process(perception.SensorData)

 // Plan trajectory with verification
 let! path = optimizeTrajectory scene

 // End-to-end safety verification
 [<SMT Assert("collision_free(path) && traffic_compliant(path)")>]
 let! verifiedPath = validatePath scene path

 return VehicleCommand(verifiedPath, ProofCertificate = true)
 }

 integrated |> Async.RunSynchronously

Advanced Implementation Examples

The following examples showcase the depth of verification possible within the Fidelity framework, though we emphasize these are forward-looking implementations that will evolve as we refine the integration between Clef, F*, and our compilation pipeline. Importantly:

The extensive proof annotations shown here represent the maximum verification depth available, not the minimum required.

The Fidelity framework embraces a “verification by choice” philosophy: most Clef developers will write standard Clef code with optional type safety features, adding verification attributes only where their domain demands it. A web application might use no proofs at all, a financial system might verify key invariants, while safety-critical aerospace systems could leverage the full depth shown below. This graduated approach lets teams adopt formal methods incrementally, starting with type safety and adding verification where the business value justifies the effort. The same framework spans casual scripting through stringent certification requirements.

Verified Fluid-Structure Interaction

module FluidStructureInteraction =
 // The problem: Coupling fluid dynamics with structural mechanics
 // Traditional: Separate solvers with weak coupling
 // Unified: Single verified computation

 // Define numerical representation
 type FluidNumber = Posit<32, 2>
 type StructureNumber = Posit<64, 3> // Higher precision for structure

 // Clef implementation with SMT verification annotations
 [<SMT Requires("compatible_interface(fluid, structure)")>]
 [<SMT Ensures("momentum_conserved(result.Fluid, result.Structure)")>]
 [<SMT Ensures("energy_conserved(result.Fluid, result.Structure)")>]
 [<SMT Ensures("stable_coupling(result.Fluid, result.Structure)")>]
 let coupled_system (fluid: FluidState) (structure: StructuralState) =

 // HPC: Fluid solver with conservation verification
 [<SMT Invariant("conserves_momentum(state)")>]
 [<SMT Invariant("conserves_mass(state)")>]
 let solveFluid (initialState: FluidState) = categorical {
 // Use quire for exact pressure accumulation
 use pressureAccumulator = Quire<32, 1024>.Zero

 // Solve Navier-Stokes with verified conservation
 let! solution = solveNavierStokes<FluidNumber> initialState

 // Assert conservation properties
 [<SMT Assert("momentum(solution) = momentum(initialState)")>]
 let! verified = verifyConservation solution

 return verified
 }

 // HPC: Structure solver with AI-enhanced fatigue prediction
 [<SMT Ensures("stress.MaxValue < yield_strength")>]
 [<SMT Ensures("fatigue.Cycles > design_life")>]
 let solveStructure (load: StructuralLoad) = categorical {
 // High-precision stress computation
 let! stress = computeStress<StructureNumber> load

 // AI: Learn fatigue from data with bounded prediction
 [<SMT Ensures("fatigue.Confidence > 0.95")>]
 let! fatigue = neuralFatigueModel.Predict stress

 // Verify stress-fatigue relationship
 [<SMT Assert("fatigue_valid(stress, fatigue)")>]
 let! verified = validateFatigue stress fatigue

 return (stress, fatigue)
 }

 // Coupled system with interface verification
 [<SMT Invariant("interface_continuity(fluid, structure)")>]
 let coupled = categorical {
 // Solve fluid domain
 let! fluidSolution = solveFluid fluid

 // Transfer loads at interface
 [<SMT Assert("load_balance(fluidSolution.Pressure, structureLoad)")>]
 let structureLoad = extractInterfaceLoad fluidSolution

 // Solve structure with transferred loads
 let! (stress, fatigue) = solveStructure structureLoad

 // Update fluid boundary from structural deformation
 [<SMT Assert("geometric_compatibility(fluidSolution, deformation)")>]
 let deformation = computeDeformation stress
 let! updatedFluid = updateFluidBoundary fluidSolution deformation

 // Verify coupling maintains all properties
 [<SMT Assert("energy(updatedFluid) + energy(stress) = energy(fluid) + energy(structure)")>]
 let! verifiedResult = {
 Fluid = updatedFluid
 Structure = { Stress = stress; Fatigue = fatigue }
 InterfaceForces = structureLoad
 CouplingStable = true
 }

 return verifiedResult
 }

 // Execute with automatic proof generation
 coupled |> Categorical.RunWithProofs

Quantum Chemistry with Neural Corrections

This shows how quantum chemical calculations integrate SMT verification as attributes, ensuring physical constraints (Hermiticity, normalization, variational principle) are maintained throughout the computation while combining HPC (Hartree-Fock) with AI (neural correlation) methods.

module QuantumChemistry =
 // The challenge: Ab initio calculations are exponentially expensive
 // Solution: Verified neural corrections to approximate methods

 // Quantum invariants as type constraints
 type QuantumInvariants = {
 Hermiticity: bool // H = H†
 Normalization: bool // ⟨ψ|ψ⟩ = 1
 Variational: bool // E_approx ≥ E_exact
 }

 // Clef implementation with SMT verification
 [<SMT Requires("molecule.IsValid && molecule.Electrons > 0")>]
 [<SMT Ensures("result.Energy >= exact_ground_state_energy(molecule)")>]
 [<SMT Ensures("abs(result.Energy - exact_energy) < 1e-6<Hartree>")>]
 [<SMT Ensures("result.Wavefunction.IsNormalized()")>]
 let molecular_energy (molecule: Molecule) =

 // HPC: Hartree-Fock baseline with variational guarantee
 [<SMT Ensures("energy >= exact_ground_state")>]
 [<SMT Invariant("hamiltonian.IsHermitian()")>]
 let computeHartreeFock (mol: Molecule) =
 let hf = HartreeFock<Posit<64,4>>()
 hf.Configure({
 BasisSet = "cc-pVTZ"
 ConvergenceThreshold = 1e-10<Hartree>
 MaxIterations = 100
 })

 // Self-consistent field iteration
 [<SMT Invariant("density_matrix.IsIdempotent()")>]
 let energy = hf.SolveSCF(mol)

 // Verify variational principle
 [<SMT Assert("energy >= exact_ground_state_energy(mol)")>]
 { Energy = energy; Orbitals = hf.Orbitals }

 // AI: Neural correction with bounded error
 [<SMT Ensures("abs(correction) < max_correlation_energy")>]
 [<SMT Ensures("sign(correction) = -1")>] // Correlation always lowers energy
 let computeCorrelation (hfResult: HartreeFockResult) =
 let nn = NeuralCorrelation<Posit<32,2>>()
 nn.Configure({
 Architecture = "EquivariantGNN" // Respects molecular symmetries
 TrainedOn = "CCSD(T)_G4_dataset"
 ErrorBound = 1.0<kcal/mol>
 })

 // Predict with uncertainty quantification
 [<SMT Invariant("nn.PreservesSymmetry(molecule.PointGroup)")>]
 let (correction, uncertainty) = nn.PredictWithUncertainty(hfResult)

 // Verify correction is physically reasonable
 [<SMT Assert("correction <= 0.0<Hartree>")>] // Always negative
 [<SMT Assert("abs(correction) < 0.5 * hfResult.Energy")>] // Bounded

 { Correction = correction; Uncertainty = uncertainty }

 // Combine with verified accuracy bounds
 [<SMT Ensures("result.TotalEnergy = hf.Energy + correlation.Correction")>]
 [<SMT Ensures("result.ErrorBound < 1.0<kcal/mol>")>]
 let combineWithProof (hf: HartreeFockResult) (corr: CorrelationResult) =
 // Total energy with SMT-discharged error bound
 let totalEnergy = hf.Energy + corr.Correction

 // Error propagation
 [<SMT Assert("total_error = sqrt(hf_error^2 + correlation_error^2)")>]
 let errorBound =
 sqrt(hf.ConvergenceError ** 2.0 + corr.Uncertainty ** 2.0)

 // Package with quantum invariants verified
 {
 TotalEnergy = totalEnergy
 HartreeFock = hf
 Correlation = corr
 ErrorBound = errorBound
 QuantumInvariants = {
 Hermiticity = true // Guaranteed by HF
 Normalization = true // Maintained throughout
 Variational = true // HF+correction ≥ exact
 }
 }

 // Execute computation with all verifications
 let hfResult = computeHartreeFock molecule
 let correlation = computeCorrelation hfResult
 let verified = combineWithProof hfResult correlation

 verified

Verified Kalman Filter with Learning

Building on our discriminated unions exploration, we can create verified filters with learned components:

// Classic algorithm enhanced with verified learning
module VerifiedKalmanFilter =
 open Universal.Posit

 // Clef implementation with SMT verification attributes
 [<SMT Requires("positive_definite(state.Covariance)")>]
 [<SMT Ensures("positive_definite(result.Covariance)")>]
 [<SMT Ensures("result.Error <= state.Error + measurement.Noise")>]
 [<SMT Ensures("numerically_stable(result)")>]
 let kalman_update (state: KalmanState<Posit<32,2>>)
 (measurement: Measurement<Posit<16,1>>)
 : KalmanState<Posit<32,2>> =

 // Predict step with covariance propagation
 [<SMT Invariant("positive_definite(predicted.Covariance)")>]
 let predictState (current: KalmanState<Posit<32,2>>) =
 let F = current.TransitionMatrix
 let Q = current.ProcessNoise

 // State prediction: x̂ₖ₊₁|ₖ = F * x̂ₖ|ₖ
 let predictedState = F * current.State

 // Covariance prediction: Pₖ₊₁|ₖ = F * Pₖ|ₖ * F' + Q
 [<SMT Assert("symmetric(predictedCovariance)")>]
 let predictedCovariance = F * current.Covariance * F.Transpose + Q

 { State = predictedState
 Covariance = predictedCovariance
 TransitionMatrix = F
 ProcessNoise = Q
 Error = current.Error }

 // AI: Learn adaptive measurement noise
 [<SMT Ensures("noise.Mean > 0.0 && noise.Variance < max_variance")>]
 [<SMT Ensures("noise.IsStationary || noise.AdaptsSlowly")>]
 let learnNoisePattern (history: Measurement<Posit<16,1>> array) =
 let nn = NoiseEstimator<Posit<16,1>>()
 nn.Configure({
 WindowSize = 100
 Architecture = "LSTM"
 MaxVariance = 1.0<unit^2>
 })

 // Learn noise characteristics with bounds
 [<SMT Invariant("forall t. noise(t) > 0")>]
 let noiseModel = nn.EstimateNoise(history)

 // Verify learned model is physically reasonable
 [<SMT Assert("noise.AutoCorrelation < 0.95")>] // Not perfectly correlated
 noiseModel

 // Update step with numerical stability
 [<SMT Ensures("result.Covariance = (I - K*H) * predicted.Covariance")>]
 [<SMT Invariant("no_overflow(computation)")>]
 let computeUpdate (predicted: KalmanState<Posit<32,2>>)
 (meas: Measurement<Posit<16,1>>)
 (noise: NoiseModel<Posit<16,1>>) =

 // Use quire for exact accumulation in critical computation
 use covarianceAccumulator = Quire<32, 2048>.Zero

 let H = meas.ObservationMatrix
 let R = noise.CovarianceMatrix

 // Innovation covariance: S = H * P * H' + R
 [<SMT Assert("positive_definite(S)")>]
 let S = H * predicted.Covariance * H.Transpose + R

 // Kalman gain: K = P * H' * S⁻¹
 [<SMT Assert("well_conditioned(S)")>] // Invertibility check
 let K = predicted.Covariance * H.Transpose * S.Inverse

 // State update: x̂ₖ₊₁ = x̂ₖ₊₁|ₖ + K * (z - H * x̂ₖ₊₁|ₖ)
 let innovation = meas.Value - H * predicted.State
 let updatedState = predicted.State + K * innovation

 // Covariance update (Joseph form for numerical stability)
 [<SMT Assert("positive_definite(updatedCovariance)")>]
 let I_KH = Matrix.Identity - K * H
 let updatedCovariance =
 I_KH * predicted.Covariance * I_KH.Transpose + K * R * K.Transpose

 // Error bound propagation
 [<SMT Assert("updatedError <= predicted.Error + noise.Bound")>]
 let updatedError = sqrt(predicted.Error ** 2.0 + noise.Bound ** 2.0)

 { State = updatedState
 Covariance = updatedCovariance
 TransitionMatrix = predicted.TransitionMatrix
 ProcessNoise = predicted.ProcessNoise
 Error = updatedError }

 // Main implementation with all verifications
 categorical {
 // Prediction step
 let! predicted = predictState state

 // Learn measurement noise from recent history
 let! noiseModel = learnNoisePattern measurement.History

 // Update with learned noise model
 let! updated = computeUpdate predicted measurement noiseModel

 // Verify all properties maintained
 [<SMT Assert("positive_definite(updated.Covariance)")>]
 [<SMT Assert("updated.Error <= state.Error + measurement.Noise")>]
 [<SMT Assert("numerically_stable(updated)")>]

 return updated
 }

This implementation shows how the Kalman filter’s mathematical properties (positive definiteness, error bounds, numerical stability) are verified through SMT attributes that also* transfer to inline proofs in MLIR, integrating learned noise models and using Universal numbers for exact computation in critical sections.

The Unified Computational Future

The Convergence Timeline

graph TB
B --> C[2027-2028: Production Systems]
C --> D[2029+: Unified Paradigm]
A[2024-2025: Separate Worlds] --> B[2025-2026: Early Field Tests]
A -->|"HPC: Case Studies"| A1[FPGA + GPU]
A -->|"AI: LLVM<br>CUDA & SPIR-V"| A2[BFloat16,8,4]
B -->|"Universal Numbers"| B1[Research on Result<br>Equivalence & Supremecy]
B -->|"Fidelity Prototypes"| B2[Hybird Hardare<br>Experiments]
C -->|"Unified Clusters"| C1[New Hardware Platforms]
C -->|"Amortizable Value"| C2[Extant/Hybrid Systems]
D -->|"No HPC vs AI vs Quantum<br>vs Conventional"| D1[Full Platform Convergence]

A Natural Path to General Quantum Compute

As we explored in our quantum optionality analysis, this categorical foundation does more than unify classical HPC and AI; it provides an algorithmic bridge to quantum computing. The same categorical morphisms that describe neural networks and physical simulations also describe quantum circuits.

The alignment is algorithmic rather than aspirational. Quantum mechanics was categorical before computer scientists discovered category theory. When Heisenberg developed matrix mechanics and Schrödinger wave mechanics in the 1920s, they were unknowingly working with functors between categories. When Dirac showed these were equivalent formulations, he was proving a categorical equivalence:

// All backends preserve the same categorical properties
[<SMT Assert("forall backend. factorize(n).Result.p * factorize(n).Result.q = n")>]
type UniversalComputation<'Input, 'Output> =
 | Classical of CPUComputation<'Input, 'Output>
 | Parallel of GPUComputation<'Input, 'Output>
 | Quantum of QuantumCircuit<'Input, 'Output>
 | Hybrid of (Classical * Quantum)<'Input, 'Output>

 // All share the same categorical structure
 [<SMT Ensures("result.Forward.Domain = this.InputType")>]
 [<SMT Ensures("result.Forward.Codomain = this.OutputType")>]
 [<SMT Ensures("adjoint(result.Forward) = result.Backward")>]
 member this.AsCategorical() =
 categorical {
 // Forward morphism
 let! forward = this.Forward

 // Adjoint (backward/inverse)
 let! adjoint = this.Adjoint

 // adjunction laws discharged by the verifier
 [<SMT Assert("compose(forward, adjoint) = identity(codomain(forward))")>]
 [<SMT Assert("compose(adjoint, forward) = identity(domain(forward))")>]
 return Morphism(forward, adjoint)
 }

// Concrete example: Prime factorization with automatic backend selection
[<SMT Requires("n > 1 && not isPrime(n)")>]
[<SMT Ensures("result.p * result.q = n")>]
[<SMT Ensures("isPrime(result.p) && isPrime(result.q)")>]
let factorize (n: bigint) : UniversalComputation<bigint, Factor * Factor> =
 match n with
 | SmallNumber when n < 10_000I ->
 // Classical trial division for small numbers
 Classical (CPUComputation.trialDivision n)
 | MediumNumber when n < 1_000_000_000I ->
 // Parallel Pollard's rho for medium numbers
 Parallel (GPUComputation.pollardRho n)
 | LargeNumber when quantumAvailable() && n.BitLength > 128 ->
 // Shor's algorithm for cryptographic-scale numbers
 Quantum (QuantumCircuit.shor n)
 | _ ->
 // Hybrid: classical preprocessing + quantum period finding
 Hybrid (Classical.reduce n, Quantum.periodFind)

We’re not adapting our framework to include quantum computing; quantum computing was already part of this mathematical structure. The 2-categorical framework that unifies HPC and AI is designed to encompass quantum without extension. As quantum hardware reaches practical maturity, our design intends for it to slot into the same categorical infrastructure as another instance of the patterns that already cover HPC and AI.

The categorical framework captures the structure of computation across substrates. As quantum hardware matures, we intend our systems to be ready through the same principles that unify HPC and AI.

Technical Hurdles and Open Questions

While this vision is compelling, we must be honest about the challenges ahead:

Mathematical Foundations: Translating category theory into efficient implementations remains an active research area. The gap between mathematical elegance and machine level computational efficiency is non-trivial.

Tool Maturity: F*, Z3, and MLIR are capable and in many cases well aligned, but integrating them cleanly for production use will require concerted effort. Our Fidelity framework implementation is still evolving to meet the challenges that each “corner case” will present.

Performance Validation: Theoretical advantages don’t always translate to equal speedups in material implementation. Bottlenecks emerge in unexpected places when targeting complex hardware. Extensive benchmarking across diverse workloads on extant and emerging hardware architectures will be necessary.

Ecosystem Integration: The machine learning community has massive investment in Python and adjacent ecosystems. Creating migration paths and interoperability layers is essential for adoption. The impedence mismatches between Python and principled use of the Fidelity framework will create challenges and opportunities for professional training, code conversion and migration tooling.

Hardware Co-design: To fully embody this vision will ultimately require new hardware architectures that natively support categorical operations and data flow based execution. While we see many vendors that show promise in this direction, it’s a growing field that will require considerable coordination, evaluations, prototyping and testing.

Despite these challenges, we believe the convergence is achievable. The inefficiencies of maintaining separate HPC and AI stacks, combined with the increasing demand for verified, efficient computation, will drive this unification.

The Unified Future is Now

The convergence of Categorical Deep Learning, Universal Numbers, and low-burden formal verification in the Fidelity framework points toward a unified way to build verified, efficient systems. Its underlying elements are available today:

CDL provides the theoretical underpinnings
Universal addresses the numerical challenges
F*/Z3 provides formal verification
Fidelity unifies the implementation through Clef

Our journey toward this unified vision wasn’t planned; it emerged from solving real engineering problems. That these solutions align with established mathematical principles gives us confidence in the direction.

What remains is the engineering effort to realize this integration, working from the position that classical compute, AI, HPC, and quantum are complementary aspects of a single computational science.

The Path Forward

We see the next generation of software platforms as a unified, verified, numerically correct landscape that combines algorithmic integration with hardware-aware engineering. For SpeakEZ Technologies, that direction begins with the design summarized in this document.

The open question is not whether quantum, classical, HPC, and AI will merge, but how quickly we can build the infrastructure to support a unified paradigm. Organizations that work in this convergence stand to compute faster, adapt more readily, and build on a verified foundation: digital twins held to physical conservation laws, autonomous systems with certified safety properties, and scientific tools that learn under formal constraints.

This convergence offers a path to efficiency improvements without waiting for Moore’s Law.

Moving from control-flow to data-flow can yield order-of-magnitude improvements, compile-time verification removes weeks of debugging, and Universal numbers cut the computational “dithering” that wastes development cycles. The direction we see is not more transistors and more engineers, but correctness in the algorithms delivering efficiency at scales that matter.

This is the work we are building toward in the Fidelity framework through Categorical Deep Learning, Universal Numbers, and formal verification: computation that is rigorous and adaptive, verified and efficient. The standing trade-off between systems that compute exactly but cannot learn and systems that learn but cannot prove is the one we are setting out to close.

I’ll keep developing this design at SpeakEZ as the framework matures, and I expect the work to keep finding confirmation in the mathematics, the same way the CDL paper did. That’s where my interest lies as the work continues.

Fewer Tests; Greater Safety

Fri, 08 Aug 2025 00:00:00 +0000

Every software engineering team knows the testing treadmill. Write code, write tests, run tests, fix failures, write more tests to catch what you missed, maintain those tests forever. We’ve accepted this as the calendar and staffing cost multiplication demanded by standard approaches to quality software. But what if this entire cycle represents an inefficiency, a workaround in the absence of something better? Our Fidelity framework’s proof-aware compilation is designed for a different path: mathematical certainty at compile time, eliminating entire categories of tests while actually increasing safety.

The Testing Paradox

It’s an uncomfortable truth woven into the fabric of the industry: despite writing more test code than ever, we’re not achieving better software quality. Modern projects often can have multiple lines of test code for every line of application code. CI/CD pipelines run thousands of tests on every commit. Yet catastrophic production failures persist, security breaches continue, and that nervous feeling before major deployments never quite goes away.

The problem isn’t that we’re bad at testing. The problem is that testing has inherent limitations. As Dijkstra famously observed, tests can show the presence of bugs but never their absence. You can test a dozen inputs and the thirteenth might still fail. This isn’t pessimism; it’s mathematics.

Consider bounds checking on array access. How many test cases would you need to prove an index is always valid? You’d need to test every possible execution path, with every possible input, in every possible state. That’s not just impractical; it’s impossible. So we write a handful of test cases, hope we’ve covered the important scenarios, and ship code with our fingers crossed.

graph TD
subgraph "Traditional Testing"
CODE[Write Code] --> TEST[Write Tests]
TEST --> RUN[Run Tests]
RUN --> FIX[Fix Failures]
FIX --> MORE[Write More Tests]
MORE --> MAINTAIN[Maintain & Expand<br>Coverage Forever]
end
subgraph "Proof-Aware Compilation"
CODE2[Write Code with Properties] --> COMPILE[Compile with Proofs]
COMPILE --> VERIFIED[Verified Binary]
end

Targeted Proofs Change Everything

Proof-aware compilation changes the model. Instead of checking whether code behaves correctly for specific inputs, we prove it behaves correctly for ALL inputs.

When you annotate a function with a proof obligation in our Fidelity framework, you’re not writing another test. You’re establishing a mathematical property that the compiler verifies. If compilation succeeds, that property holds for every possible execution, not just the cases you remembered to test.

Take that array bounds example again. With proof annotations:

[<SMT Requires("index >= 0 && index < array.Length")>]
[<SMT Ensures("result = array.[index]")>]
let getElement array index = array.[index]

The compiler doesn’t just check this; it proves it. Every call site must satisfy the precondition, either through explicit checks or through its own proofs. The result is zero bounds-check exceptions in production, guaranteed. Not “we haven’t seen any,” but a discharged obligation that holds for every call.

The Time and Money Equation

Let’s talk about what this means for real development teams. Teams can spend 35-65% of their time on testing and troubleshooting. This includes writing and maintaining tests, debugging failures, and updating all of those artifacts when requirements change. That’s essentially half your engineering payroll.

With proof-aware compilation, many of these tests (and the costs and headaches that come with them) disappear. Not because of being reckless, but because they’re redundant. When the compiler proves memory safety, you don’t need memory leak tests. When it proves bounds safety, you don’t need bounds checking tests. When it proves state machine correctness, you don’t need state transition tests.

There is a further benefit: proofs don’t rot. Tests require constant maintenance as code evolves. Change a function signature, update twenty tests. Refactor a module, fix fifty test dependencies. Proofs, on the other hand, compose and adapt. Change that function signature and the compiler automatically reverifies all dependent proofs. The safety net repairs itself.

Actor Systems: Where Proofs Really Shine

The benefits multiply in concurrent systems like our Olivier actor framework. Testing concurrent behavior is notoriously difficult. Race conditions hide during testing and emerge in production. Message ordering issues appear only under specific timing conditions. Deadlocks lurk in state spaces too large to explore exhaustively.

Proofs address concurrency directly. When you prove absence of deadlock, you’re not hoping your tests stressed the system enough; you have a structural guarantee that it can’t happen.

graph LR
subgraph "What Tests Check"
T1[Test Case 1: A→B→C]
T2[Test Case 2: B→A→C]
T3[Test Case 3: A→C→B]
MISSING[??? Unknown Cases ???]
end
subgraph "What Proofs Guarantee"
ALLPATHS[NO Possible Deadlocks:<br/>Proven Correct]
end

Heat Shielding for Production

Think of proofs as heat shielding for your production environment. Just as spacecraft heat shields protect against the extreme conditions of reentry, compile-time proofs protect against the extreme conditions of production: unexpected inputs, resource exhaustion and concurrency storms.

Traditional testing is like checking heat shield integrity by running blowtorches over sample tiles. You gain confidence, but you’re only testing what you thought to test. Proof-aware compilation is like having the mathematical equations that guarantee the heat shield will perform correctly across all possible reentry scenarios.

Our Fidelity framework’s design aims to make this heat shielding practical and accessible. You won’t need a PhD in formal methods. You won’t need to write proofs in specialized languages. You annotate your Clef code with properties you care about, and the compiler does the heavy lifting. As we currently conceive it, the proofs discharge first through an external proof assistant via a code-generation framework, and the resulting obligations then travel through the compilation pipeline as hyperedges, guiding optimization while carrying the safety constraints forward. It’s a type of double-entry accounting for verifying the application you’re building.

The Optimization Bonus

The same proofs that eliminate tests also enable better optimization. When the compiler knows certain properties hold, it can optimize aggressively within those boundaries. Bounds checks proven unnecessary? Eliminated. Error paths proven unreachable? Removed. Independence proven between operations? Parallelized automatically.

This creates a virtuous cycle. Stronger proofs enable better optimization, which produces faster code, which runs more efficiently in production. You’re not trading safety for speed or speed for safety; you’re getting both through mathematical certainty.

Easing Into Proof-Aware Development

The transition from test-heavy to proof-aware development shouldn’t require a big-bang rewrite. Start with critical paths; those functions where bugs would hurt most. Add proof annotations that capture essential properties. Let the compiler verify them. Watch as entire categories of tests become unnecessary.

In the future as your team gains confidence, expand proof coverage. Our Fidelity framework’s progressive approach is meant to give you options to mix modes between traditional tests and formal proofs, gradually shifting the balance as you see the benefits.

Most importantly, remember that proofs aren’t just stronger than tests; they’re often simpler. A one-line proof annotation can replace dozens of test cases. Mathematical certainty replaces lingering doubt.

The Future is Solvable

We’re entering an era where software complexity exceeds our collective ability to test comprehensively. AI systems, distributed architectures, heterogeneous hardware; these create state spaces beyond traditional testing’s reach. The choice isn’t between testing more or testing less; it’s between continuing on the testing treadmill or stepping off into a world of mathematical certainty.

Our Fidelity framework is designed to make this transition practical: embedding proofs in the compilation pipeline, making them first-class artifacts that guide optimization, and providing “free” verification through compile-time analysis. The effect we’re after is not only reducing test burden but changing how a team builds reliable software.

Fewer tests doesn’t mean less safety. It means greater safety through stronger guarantees on the deterministic layer, with the heat shielding of narrowly scoped formal methods standing in for whole categories of treadmill testing. When the compiler becomes the theorem prover, every successful build carries a proof of the properties you annotated. That’s the design we will keep building toward as the rest of the framework comes into place.

Quantum Optionality

Mon, 04 Aug 2025 00:00:00 +0000

The quantum computing landscape in 2025 presents both advances and sobering realities. The technology has moved beyond pure research into early commercial deployments, and it remains years away from the applications often promised in popular media. For our Fidelity framework, this raises a design question. How can we architect the system to leverage quantum acceleration when it becomes practical, without over-committing to a technology still finding its footing?

From our design perspective, this document examines how the Clef language’s functional basis, combined with our forward-looking Program Hypergraph (PHG) architecture and interaction net foundations, opens a path toward future quantum-classical integration. Fault-tolerant quantum computers remain on the horizon, with expert consensus suggesting 2030 ± 2 years. We are preparing architectural foundations that could adapt to quantum acceleration when specific use cases demonstrate a measurable advantage.

An Emerging Quantum Reality

Before exploring integration possibilities, it’s important to acknowledge where quantum computing stands today. Government agencies are leading concrete deployments, with the U.S. Department of Defense awarding contracts like IonQ’s $54.5 million Air Force Research Lab project. Financial institutions, particularly JPMorgan Chase with their dedicated quantum team, have achieved specific milestones like demonstrating Certified Quantum Randomness on Quantinuum’s 56-qubit system.

Current systems face technical barriers. Error rates remain 1-2 orders of magnitude above fault-tolerance thresholds, and coherence times vary by technology. The path to practical quantum computing requires substantial overhead. Current estimates suggest 100-1,000 physical qubits per logical qubit for effective error correction.

This reality shapes our approach. We are designing for selective integration, where quantum acceleration could provide a computational advantage for specific subroutines within larger classical applications.

One often overlooked challenge in current quantum simulation is numerical precision. Most quantum simulators rely on IEEE-754 floating point, which distributes precision uniformly across its range. That uniform distribution wastes bits on regions far from the quantum amplitudes that cluster near superposition states:

// Quantum amplitude calculation showing IEEE754 precision loss
let demonstratePrecisionLoss () =
 // Near-zero amplitude (common in quantum superposition)
 let smallAmplitude = 1e-8
 let float64Result = sqrt(1.0 - smallAmplitude * smallAmplitude)
 let posit32Result = sqrt(1.0r - smallAmplitude * smallAmplitude) // 'r' suffix for posit

 // IEEE754 loses significant precision in this critical region
 printfn "Float64: %.15f (uniform precision, wasted bits)" float64Result
 printfn "Posit32: %.15f (tapered precision, optimized)" posit32Result

 let accumulatedError_IEEE = pown (float64Result - 1.0) 100 // 100 gate operations
 let accumulatedError_Posit = pown (posit32Result - 1.0r) 100

 printfn "After 100 operations - IEEE error: %e" accumulatedError_IEEE
 printfn "After 100 operations - Posit error: %e" accumulatedError_Posit

Float64: 0.999999999999995 (uniform precision, wasted bits)
Posit32: 0.999999999999999 (tapered precision, optimized)
After 100 operations - IEEE error: 2.512e-28
After 100 operations - Posit error: 1.000e-30

This precision difference has measurable implications for quantum-classical integration. In quantum computing, unitarity preservation is mathematically required. When IEEE-754 precision loss causes amplitude normalization to drift from 1.0, the quantum state becomes non-physical. The drift cascades into errors in probability calculations and measurement outcomes, and entanglement fidelity degrades.

The downstream impact extends from quantum simulation into classical processing. Financial risk calculations that rely on quantum amplitude amplification for tail-risk sampling become unreliable when amplitude precision degrades. Cryptographic protocols that depend on quantum random number generation lose their security properties when the underlying quantum states deviate from theoretical predictions. Hybrid quantum-classical optimization algorithms become unstable as precision errors accumulate across the quantum-classical interface.

For regulated industries like finance and aerospace, these precision-induced deviations represent a hard problem. Regulatory compliance requires mathematical proof of correctness, which is impossible to achieve when the underlying numerical representation systematically introduces uncontrolled errors. Our posit arithmetic addresses this by concentrating precision exactly where quantum amplitudes reside. With precision held in that region, we can prove error bounds on simulation fidelity that current IEEE-754-based approaches cannot match.

Current quantum simulation efforts frequently encounter these precision-induced deviations from unitarity, which produce non-physical results that compromise algorithm fidelity. This numerical degradation compounds through multi-qubit systems, making large-scale quantum emulation unreliable for verification purposes. Our Fidelity framework resolves this limitation by combining posit arithmetic’s quantum-optimized precision with Clef verification to prove error bounds on simulation fidelity, which moves quantum emulation from a statistical approximation toward a verifiable computational method.

Update: April 2026

The landscape described above has changed materially since this post was written.

On March 30, 2026, two independent results collapsed the resource estimates for cryptographically relevant quantum computers. Google Quantum AI published revised ECDLP circuit compilations requiring fewer than 1,200 logical qubits and 90 million Toffoli gates to break 256-bit elliptic curve cryptography, executable on fewer than 500,000 physical qubits. The same day, Cain et al. (arXiv:2603.28627), a collaboration between Oratomic, Caltech, and UC Berkeley, demonstrated that high-rate qLDPC codes on reconfigurable neutral-atom architectures bring ECC-256 within reach of as few as 10,000 atomic qubits. The “100-to-1,000 physical qubits per logical qubit” estimate cited in this post has been substantially undercut by these codes achieving approximately 30% encoding rates. The “2030 ± 2 years” timeline for fault-tolerant quantum computers is no longer the operative planning constraint; Google’s own 2029 deadline is a migration lead-time target, not a hardware arrival prediction.

The QIR critique in this post also warrants context. Subsequent work has re-routed QIR-lineage approaches through MLIR, which addresses the static single-assignment concerns that motivated our original assessment. The Fidelity framework’s commitment to MLIR as the compilation substrate, and to Appel’s SSI formulation as the correct foundation for program analysis, remains unchanged.

The architectural choices described here, building for quantum optionality within a verified compilation framework, remain sound. The urgency of those choices has increased. Our current assessment of the CRQC landscape and its implications for the Fidelity framework’s verification architecture is developed in the SpeakEZ research entry Zero Knowledge Proofs: Verification as Product. The formal substrate for the decidable fragment discussed here is expanded in Building Proofs for the Real World and “Free” Proofs from Dimensional Types.

Beyond QIR: Building on Early Experiments

The Quantum Intermediate Representation (QIR) Alliance and Microsoft’s Q# were pioneering efforts that established foundations for quantum-classical integration. These early experiments demonstrated the viability of unified compilation frameworks and helped identify the challenges in bridging quantum and classical domains. The QIR repositories show reduced activity, with key updates dormant since 2022-2024, and the lessons from these initiatives inform our approach.

Where QIR and Q# laid groundwork, our Fidelity framework extends their initial scope along four directions:

Posit arithmetic for quantum amplitude representation, with higher precision near quantum superposition states than IEEE-754 offers
Proof-carrying compilation via Clef integration, which supports mathematical verification of quantum circuit structural correctness
Memory mapping with native machine layouts through our patented BAREWire protocol, giving zero-copy data exchange between quantum emulation and classical processing
Program Hypergraph architecture that represents quantum-classical boundaries as hyperedges

These capabilities position our framework as more than another quantum IR. We have found no other representative implementation in the standing literature we have reviewed that combines verified compilation, posit precision, and zero-copy memory mapping for quantum-classical computing.

The Emulation Alternative with Proven Bounds

While quantum hardware matures, high-fidelity emulation with proven error bounds offers an intermediate approach. Through posit arithmetic’s tapered precision and formal verification, we can produce quantum-algorithm-like results with proven error bounds rather than statistical confidence alone. This matters in regulated industries where proof of correctness ranks above raw speed.

Posit arithmetic carries a precision advantage for quantum amplitude calculations. Where IEEE-754 floating point spends precision on regions irrelevant to quantum computation, posits concentrate their precision near zero and one, the region where quantum amplitudes typically reside. The tapered precision of posit32_2 delivers approximately 100x better relative error for amplitudes near superposition states than standard float32.

The Program Hypergraph Vision

Our move from traditional graph representations to the Program Hypergraph (PHG) architecture changes how a compiler bridges different computational paradigms. Traditional compiler IRs decompose multi-way relationships into binary connections. Our hypergraph edges instead hold the simultaneity of quantum-classical interactions in one place.

The Natural Quantum-Classical Bridge

Our hyperedges capture quantum phenomena directly. Multi-qubit entanglement becomes a single hyperedge connecting all participating qubits, which holds the semantic unity that binary graph edges would fragment. Quantum measurements that collapse multiple qubits into classical bits are represented as measurement hyperedges connecting the quantum and classical domains in one edge.

With those relationships intact, the compiler retains the information it needs to partition work across the quantum-classical boundary.

Proof-Carrying Quantum Computation

Our architecture supports proof-carrying quantum computation through the integration of Clef verification, posit arithmetic, and memory protocols mapped to native machine layouts. Clef’s verification annotations over ordinary Clef functions generate proofs about error bounds and structural correctness, and they track precision bounds throughout quantum emulation.

[<Requires("qubits <= maxSystemQubits")>]
[<Requires("depth <= maxCircuitDepth")>]
[<Ensures("result.errorBound < physicalQuantumError")>]
let quantumEmulationWithProofs (circuit: QuantumCircuit) (initialState: QubitState[]) =
 // structural validity, discharged by the verifier
 let validatedCircuit = QuantumCircuit.validate circuit

 // posit precision near |0⟩ and |1⟩
 let positState = QuantumEmulation.executeWithPosit32_2 validatedCircuit initialState

 // accumulated error across gate operations
 let errorBound = PositAnalysis.computeAccumulatedError circuit

 // zero-copy transfer, native layout preserved
 let classicalResult = BAREWire.transferToClassical positState

 (classicalResult, ProofCertificate errorBound)

Where traditional quantum computing approaches provide statistical confidence about results, our proof-carrying approach provides a proof of structural correctness and a discharged bound on accumulated error. This distinction matters in regulated industries where compliance requires demonstrable correctness.

Direct Backend Integration via PHG

Our Program Hypergraph architecture targets multiple quantum backends while preserving its verification obligations:

flowchart TD
subgraph "Fidelity Frontend"
PHG[Program Hypergraph<br/>Multi-way relationships preserved]
CFG[Control Flow View]
DFG[Data Flow View]
PHG --> CFG
PHG --> DFG
end
subgraph "Verification Layer"
CFG --> CLEF[Clef Verification<br/>Proof Generation]
DFG --> POSIT[Posit Arithmetic<br/>Precision Tracking]
CLEF --> PROOF[Proof-Carrying IR]
POSIT --> PROOF
end
subgraph "Backend Selection"
PROOF --> SPLIT{Execution Strategy}
SPLIT -->|Verified Emulation| EMUL[Proven Emulation<br/>Mathematical Certainty]
SPLIT -->|Statistical Quantum| QHARDWARE[Quantum Hardware<br/>When Available]
SPLIT -->|Hybrid| MIXED[CXL-Connected<br/>CPU+QPU]
end
subgraph "Memory Integration"
BARE[BAREWire Protocol<br/>Zero-Copy]
EMUL -.->|Strongly Typed| BARE
QHARDWARE -.->|Type-Safe Transfer| BARE
MIXED -.->|Unified Memory| BARE
end
style PHG fill:#f2aa72,stroke:#4f2607,stroke-width:3px

Real-World Scenario: Financial Risk with Verified Computation

The Business Challenge

Consider a major investment bank calculating Value at Risk (VaR) across a portfolio containing millions of positions and complex derivatives. Traditional Monte Carlo simulations face two critical limitations:

Computational Time: Hours of processing for daily risk reports
Tail Risk Blindness: Rare “black swan” events are undersampled

This is a genuine quantum opportunity with a constraint. Financial regulators require mathematical proof of accuracy, not statistical confidence alone.

The Proof-Carrying Solution

Our approach draws on the full Fidelity stack. Our PHG carries the representation, posit arithmetic carries precision, Clef carries verification, and BAREWire carries zero-copy data movement:

// Financial risk calculation with formal verification
[<Requires("scenarios.Length <= maxQuantumAmplitudes")>]
[<Ensures("result.confidence >= 0.95")>]
[<Ensures("result.positErrorBound < regulatoryThreshold")>]
let calculatePortfolioRisk (portfolio: Portfolio) (market: MarketData) =
 // classical preparation: correlation matrix
 let correlations = FinancialMath.computeCorrelationMatrix market

 let tailRiskScenarios =
 match ComplianceRequirements.current with
 | RequiresProvenBounds ->
 // proven emulation, posit32_2 amplitude precision
 let oracle = TailRiskOracle.construct portfolio.scenarios
 let amplifiedSamples = QuantumAmplification.execute oracle

 // Clef tracks error accumulation through posit operations
 let errorBound = PositArithmetic.getAccumulatedError amplifiedSamples

 // BAREWire zero-copy transfer to classical analysis
 BAREWire.transferQuantumToClassical amplifiedSamples

 | StatisticalSufficient ->
 // Traditional Monte Carlo for comparison
 MonteCarloSampler.generateTailScenarios portfolio 1_000_000

 // Generate risk metrics with proof certificate
 { VaR95 = RiskMetrics.calculateValueAtRisk tailRiskScenarios
 ExpectedShortfall = RiskMetrics.calculateExpectedShortfall tailRiskScenarios
 ProofCertificate = DischargeObligations()
 ErrorBounds = PositArithmetic.getErrorAnalysis () }

For regulatory compliance, the proven emulation path using posit arithmetic discharges a proven bound on accumulated error, while BAREWire carries zero-copy data transfer between quantum emulation and classical analysis phases. Our Clef verification system generates proof certificates that demonstrate compliance with regulatory accuracy requirements.

Why Our Approach Exceeds Early Experiments

This example shows capabilities beyond what QIR or basic Q# reached:

Posit arithmetic holds precision through financial calculations where IEEE-754 would lose significant digits
Proof generation supports the regulatory compliance that statistical quantum results leave open
Zero-copy transfer via BAREWire removes the memory bottleneck between quantum emulation and classical analysis
PHG architecture transitions between control flow (data preparation) and data flow (quantum simulation) from one representation

Conclusion

Quantum optionality in our Fidelity framework takes lessons from early experiments like QIR and Q# while extending their initial reach. Our Program Hypergraph architecture, posit arithmetic, proof-carrying compilation, and zero-copy protocols mapped to native machine layouts together prepare the framework for quantum computing while keeping the emulation path verifiable and precise.

The PHG transitions between control flow and data flow representations from one semantic foundation, so we can target both traditional architectures and emerging quantum processors. With posit arithmetic’s precision for quantum amplitudes and Clef’s verification, the emulation path carries a proof certificate alongside its result. We have found no other representative implementation in the standing literature we have reviewed that pairs a quantum-classical IR with discharged error bounds in this way.

We are early in this design, and we will keep building toward the seam where verified emulation hands off to quantum hardware as that hardware arrives. That is where our current interest lies as the work continues.

Fidelity.Rx: Native Reactivity in Clef

Sun, 03 Aug 2025 00:00:00 +0000

Reactive programming sits at the intersection of practical engineering and algorithmic integrity in our Fidelity framework. While exploring reactive models, we drew on Ken Okabe’s Timeline library, a minimalist F# implementation that built a reactive system with little code. That economy was a key inspiration for Fidelity.Rx, though we evolved the concepts to fit the framework’s architectural requirements.

The driver for our Fidelity.Rx is that reactive programming patterns represent a distinct form of codata, one where the observation method registers callbacks rather than explicitly pulling. This foundation would let the Composer compiler apply the same analysis techniques to reactive code that it uses for async operations, while keeping the transparency and efficiency the framework is built around.

Architectural Update: The Signal-Actor Isomorphism

Since this entry was originally published, work on Fidelity.UI revealed a deeper structural truth: signals and actors are the same abstraction. A signal that notifies its subscribers when it changes is an actor that sends messages to its dependents. The dependency graph is the message routing topology. Batching is mailbox coalescing.

This isomorphism resolved the question of whether Fidelity.Rx should exist as a separate library. The answer, informed by the Alloy precedent, is that the concepts described here are real, but they decompose naturally into layers that already exist:

Fidelity.Rx Concept	Architectural Home
Multicast observables	Fidelity.UI signal primitives (`createSignal` - actor with subscriber set, broadcast to all)
Unicast observables	Fidelity.UI resource primitives (`createResource` - per-subscriber isolated state, arena-backed)
Coeffect composition	Composer compiler passes (codata analysis applied to reactive code)
Memory model gradient	Fidelity.Platform arena management (zero-alloc through actor-based)
Push-based codata model	Prospero/Olivier actor substrate (actors are inherently push-based)

The design thinking in this entry remains sound, particularly the push/pull duality, the multicast/unicast distinction, and the transparency principles. What changed is the realization that these don’t require a dedicated reactive library. The actor model is the reactive model. Fidelity.UI’s signal primitives provide the developer-facing API, Prospero/Olivier provides the runtime substrate, and Composer provides the compiler optimizations. The concepts land in their natural homes rather than being collected into an intermediate abstraction.

What follows is the original design exploration, preserved because the analysis of push-based codata and the multicast/unicast distinction directly informed the signal-actor architecture that emerged.

Implementation Choices: The Multicast/Unicast Distinction

Within the push-based model, a secondary but important distinction emerges: how do we handle multiple subscribers? This is not about push vs pull, since both multicast and unicast are push-based. It is about whether subscribers share execution or get isolated processing.

Multicast Observables: Shared Push

Multicast observables implement push-based codata with shared state among all subscribers:

// Multicast: One execution, many observers
type MulticastObservable<'T> = {
 mutable Last: 'T
 mutable Observers: array<'T -> unit>
 mutable Count: int
}

let temperatureSensor = Multicast.create 25.0
temperatureSensor |> Multicast.subscribe updateDisplay
temperatureSensor |> Multicast.subscribe checkThreshold
temperatureSensor |> Multicast.subscribe logToFlash

// Single push, broadcast to all
temperatureSensor |> Multicast.next 26.5 // All three observers notified

This model aligns with zero-allocation requirements. Multicast observables suit sensor data, UI events, and system notifications, where broadcast semantics are natural.

Unicast Observables: Isolated Push

Unicast observables implement push-based codata with per-subscriber isolation:

// Unicast: Independent execution per observer
type UnicastObservable<'T> = {
 Factory: Arena -> ObserverChain<'T>
 RequiredArenaSize: int64
}

// Each subscriber gets independent execution
let dataQuery = Unicast.create queryFactory

dataQuery |> Unicast.subscribe processUser1 // Independent chain
dataQuery |> Unicast.subscribe processUser2 // Separate chain

The critical insight: unicast observables require memory allocation for per-subscriber state. In Fidelity, this means they only become available when using Prospero’s arena-based memory management.

Why This Distinction Matters

The multicast/unicast choice is orthogonal to push/pull:

Push vs Pull: Who controls timing? (Producer vs Consumer)
Multicast vs Unicast: How is execution shared? (Broadcast vs Isolated)

Both dimensions matter, but push vs pull is the distinction that makes reactive programming necessary. The multicast/unicast choice is an implementation detail. It matters for performance and semantics, and stays secondary to the point that some phenomena are inherently push-based.

Push vs Pull Codata

The role our Fidelity.Rx plays in the framework rests on the distinction between pull-based and push-based codata structures. This is the duality that makes reactive programming necessary. Async/await gives us pull-based codata, while many real-world scenarios require push-based codata.

Pull-Based Codata (Async/Await)

In category theory, pull-based codata is defined by its elimination rule, the way we observe or consume it:

\text{Stream}\langle A \rangle = \nu X. 1 + A \times X

\text{observe} : \text{Stream}\langle A \rangle \to 1 + A \times \text{Stream}\langle A \rangle

This translates to Clef async patterns where the consumer explicitly requests each value:

let processData() = async {
 let! sensor = readSensor() // Pull: "I need sensor data now"
 let! processed = transform sensor // Pull: "I need transformation now"
 return processed
}

The consumer controls the timing. They decide when to request the next value. This is perfect for scenarios where you want to control pacing, like reading from a file or making API calls.

Push-Based Codata

Push-based codata inverts the control flow. Instead of consumers pulling values, producers push updates to registered observers:

\text{Observable}\langle A \rangle = \nu X. (A \to \text{Unit}) \to X

\text{register} : (A \to \text{Unit}) \to \text{Observable}\langle A \rangle \to \text{Unit}

This manifests in reactive programming:

let temperatureSensor = Multicast.create 0.0

// Register observer (push-based)
temperatureSensor |> Multicast.subscribe (fun temp ->
 printfn "Temperature changed to: %f" temp
)

// Producer pushes updates when ready
temperatureSensor |> Multicast.next 25.5 // All observers notified

The producer controls the timing. Values arrive when they’re available, not when requested. This is essential for modeling events, sensor data, user input, and other inherently push-based phenomena.

The Push/Pull Duality

Both patterns are codata with dual observation strategies:

Pull (Async): Consumer requests → Producer responds
Push (Observable): Producer notifies → Consumer reacts

Both are valid and necessary. Modeling mouse movements with pull-based async forces constant polling that wastes resources. Modeling file reading with push-based observables overwhelms the consumer with data. Each phenomenon has the observation strategy that fits it.

How Little It Takes

Push-based observables in our Fidelity.Rx take little code to implement. The core fits in a few lines:

// Push-based observable
type Observable<'a> = {
 mutable Observers: ('a -> unit) list
}

let create() = { Observers = [] }

let subscribe f obs =
 obs.Observers <- f :: obs.Observers

let next value obs =
 for observer in obs.Observers do
 observer value

// push-based reactivity, ~10 lines

For production use, we refine this into multicast (zero-allocation with arrays) and unicast (arena-based with isolation), but the core stays the same. Push-based codata is plain: functions called when events occur, with no hidden state machines, no runtime services, and no subscription management layer.

Coeffect Composition in Reactive Operations

Just as with async operations, reactive operations in Fidelity.Rx form a coeffect algebra that enables compositional analysis. When we compose reactive operations, their coeffects combine according to precise mathematical rules:

\frac{\text{source} @ R_1 \vdash \text{Observable}\langle A \rangle \quad f @ R_2 \vdash A \to B}{\text{source.map}(f) @ R_1 \sqcup R_2 \vdash \text{Observable}\langle B \rangle}

This rule states that mapping a function over an observable combines the coeffects of both the source observable and the mapping function. The $\sqcup$ operator represents the least upper bound in the coeffect semilattice.

For reactive operations, the coeffect composition follows these patterns:

\begin{align} \text{Pure} \sqcup \text{Pure} &= \text{Pure} \\ \text{Pure} \sqcup \text{UIThread} &= \text{UIThread} \\ \text{UIThread} \sqcup \text{BackgroundThread} &= \text{CrossThread} \\ \text{ResourceAccess}(S_1) \sqcup \text{ResourceAccess}(S_2) &= \text{ResourceAccess}(S_1 \cup S_2) \end{align}

This structure would enable the Composer compiler to make optimal decisions about where and how to execute reactive pipelines, whether they’re multicast or unicast.

Transparency vs Runtime Opacity

A clear difference between our Fidelity.Rx and traditional .NET reactive implementations lies in observability. The contrast shows in what happens under the hood in each approach.

The Rx.NET Black Box

In traditional .NET reactive programming, even simple operations involve multiple layers of abstraction:

// What looks simple in C#...
observable
 .Where(x => x > 0)
 .Select(x => x * 2)
 .Subscribe(Console.WriteLine);

// ...actually creates:
// - Multiple heap-allocated operator objects
// - Hidden subscription chains
// - Internal schedulers with thread pools
// - Disposable wrappers for cleanup
// - Concurrent collections for thread safety

When debugging, you encounter:

Stack traces filled with internal framework methods
Heap dumps showing mysterious AnonymousObserver<T> instances
Performance profiles dominated by allocation and GC overhead
Race conditions hidden in the depths of scheduler implementations

The runtime machinery becomes a black box that obscures the actual data flow. When a value doesn’t appear where expected, you’re left wondering: Is it stuck in a scheduler queue? Lost in a concurrent collection? Blocked by a synchronization primitive?

Fidelity.Rx Transparency

Our Fidelity.Rx takes a different approach. There is no hidden runtime machinery, yet coordination is still present: synchronization is explicit and observable.

// Multicast observable - what you see is what you get
let observable = { Current = 0; Previous = 0; Observers = [||]; Count = 0 }

// Direct observation
observable.Observers.[observable.Count] <- (fun (old, new) -> printfn "%d -> %d" old new)
observable.Count <- observable.Count + 1
observable.Previous <- observable.Current
observable.Current <- 42
for i in 0 .. observable.Count - 1 do
 observable.Observers.[i] (observable.Previous, observable.Current)

// Unicast observable - explicit arena usage
let unicast = Unicast.create 0
// Arena allocation is visible and debuggable

Every aspect is visible and debuggable:

Direct observation: Set a breakpoint and see exactly which observers are registered
Explicit updates: Watch values propagate through the system in real-time
Stack-allocated operations: No hidden heap allocations or GC pressure (for multicast)
Arena-based allocation: Explicit and visible when used (for unicast)

This transparency extends to production diagnostics. When an issue arises, you can:

Inspect the exact observer list at any moment
Trace value propagation with simple logging
Profile without wading through framework overhead
Reason about concurrency because it’s explicit through actor boundaries or arena scopes

Zero-Allocation Reactive Patterns

The plain structure of multicast observables in our Fidelity.Rx supports zero-allocation reactive programming:

// This reactive pipeline...
source
|> Multicast.map (fun x -> x * 2)
|> Multicast.filter (fun x -> x > threshold)
|> Multicast.subscribe handler

// ...compiles to something like:
source.Observers.[source.Count] <- (fun x ->
 let doubled = x * 2
 if doubled > threshold then
 handler doubled
)
source.Count <- source.Count + 1

The pipeline compiles to direct function calls, with no intermediate observables and no per-event allocation.

What Fidelity Provides “For Free”

Our Composer compiler’s coeffect and codata analysis is designed to provide infrastructure that Fidelity.Rx can leverage:

1. Efficient Suspension and Resumption

When async operations compose with reactive operations, the compiler would recognize the suspension points:

// Developer explicitly chooses multicast for broadcast semantics
let broadcastResults =
 sourceObservable |> Multicast.bind(fun value -> async {
 let! data = fetchFromServer value // Coeffect: AsyncBoundary @ Network
 return Multicast.create data // Explicit multicast choice
 })

// Or explicitly chooses unicast for isolation
let isolatedResults =
 sourceObservable |> Unicast.bind(fun value -> async {
 let! data = fetchFromServer value // Coeffect: AsyncBoundary @ Network
 return Unicast.create data // Explicit unicast choice
 })

The compiler would optimize these patterns by:

Recognizing the async boundary within the reactive bind
Preserving delimited continuations for the async portion
Enforcing allocation requirements based on the explicit model choice

2. Automatic Resource Management

RAII principles extend naturally to observable subscriptions:

// Multicast observable - lightweight subscription
use subscription =
 sensorObservable
 |> Multicast.map processSensorData
 |> Multicast.subscribe

// Unicast observable - arena-scoped subscription
use subscription =
 queryObservable
 |> Unicast.scan accumulate initial
 |> Unicast.subscribe

// Subscription automatically cleaned up at scope exit

The compiler would track subscription lifetimes as resources, ensuring deterministic cleanup without garbage collection.

3. Zero-Allocation Streaming

When multicast observable operations form pipelines, the compiler could eliminate intermediate allocations:

// Developer writes:
dataStream
|> Multicast.map transform1
|> Multicast.map transform2
|> Multicast.map transform3

// Compiler recognizes codata pattern and fuses:
dataStream
|> Multicast.map (transform1 >> transform2 >> transform3)

This optimization would emerge from recognizing that observable transformations form a category where composition is associative.

The Push-Based Reactivity Gap

Async/await covers a wide range of work, yet pull-based patterns cannot naturally express certain reactive scenarios. This is about phenomena that are inherently push-based, separate from the multicast vs unicast question:

Events Are Push-Based

User input, sensor readings, and network packets do not wait for you to pull them:

// Inefficient: Polling with pull-based async
let pollMouse() = async {
 while true do
 let! position = getMouse() // Constantly pulling
 if position <> lastPosition then
 updateUI position
 do! Async.Sleep 10 // Wasted cycles
}

// Natural: Push-based observable
let mouseEvents = Multicast.create MousePosition.origin
mouseEvents |> Multicast.subscribe (fun (oldPos, newPos) ->
 updateUI newPos) // Updates pushed when available

Multiple Concurrent Observers

When multiple components need the same data stream, pull-based patterns become awkward:

// Awkward with async: Each consumer pulls separately
let consumer1() = async { let! data = getData() in process1 data }
let consumer2() = async { let! data = getData() in process2 data }
let consumer3() = async { let! data = getData() in process3 data }
// Three separate pulls, possibly inconsistent data

// Natural with observables: Single source, multiple observers
let dataStream = Multicast.create NoValue
dataStream |> Multicast.map process1 |> Multicast.subscribe handler1
dataStream |> Multicast.map process2 |> Multicast.subscribe handler2
dataStream |> Multicast.map process3 |> Multicast.subscribe handler3
// Single push, consistent data to all

Decoupled Producer-Consumer Communication

Push-based patterns naturally decouple producers from consumers:

// Producer doesn't know or care about consumers
let sensorProducer = async {
 while true do
 let! reading = readSensor()
 sensorObservable |> Multicast.next reading
 do! Async.Sleep 1000
}

// Consumers register independently
sensorObservable1 |> Multicast.map (fun (_, curr) -> processReading curr)
 |> Multicast.subscribe (fun (_, result) -> updateDisplay result)
sensorObservable2 |> Multicast.filter (fun (_, curr) -> isCritical curr)
 |> Multicast.subscribe (fun (_, reading) -> sendAlert reading)
// Producer remains unchanged as consumers come and go

Fidelity.Rx Design Principles and API

The design of our Fidelity.Rx separates multicast and unicast observables by their allocation requirements:

Core Type Design: Model-Aware Types

module Fidelity.Rx

// Subscription model distinction at the type level
type SubscriptionModel = Multicast | Unicast

type Observable<'a, 'Model> =
 | Multicast of MulticastObservable<'a>
 | Unicast of UnicastObservable<'a>

// Multicast observable - zero allocation
and MulticastObservable<'a> =
 { mutable Current: 'a
 mutable Previous: 'a
 mutable Observers: (('a * 'a) -> unit)[]
 mutable Count: int }

// Unicast observable - requires arena for per-subscriber isolation
and UnicastObservable<'a> =
 { mutable Current: 'a
 mutable Previous: 'a
 CreateSubscription: Arena -> ('a * 'a -> unit) -> IDisposable }

// Principled handling of reactive values
type ReactiveValue<'a> =
 | HasValue of 'a
 | NoValue
 | Error of exn

Core API Design Concept

module Multicast =
 // Zero-allocation operations
 let create (initial: 'a) : Observable<'a, Multicast> =
 Multicast { Current = initial
 Previous = initial
 Observers = Array.zeroCreate 16
 Count = 0 }

 let next (value: 'a) (Multicast obs) =
 obs.Previous <- obs.Current
 obs.Current <- value
 for i in 0 .. obs.Count - 1 do
 obs.Observers.[i] (obs.Previous, obs.Current)

 let subscribe (handler: 'a * 'a -> unit) (Multicast obs) =
 if obs.Count < obs.Observers.Length then
 obs.Observers.[obs.Count] <- handler
 obs.Count <- obs.Count + 1
 else
 failwith "Observer limit reached"

 let map (f: 'a -> 'b) (source: Observable<'a, Multicast>) : Observable<'b, Multicast> =
 match source with
 | Multicast src ->
 let target = create (f src.Current)
 subscribe (fun (oldVal, newVal) ->
 next (f newVal) (Multicast target)) (Multicast src)
 Multicast target

module Unicast =
 // Arena-based operations for per-subscriber isolation
 let create (initial: 'a) : Observable<'a, Unicast> =
 Unicast { Current = initial
 Previous = initial
 CreateSubscription = fun arena handler ->
 // Each subscriber gets isolated state in arena
 let state = arena.Allocate<'a * 'a>((initial, initial))
 { new IDisposable with
 member _.Dispose() = () } }

 let subscribe (handler: 'a * 'a -> unit) (Unicast obs) =
 let arena = Arena.current()
 obs.CreateSubscription arena handler

 let map (f: 'a -> 'b) (source: Observable<'a, Unicast>) : Observable<'b, Unicast> =
 match source with
 | Unicast src ->
 Unicast { Current = f src.Current
 Previous = f src.Previous
 CreateSubscription = fun arena handler ->
 src.CreateSubscription arena (fun (old, curr) ->
 handler (f old, f curr)) }

Compiler Transformation Strategy

The Composer compiler is designed to respect the developer’s explicit choice of subscription model and optimize accordingly:

// Developer explicitly chooses multicast for shared state
let sharedCounter =
 Multicast.create 0
 |> Multicast.map (fun x -> x * 2)

// Developer explicitly chooses unicast for isolation

let isolatedCounter =
 Unicast.create (fun arena -> 0)
 |> Unicast.map (fun x -> x * 2)

// Compiler enforces constraints:
// - Multicast works everywhere (zero allocation)
// - Unicast requires arena context

// Multicast observable in MLIR - developer's explicit choice
func @counter_multicast() -> !rx.observable<i32, multicast> {
 %0 = rx.create_multicast %c0 : i32
 %1 = rx.map_multicast %0, @multiply_by_2 : (!rx.observable<i32, multicast>, (i32) -> i32) -> !rx.observable<i32, multicast>
 return %1 : !rx.observable<i32, multicast>
}

// Unicast observable in MLIR - developer's explicit choice
func @counter_unicast() -> !rx.observable<i32, unicast> {
 %arena = arena.current : !arena.ref
 %0 = rx.create_unicast %arena, @factory : (!arena.ref) -> !rx.observable<i32, unicast>
 %1 = rx.map_unicast %0, @multiply_by_2 : (!rx.observable<i32, unicast>, (i32) -> i32) -> !rx.observable<i32, unicast>
 return %1 : !rx.observable<i32, unicast>
}

Observable Collections

Beyond single-value observables, Fidelity.Rx provides reactive collections that adapt to multicast/unicast semantics and maintain alignment with Fidelity’s memory model tiers.

The Memory Model Gradient

Reactive collections in Fidelity.Rx follow the same progression as Fidelity’s overall memory architecture, with multicast/unicast distinctions at each level:

graph TD
subgraph "Memory Model Tiers"
L1[Level 1: Zero Allocation<br/>Multicast observables only]
L2[Level 2: Arena Memory<br/>Unicast observables available]
L3[Level 3: Linear Types<br/>Ownership transfer]
L4[Level 4: Actor Model<br/>Distributed observables]
end
subgraph "Observable Models"
M[Multicast Collections<br/>Shared state, broadcast]
U[Unicast Collections<br/>Per-subscriber state]
end
L1 --> M
L2 --> U
L3 --> M
L3 --> U
L4 --> U

Level 1: Zero Allocation - Multicast Only

At the simplest level, when coeffect analysis determines single-threaded usage with no concurrent access:

// Single-threaded scenario - multicast observables only
let processLocalData (data: float[]) =
 // Compiler detects: Pure + SingleThreaded coeffects
 let results = MulticastObservableList.create()

 // All operations happen on the stack
 for value in data do
 if value > threshold then
 results |> MulticastObservableList.add (transform value)

 // Broadcast to all observers
 results |> MulticastObservableList.toArray

Level 2: Arena-Scoped Unicast Observables

When collections need per-subscriber state:

type DataProcessor() =
 // Arena provides lifetime management
 let arena = Arena.create { Size = 10<MB>; Strategy = Sequential }

 // Multicast collections for shared state
 let sharedCache = MulticastObservableMap.create()

 // Unicast collections for per-subscriber processing

 let processingPipeline =
 UnicastObservableList.create()
 |> Unicast.scan accumulateResults Results.empty

 member this.Process(input: DataBatch) =
 // Multicast observable for broadcast
 sharedCache |> MulticastObservableMap.set input.Key input

 // Unicast observable for isolated processing
 processingPipeline |> Unicast.next input

Level 3: Linear Types for Ownership Transfer

For scenarios requiring explicit ownership transfer:

// Linear types enable safe mutation with ownership tracking
module LinearCollections =
 type LinearList<'a, 'Temp> =
 private | Linear of Observable<list<'a>, 'Temp> * LinearToken

 // Operations consume and return ownership
 let add (item: 'a) (Linear(obs, token)) =
 match obs with
 | Hot hot ->
 // In-place update for hot
 hot.Last <- item :: hot.Last
 Linear(Hot hot, token)
 | Cold cold ->
 // Arena-based for cold
 failwith "Use Cold.add for cold observables"

Level 4: Actor-Based Coordination

For distributed scenarios:

// Actor-based collections with temperature awareness
type DistributedCache() =
 inherit Actor<CacheMessage>()

 // Hot observable for system-wide events
 let cacheEvents = Hot.create CacheEvent.Empty

 // Cold observable for per-client queries
 let clientQueries = Cold.create (fun arena ->
 arena.Allocate<QueryProcessor>())

 override this.Receive message =
 match message with
 | SystemEvent event ->
 // Multicast to all via hot observable
 cacheEvents |> Hot.next event

 | ClientQuery(query, replyTo) ->
 // Per-client processing via cold observable
 clientQueries
 |> Cold.map (fun processor -> processor.Execute(query))
 |> Cold.subscribe (fun result -> replyTo <! result)

Memory Management for Reactive Systems

RAII integration scales differently for multicast and unicast observables:

Multicast: Stack-Based RAII

// Simple subscription with stack-based cleanup
let processEvents (source: Observable<Event, Multicast>) =
 use subscription = source |> Multicast.subscribe handleEvent

 // Process until condition met
 while continueProcessing() do
 doWork()

 // Subscription automatically disposed at scope exit
 // No heap allocation, no GC pressure

Unicast: Arena-Based RAII

type DataPipeline() =
 // Arena owns unicast observable resources
 let arena = Arena.create {
 Size = 50<MB>
 Strategy = GrowthOptimized
 }

 // Unicast observables allocated within arena
 let processed = UnicastObservableList.create()

 member this.ConnectSensor(sensor: Sensor) =
 arena.transaction (fun () ->
 // Unicast subscription within arena
 let sub = sensor.DataStream
 |> Unicast.subscribe (fun (old, curr) ->
 match processReading curr with
 | Ok result -> processed.Add(result)
 | Error e -> logError e)

 subscriptions.Add(sub)
 )

Cross-Model References

When multicast and unicast observables interact:

module ModelBridge =
 // Convert multicast to unicast (uses arena for isolation)
 let toUnicast (multi: Observable<'T, Multicast>) : Observable<'T, Unicast> =
 let arena = Arena.current()
 Unicast.create (match multi with
 | Multicast m -> m.Current
 | _ -> failwith "Type error")

 // Convert unicast to multicast (loses isolation)
 let toMulticast (uni: Observable<'T, Unicast>) : Observable<'T, Multicast> =
 let shared = Multicast.create (match uni with
 | Unicast u -> u.Current
 | _ -> failwith "Type error")
 // Set up forwarding from unicast to multicast
 uni |> Unicast.subscribe (fun (_, curr) ->
 shared |> Multicast.next curr) |> ignore
 shared

Spec Example: Sensor Network

Let’s ruminate on how multicast and unicast observables might work together:

module SensorNetwork =

 // Embedded sensor nodes - multicast only
 module SensorNode =
 let temperature = Multicast.createBounded<float, 8> 0.0
 let pressure = Multicast.createBounded<float, 8> 0.0

 // Broadcast to all subsystems
 temperature |> Multicast.subscribe transmitReading
 temperature |> Multicast.subscribe updateLocalDisplay

 // Timer interrupt updates all
 let TIMER_ISR() =
 temperature |> Multicast.next (ADC.read TEMP_CH)
 pressure |> Multicast.next (ADC.read PRESS_CH)

 // Edge gateway with Prospero
 module EdgeGateway =
 type TelemetryActor() =
 inherit Actor<TelemetryMessage>()

 // Arena enables unicast observables
 let arena = Arena.forCurrentActor()

 // Multicast for real-time broadcast
 let realtimeData = Multicast.create TelemetryData.empty

 // Unicast for per-client analysis
 let analysisPipeline =
 Unicast.create TelemetryData.empty

 override this.Receive message =
 match message with
 | SensorData data ->
 // Update multicast (broadcast)
 realtimeData |> Multicast.next data

 // Trigger unicast (per-subscriber)
 analysisPipeline |> Unicast.next data

 // Cloud aggregation - unicast observables with BAREWire
 module CloudAggregator =
 let createAggregationPipeline (source: DataSource) =
 source.GetStream()
 |> Unicast.window (TimeSpan.FromMinutes 5.0)
 |> Unicast.scan aggregateWindow WindowStats.empty
 |> Unicast.map detectAnomalies
 |> Unicast.subscribe alertOperations

Integration with UI: From Fabulous to Fidelity.UI

The original design explored adapting Fabulous to the Fidelity framework. Subsequent work revealed that the signal-actor isomorphism offers a more direct path: rather than bridging reactive observables into an MVU/virtual-DOM framework, the reactive primitives become the UI primitives directly.

In Fidelity.UI, the multicast/unicast distinction maps to signal primitives that developers already understand from SolidJS and Partas.Solid:

// What Fidelity.Rx described as Observable<'T, Multicast>
// becomes createSignal - broadcast to all subscribers
let searchText, setSearchText = createSignal ""
let isLoading, setIsLoading = createSignal false

// What Fidelity.Rx described as Observable<'T, Unicast>
// becomes createResource - per-subscriber isolated state with arena backing
let results = createResource
 (fun () -> searchText() |> Some)
 (fun query _ -> searchApi query)

// Fidelity.UI component - signals drive rendering directly
[<Component>]
let SearchView () =
 Stack() {
 TextInput(value = searchText(), onInput = fun e -> setSearchText e.Value)

 Show(when' = isLoading()) { Spinner() }
 |> withFallback (
 For(results.current, key = fun r -> r.Id) (fun result _ ->
 ResultCard(result)
 )
 )
 }

The multicast/unicast distinction is still present, but expressed through familiar signal vocabulary rather than explicit observable types. createSignal is multicast: a value broadcast to all subscribers with zero-allocation semantics. createResource is unicast: per-subscriber isolated state backed by Prospero’s arena management. The coeffect analysis described earlier applies identically, and the Composer compiler recognizes these as codata patterns and optimizes accordingly.

This resolution echoes the Alloy absorption: the abstraction was real, but its natural expression was through existing architectural layers rather than a standalone library.

The Architectural Synthesis

Mathematical Structure

Multicast observables form a comonad with shared state
Unicast observables form a monad with isolated state
Coeffect tracking carries context through to compilation
Subscription model inference guides optimization

Performance

Multicast observables: zero-allocation for embedded systems
Unicast observables: arena-based for predictable cleanup
Compiler fusion based on subscription model analysis
Allocation and runtime behavior visible at the call site

Developer Ergonomics

Explicit models make performance predictable
Progressive disclosure: start multicast, add unicast when needed
Familiar FRP patterns with clear allocation boundaries
Integration with Prospero and BAREWire when available

Future Directions

With the signal-actor isomorphism as the foundation, the reactive concepts described here continue to evolve through their architectural homes:

Compile-Time Signal Analysis (Composer)

Composer’s coeffect passes can analyze reactive graphs at compile time, applying the same subscription-model awareness described in this entry:

// Composer detects: signal with scan pattern shares state across all subscribers
let count, setCount = createSignal 0
let accumulated = createMemo (fun () -> count() + previousTotal)
// Compiler recognizes: memo is a multicast derived computation (zero allocation)

// Composer detects: resource with single subscriber
let data = createResource source fetcher
// Compiler can optimize: unicast semantics, arena-scoped, no broadcast overhead

The compiler would provide warnings and optimization based on usage patterns, while the choice of primitive remains with the developer.

Hardware-Accelerated Signal Actors (Embedded)

For embedded targets, we intend signal actors to compile to interrupt-driven patterns with zero overhead:

let temperature, setTemperature = createSignal 0
let filtered = createMemo (fun () ->
 let t = temperature()
 if t > minValid && t < maxValid then t else temperature.Previous)

createEffect (fun () -> updateControl (filtered()))
// Compiles to zero-overhead interrupt service routine
// The signal-actor graph becomes direct function calls

Distributed Signal Actors (Prospero)

With the unified actor architecture, signal actors naturally distribute across process and machine boundaries via BAREWire. A signal change on one node can notify subscribers on another, with the protocol/transport separation ensuring the reactive semantics are preserved regardless of whether messages travel through IPC, WebSocket, or MoQ channels.

From Fidelity.Rx to Signal-Actor Isomorphism

The design exploration in this entry recognized two orthogonal distinctions in reactive systems:

Push vs Pull: The duality of codata. Who controls timing?
Multicast vs Unicast: An implementation choice. How is execution shared?

Both distinctions remain valid and important. What evolved was the realization that these don’t require a dedicated reactive library to express. The signal-actor isomorphism reveals that the actor model already embodies push-based reactivity. An actor with a subscriber set is a multicast observable. An actor that spawns per-subscriber state in an arena is a unicast observable. Mailbox coalescing is batching. The dependency graph is the message routing topology.

The concepts land in their natural architectural homes:

Embedded developers use createSignal, which compiles to zero-allocation multicast actors, with the same confidence described here about no hidden allocations
Application developers use createResource and createStore, which compile to arena-backed unicast actors, providing isolation without garbage collection
System architects choose between signal primitives (Fidelity.UI) and actor primitives (Prospero/Olivier) based on whether they’re building UI or infrastructure
The compiler applies coeffect analysis to all of these uniformly, because they’re all codata patterns over the same actor substrate

The Alloy precedent proved instructive. Just as Alloy’s concepts were real but decomposed into Fidelity.Platform + CCS, Fidelity.Rx’s concepts are real but decompose into Fidelity.UI + Prospero/Olivier + Composer. The intermediate abstraction served its purpose as a thinking tool, then yielded to the layers that were always there.

Reactive programming in our Fidelity ecosystem does not need a separate reactive library. Reactivity is intrinsic to the actor model, signals are the developer-facing vocabulary for that reactivity, and the compiler can optimize both because they share the same mathematical foundation: push-based codata with explicit subscription semantics.

These are the abstractions that already contain the reactive concepts this entry set out to build. They are meant to scale from microcontrollers (zero-allocation signal actors) to distributed cloud systems (BAREWire-connected actor networks), with the same predictable performance and transparent operation. That is the direction we keep building toward as the signal-actor architecture takes shape across the framework.

Hardware Lessons from LISP

Tue, 22 Jul 2025 00:59:54 +0600

The computing industry sits at a notable juncture in 2025. After decades of general-purpose processor dominance that led to the accidental emergence of general purpose GPU, we’re witnessing what appears to be a reverse inflection point. Specialized architectures are re-emerging as an economic imperative, with important differences from the LISP machines of the past. We examine here how languages inheriting from LISP’s legacy, particularly Clef and others with lineage to OCaml and StandardML, are positioned to realize the advantages of new hardware coming from vendors like NextSilicon, Groq, Cerebras and Tenstorrent. We call this concept Dataflow Graph Architecture (DGA).

The Phoenix Pattern: An Electric Car Saga

Before diving into the LISP machine story, consider a striking historical parallel: the rise, fall, and resurrection of electric vehicles. In the late 1890s and early 1900s, electric cars weren’t just viable; they dominated. By 1900, electric vehicles held the land speed record. In 1912 United States, 38% of automobiles were powered by electricity, with over 33,000 electric cars registered. They were the preferred choice of well-heeled urban customers, featuring luxurious interiors and quiet, crank-free operation.

An employee of the Great Eastern Railway Company driving a battery-powered rail parcel truck somewhere in Britain c. 1914-1918 [Wikipedia]

Yet by 1920, electric vehicles had virtually vanished from the market. The culprits? Henry Ford’s mass-produced Model T cost three times less than a typical electric car. Gasoline’s energy density provided 20-30 times the range. The electric starter eliminated hand-cranking, removing one of electric’s key advantages. Infrastructure development favored gasoline distribution over electrical grids.

Sound familiar? This is the pattern that played out with LISP machines seventy years later. Superior in many technical dimensions, defeated by economics and ecosystem realities, only to return when constraints made the “old” approach newly essential. For electric vehicles those constraints were energy efficiency and the promise of reduced environmental impact. For specialized processors the analog is computational efficiency.

Today’s electric vehicle renaissance isn’t merely nostalgia. Tesla’s market capitalization exceeds that of the next nine automakers combined. Governments worldwide are mandating the phase-out of internal combustion engines. The technology that seemed permanently obsolete has returned, transformed by lithium-ion batteries, modern power management, and most critically an environmental imperative that changed the entire cost equation.

This phoenix pattern, where superior but economically disadvantaged technologies lie dormant until external pressures resurrect them in evolved forms, is what we’re witnessing with specialized computing architectures.

The First Golden Age of AI Hardware

To understand where we’re heading, we must first appreciate where we’ve been. In the 1980s, LISP wasn’t just another programming language, it was the language of artificial intelligence. From MIT’s AI Lab to Stanford’s Knowledge Systems Laboratory, if you were working on AI, you were almost certainly working in LISP.

The UX-1200 board from Symbolics. It could be plugged into a Sun VME-Bus Server. Each Lisp machine (a Sun could host up to five) needed a “world”, which contained their files and programs. From the outside view this is a (300 MB and more) file.

The dominance was staggering. By the mid-1980s, the AI industry had grown to over $1 billion annually (before adjustment for inflation), with LISP machines representing a significant portion of that market. Companies like Symbolics, LISP Machines Inc. (LMI), Texas Instruments, and Xerox commanded premium prices for their specialized hardware. A single Symbolics 3600 workstation cost $70,000-80,000 in 1983 (approximately $240,000 in 2024 dollars), yet research labs and corporations eagerly paid these prices.

What made LISP machines so compelling? They offered hardware-accelerated features that had no equivalent on the general-purpose hardware of the time:

Tagged memory architectures that performed runtime type checking in hardware.
Graph-based memory models where pointers were edges and data structures were subgraphs
Hardware-assisted garbage collection that made memory management transparent
Message-passing primitives built into the architecture
Microcoded instructions optimized for list processing and symbolic computation

These weren’t performance optimizations. They represented a different view of computation. LISP machines viewed computation as a graph-structured process. Every piece of data carried its type, enabling dynamic hardware dispatch. This tenet of embedding data semantics directly into the hardware is what modern neuromorphic processors and Coarse-Grained Reconfigurable Arrays (CGRAs) are rediscovering. These new architectures are designed to understand beyond the nature of an operation to what kind of data it is operating on. This enables levels of parallelism and efficiency that were dismissed as overengineering when general-purpose processors won on cost.

When That Market Fell Off a Cliff

The collapse of LISP machines, when it came, was swift and brutal, much like how electric vehicles disappeared from roads by 1920. What had been a billion-dollar industry dominated by companies like Symbolics and LISP Machines Inc. was effectively extinct by 1992, replaced by general-purpose RISC workstations that cost three times less while delivering 2-4 times the raw performance.

Just as gasoline cars won on cost and convenience despite electric vehicles’ technical merits, RISC workstations crushed LISP machines through economic force. The LISP machine’s downfall wasn’t merely economic. Software advances in garbage collection algorithms and compiler technology progressively reduced the need for specialized hardware. By the late 1980s, Common LISP compilers on RISC workstations matched the performance of LISP machines for most applications. The broader AI Winter of 1987-1993 delivered the final blow, as expert systems failed to meet inflated expectations and military funding for AI research evaporated. [2]

But here’s what the industry overlooked in its focus on cost-efficiency:

LISP machines weren’t wrong about the nature of computation, they were ahead of their time and constrained by the economics and technologies of their era.

Like electric vehicles waiting for lithium-ion batteries and climate urgency, LISP’s architectural insights would need to wait for their resurrection moment.

The Pendulum Swings Back

Decades later, the architectural landscape is transforming in ways that mirror the LISP machine era while learning from its failures. Energy efficiency has emerged as the primary constraint driving innovation, with AI workloads consuming unprecedented amounts of power. This is the same pressure that brought electric vehicles back to relevance. It has catalyzed development of different approaches to computing that sacrifice generality for domain-specific optimization, with one further change: these new architectures move past traditional Instruction Set Architectures (ISAs) to embrace a spatial computing model.

This puts a double burden on the developer community. There’s an imperative to move faster, do more, and create more opportunities in less time. Dataflow architectures offer this inherently, but it requires new thinking. This new paradigm can be aided by the valuable lessons learned in previous technology eras.

Dataflow Architectures: More Than ISAs

Traditional ISAs have served sequential computation well, embedding assumptions that made sense for their era:

Sequential Execution: ISAs assume a program counter advancing through memory, perfect for single-threaded work
Temporal Ordering: Operations happen “before” or “after” each other, ideal for deterministic execution
Thread-Based Control: Even parallel ISAs model multiple sequential threads, a natural extension of the sequential model
Imperative Commands: Instructions tell the processor what to do next, clear and unambiguous

Modern dataflow architectures, CGRAs, and neuromorphic systems build upon these foundations while adding new dimensions:

Spatial Execution: Operations exist at physical locations, complementing temporal sequencing
Data-Triggered Execution: Operations fire when inputs arrive, enabling natural parallelism
Massive Parallelism: Thousands of operations execute simultaneously without coordination overhead
Declarative Configuration: The “program” is a graph topology that describes relationships rather than sequences

As a pointed example of this, SambaNova speaks to these concepts directly: “The RDA does not have a fixed instruction set architecture (ISA) like traditional architectures, but instead is programmed specifically for each model resulting in a highly optimized, application-specific accelerator.” The question then turns to how the current “AI” boom is being supported, where it came from, and why the Von Neumann underpinnings of modern GPU architecture is part of the critical power problems that are limiting the growth of AI.

GPUs as Accidental AI Infra

Before examining today’s purposefully designed dataflow accelerators, we must acknowledge NVIDIA’s accidental dominance. Unlike LISP machines’ deliberate design for symbolic AI, GPUs stumbled into their AI role through fortunate accidents. Stanford researcher Ian Buck observed: “We started by seeing a fit for matrix multiplies and linear algebra… render[ing] a triangle that could do a matrix multiply.” [13] The watershed moment came with AlexNet’s 2012 ImageNet victory using just two $500 GTX 580 GPUs.

This accidental infrastructure, subsidized by gaming economics, democratized AI & HPC research in ways LISP machines never could. But what was tolerable as an externality for video games, power-hungry GPUs consuming 300-500 watts for entertainment has metastasized into a global crisis. Training a single large language model now consumes as much electricity as thousands of homes use in a year. Data centers housing GPU clusters strain power grids, consume millions of gallons of water for cooling, and contribute meaningfully to carbon emissions. The same architectural inefficiencies that gamers accepted for better frame rates now threaten the sustainability of AI advancement itself.

This environmental and economic crisis isn’t a minor optimization problem. It is a challenge that demands architectural change, the same forcing function that resurrected electric vehicles. GPUs, never designed for AI, force square-peg parallel computation through round-hole graphics pipelines. Their power consumption scales steeply with model size, creating a hard ceiling on AI progress defined not by algorithmic limits but by the physics of heat dissipation and the economics and logistics of power generation.

The Renewed Dataflow Landscape

The non-sustainability of the GPU trajectory has catalyzed a renaissance in computer architecture. Companies aren’t just seeking marginal improvements, they’re racing to find different approaches before the current paradigm collapses under its own power consumption. This isn’t speculative. Major tech companies are already constrained by power availability, not computational capacity.

Reversible computing pioneers like Vaire suggest orders of magnitude in energy improvements by avoiding information erasure entirely. [3] Wafer-scale integration from Cerebras packs 4 trillion transistors onto a single die, achieving 21 PB/s memory bandwidth. [4] RISC-V’s extensibility enables heterogeneous architectures like Tenstorrent’s combination of general-purpose cores with specialized accelerators. [6]

The emergence of neuromorphic architectures represents perhaps the most radical departure from Von Neumann principles. Companies and research institutions are developing brain-inspired processors that compute through networks of artificial neurons and synapses. Intel’s Loihi 2 processor exemplifies this approach with 1 million programmable neurons interconnected through 120 million synapses, all while consuming mere milliwatts of power. Unlike traditional processors that separate memory and computation, Loihi 2 co-locates both functions in its neuron cores, enabling asynchronous, event-driven computation that processes information only when spikes occur.

This spike-based computing paradigm achieves high efficiency for specific workloads. Intel has demonstrated solving optimization problems 50 times faster and with 100 times less energy than conventional CPUs. The architecture excels at tasks requiring adaptation and learning, from odor recognition to robotic control, processing sensory data streams with latencies measured in microseconds.

The Intel Loihi 2’s exposed die reveals its 128 neuromorphic cores arranged in a mesh topology, each containing up to 8,192 neurons that communicate through ‘synaptic’ connections.

There are many other designs that are at various stages of development. The Fraunhofer EMFT, working within the EU project NeurONN, is developing neurologically inspired computer architectures using novel 2D materials for memristor applications that are up to 330 times more efficient than current technologies in terms of switching speed, lifetime and energy consumption. [15] These neuromorphic systems encode information through coupled oscillating elements, mimicking the distributed computing and storage of biological neural networks. The approach would have seemed familiar to LISP machine architects, who understood computation as graph transformation decades before the current generation of processors.

Each of these approaches shares a common thread: they to one degree or another describe computation in terms of graphs, not instruction sequences. This is where our Clef language, and in particular the Composer compiler within our Fidelity framework, is designed to reach the potential of these architectures while continuing to support ‘standard die’ CPU, GPU and TPU platforms. We have found no other representative implementations targeting this full range in the standing literature we have reviewed.

Functional Takes Center Stage (Again)

The architectural features driving these new designs create a favorable environment for functional programming languages, but not for the reasons you might expect. The reason is not functional purity or mathematical structure for its own sake. It is how that semantic model aligns with dataflow architectures’ execution model, the same way electric vehicles’ instant torque and simplified drivetrains exceed internal combustion’s complexity once battery technology caught up.

Direct Lineage: LISP → ML → OCaml → Clef

The most authentic line of descent runs through the ML family. When Robin Milner created ML in the 1970s, he preserved LISP’s core computational model, computation as graph reduction, while adding static typing. This lineage represents languages that maintain LISP’s essential character: viewing computation as transformation of immutable structures through a graph of operations.

Clef as a Modern Dataflow Bridge

Clef occupies a distinct position as a modern functional language with both the theoretical foundations and practical tooling for dataflow and control flow architectures. Clef descends from F#, and through it from the ML family, and carries forward a set of features that extend beyond the OCaml core. That inheritance is what makes Clef well-suited to this emerging hybrid paradigm:

Computation Expressions are one of the features Clef carries forward from its F# lineage that extend beyond OCaml. These enable building domain-specific languages with custom control flow, state management, and sequencing semantics. Unlike monads in other languages that focus primarily on sequencing effects, computation expressions provide a general framework for defining how computations compose, a fit for describing dataflow graphs where the composition rules determine how operations connect and execute. The feature brings readable syntax to what would otherwise require heavier type machinery.

Asynchronous Workflows trace to 2007, when Don Syme pioneered the async/await pattern in F# that over the years proliferated throughout other programming languages. A continuation-based pattern for managing asynchronous operations, async workflows proved valuable for both sequential and parallel composition. They handle the reality that modern systems must coordinate I/O, network calls, and parallel computations without blocking threads. In the context of dataflow architectures, async workflows provide a foundation for expressing computations that can be composed and scheduled efficiently, though the reactive patterns that match dataflow’s event-driven nature would come from other Clef libraries and patterns built atop this async foundation.

Units of Measure provides compile-time dimensional analysis that becomes essential when compiling to spatial architectures and native hardware. Clef inherits Andrew Kennedy’s units-of-measure inference through its F# lineage. The feature, absent from OCaml and most functional languages, turns what would be runtime errors into compile-time guarantees.

For example, in dataflow and spatial computing architectures, confusing timing with data flow creates subtle bugs that are nearly impossible to debug at runtime. Units of Measure catch these at compile time:

// Dataflow timing and throughput units
[<Measure>] type token // Data tokens in flight
[<Measure>] type stage // Pipeline stages
[<Measure>] type ns // Nanoseconds

type DataflowNode<'input, 'output> = {
 Latency: int<cycles>
 Throughput: int<token/cycles>
 BufferDepth: int<token>
}

let calculateBackpressure (node: DataflowNode<_,_>) (clockRate: int<cycles/ns>) =
 // would not compile: can't add tokens to cycles
 // let invalid = node.BufferDepth + node.Latency

 let maxTokensInFlight = node.BufferDepth
 let processingCapacity = node.Throughput * node.Latency
 min maxTokensInFlight processingCapacity // Result: int<token>

Clef’s Units of Measure are erased at runtime, so this compile-time safety carries no runtime cost. In our compilation design, types are carried through MLIR as attributes to serve memory mapping and optimization, then elided where appropriate to leave primitive numeric types while preserving the verified dimensional consistency. This suits the goal of dependency-free, native compilation: the checking happens during development, and the compiled output runs as plain numerics.

Clef’s Agent-Based Concurrency via MailboxProcessor, inherited from Erlang’s actor model, provides a message-passing primitive that aligns with dataflow semantics. Each agent maintains its own state and processes messages asynchronously, essentially a dataflow node that activates on message arrival. This Erlang-inspired feature means Clef developers already have the optionality to think in terms of isolated computational units communicating through messages, exactly the mental model needed for dataflow architectures where data packets trigger computation as they arrive at processing junctures.

Type Providers enable compile-time integration with external data sources, a critical capability for real-world dataflow systems that must ingest, validate, and process heterogeneous data streams. Unlike traditional approaches that discover data format errors at runtime, Clef’s type providers generate strongly-typed representations from external schemas at compile time. This means a dataflow graph processing financial feeds, sensor streams, or scientific datasets knows the exact shape and constraints of its inputs before a single operation executes.

This capability extends to emerging standards like the Hypergraph Interchange Format (HIF), which provides a unified JSON schema for higher-order network data. Clef’s type providers could automatically generate type-safe representations from HIF-compliant datasets, integrating complex relational data, from co-authorship networks to chemical reactions to biological interactions, directly into computational pipelines.

This isn’t just about building smarter chatbots.

It’s about enabling cross-disciplinary computation where a materials science simulation can safely consume protein interaction networks, or where economic models can incorporate social network dynamics with compile-time guarantees of data compatibility. This brings dataflow to the messy, heterogeneous data streams that define real-world computation.

What is Old is New Again

The current specialized architecture renaissance differs from the LISP machine era in ways that suggest greater staying power, much as today’s electric vehicle revolution has economic and environmental tailwinds the 1900s pioneers lacked:

Economic Incentives Align Differently

Energy costs and AI compute demands create sustained market pressure. Training large language models costs millions of dollars per run [9], justifying specialized hardware investments unthinkable in the 1980s. The market is also broader, from edge inference to datacenter training, providing multiple revenue streams. Just as electric vehicles now benefit from carbon pricing, government incentives, and corporate ESG commitments, specialized AI hardware rides a wave of economic forces that didn’t exist for LISP machines.

Technical Advantages Are More Durable

While compiler improvements eroded LISP machines’ advantages (just as improved gasoline engines temporarily defeated early electric cars), the physical limits of energy dissipation (Landauer’s principle) and memory bandwidth (the memory wall) create harder constraints that software alone cannot overcome. Dataflow architectures address these fundamental physics limitations, not just software inefficiencies. Electric vehicles similarly now benefit from fundamental physics advantages: electric motors are inherently more efficient than combustion engines, a fact that no amount of gasoline engine optimization can overcome.

Open Ecosystems Enable Innovation

Modern dataflow architectures benefit from lessons learned about ecosystem development. MLIR provides common compilation infrastructure in a continuation-passing style “hidden” in C++. An emerging ecosystem around dataflow architectures will inevitably lead to standards and practices for vendor-neutral representations as a common entry point for software developers. One of the main bottlenecks of the LISP era was actually the closed-source nature of the business environment. There was some proliferation of ideas among academic institutions. But the shared resources of the open source model is one of the ways that make this era of intelligent systems design particularly promising and materially different from previous technology eras.

Dataflow Graph Architectures

The evidence supports this theory of a “reverse inflection point”, with important differences. We’re not simply returning to specialized architectures, we’re expanding our computational model to encompass both sequential and spatial paradigms. Dataflow Graph Architectures complement traditional instruction-based computing by representing the natural parallelism in modern workloads: graphs of operations triggered by data availability, working alongside sequential control where it makes sense. Actor model architectures and other distributed systems concepts have been pointing in this direction for years.

For developers, this means not hard-pivoting but rather enriching how we express computation. We can still ask “what happens next?” when sequence matters, while also asking “what depends on what?” when parallelism dominates. We can manage state through time when appropriate, while describing transformations through space when beneficial. We can write sequential code where it’s natural, while preserving inherent parallelism where it emerges.

The languages best positioned for this transition are those that maintained graph-reduction principles alongside sequential capabilities. Clef and its ML-family siblings, with their dual heritage from LISP and practical programming, treat computation as transformation while supporting imperative patterns where helpful. Their type systems provide the safety that untyped dataflow would lack. Their functional patterns map naturally to spatial computation while retaining the ability to express sequential logic clearly.

Implications for Fidelity and Beyond

For our Fidelity framework, the timing lines up. As specialized architectures mature over the next few years, frameworks that can:

Preserve computational intent through hypergraph representations
Support arbitrary numeric types (like posits) natively
Maintain proof obligations through compilation
Target diverse hardware through generalized dataflow and control flow representations

…will become increasingly valuable. Our PHG isn’t just another intermediate representation, it’s a native encoding that can pivot between dataflow computation and control flow mechanics while preserving the structure each architecture needs.

A Final Homage to LISP

The LISP machines’ ghost reads to modern computing as vindication rather than cautionary tale. Their graph-based memory, tagged architectures, and message-passing primitives weren’t wrong, they were both of and ahead of their time. As the marketplace takes up native dataflow architectures that expand beyond traditional instruction sets, the vision the LISP pioneers glimpsed comes back into reach, alongside the sequential foundations built since.

The phoenix pattern shows that good architectural ideas often hibernate rather than die. Electric vehicles waited ninety years for their resurrection. LISP machines waited forty. Both returned as evolved solutions to problems their original incarnations couldn’t have imagined. The lesson from both is that being right about architecture isn’t enough when the timing and the ecosystem aren’t there. In 2025, with AI’s appetite for efficient computation, climate pressure on power-hungry compute, and an open ecosystem forming around dataflow architectures, the timing has caught up. The graph-based, type-aware, parallel-by-default model that LISP machines imagined is coming into reach, enhanced by, not replacing, the sequential computing foundations built over seven decades.

This is the ground we want our Clef language and our Fidelity framework to stand on as specialized hardware matures, and it is where our attention stays as the work continues.

References

Computer History Museum. (1992). 1992 Timeline of Computer History.
Wikipedia. (2024). AI winter.
IEEE Spectrum. (2024). Reversible Computing Has Potential For 4000x More Energy Efficient Computation.
Cerebras. (2024). Product - Chip.
Next Platform. (2020). Groq Shares Recipe for TSP Nodes, Systems.
Tom’s Hardware. (2024). Tenstorrent Shares Roadmap of Ultra-High-Performance RISC-V CPUs and AI Accelerators.
Wikipedia. (2024). Tail call.
Madhavapeddy, A. (2024). Programming FPGAs using OCaml.
IMF eLibrary. (2024). The Economic Impacts and the Regulation of AI: A Review of the Academic Literature and Policy Actions. IMF Working Papers Volume 2024 Issue 065.
Modular. (2024). What about the MLIR compiler infrastructure? (Democratizing AI Compute, Part 8)
Wikipedia. (2024). Jensen Huang.
Tom’s Hardware. (2024). Intel’s former CEO reportedly wanted to buy Nvidia for $20 billion in 2005 , Nvidia is worth over $3 trillion today.
NVIDIA Developer. (2015). Inside the Programming Evolution of GPU Computing.
Wikipedia. (2024). AlexNet.
Fraunhofer EMFT. (2025). Energy-efficient neuromorphic computing.

Danger Close: Why Types Matter

Tue, 24 Jun 2025 10:00:00 +0000

A startup’s gene analysis samples nearly melted because someone confused Fahrenheit and Celsius in their monitoring system. A Mars orbiter was lost because of mixed metric and imperial units. Medication dosing errors have killed patients due to milligrams versus micrograms confusion. These are not edge cases. They are symptoms of a recurring problem in how we build mission-critical systems:

Most languages approach types as an afterthought rather than a first line of defense.

The Near-Disaster That Should Never Have Been Possible

Consider a gene analysis startup that nearly lost years of irreplaceable samples due to a simple type confusion. A developer accidentally configured their freezer monitoring system to read temperatures in Fahrenheit while leaving alarm thresholds in Celsius. When 32°F (0°C) triggered alerts, they were dismissed as a configuration error.

The real crisis came over the weekend when an electrical contractor tripped every circuit breaker in the building and left without telling anyone. The monitoring system, disabled due to the earlier “false” alarms, failed to notify anyone of the power loss. By pure luck, the samples survived. This near-catastrophe exposed a flaw in the design: their solution allowed temperature values to exist without their units, creating a disaster waiting to happen.

This wasn’t a failure of testing, monitoring, or procedures. It was a failure of types. The system allowed temperature values to exist without their units of measure, creating a ticking time bomb that eventually exploded.

Clef Units of Measure: Your First Line of Defense

Our Clef language carries units of measure with zero runtime cost. We have found no other representative implementations of native zero-cost units of measure in the standing literature we have reviewed. Here is how that freezer monitoring system would be written in Clef:

[<Measure>] type celsius
[<Measure>] type fahrenheit
[<Measure>] type kelvin

// Temperature conversion functions that preserve units
let celsiusToFahrenheit (temp: float<celsius>) : float<fahrenheit> =
 temp * 9.0<fahrenheit/celsius> / 5.0 + 32.0<fahrenheit>

let fahrenheitToCelsius (temp: float<fahrenheit>) : float<celsius> =
 (temp - 32.0<fahrenheit>) * 5.0<celsius/fahrenheit> / 9.0

// This would NEVER compile - catching the error at build time
type FreezerMonitor = {
 CurrentTemp: float<celsius>
 AlarmThreshold: float<celsius>
}

// Attempting to mix units causes compile-time error
let checkTemperature (monitor: FreezerMonitor) (reading: float<fahrenheit>) =
 // COMPILE ERROR: Expected float<celsius> but got float<fahrenheit>
 if reading > monitor.AlarmThreshold then
 Alert.Critical "Temperature exceeded threshold"

These unit checks disappear entirely at runtime. The compiled code is identical to using plain floats, and the type system ensures you can never mix units incorrectly.

Beyond Numbers: UMX for Domain Safety

Our Fidelity Framework extends this concept beyond numeric types using the UMX (Units of Measure eXtensions) approach. This brings the same compile-time safety to any type in your domain:

// Define domain-specific units for a medical system
type [<Measure>] patient
type [<Measure>] practitioner
type [<Measure>] prescription

// Type-safe identifiers prevent catastrophic mix-ups
type PatientId = Guid<patient>
type PractitionerId = Guid<practitioner>
type PrescriptionId = Guid<prescription>

// This function signature makes invalid states unrepresentable
let prescribeMedication
 (doctor: PractitionerId)
 (patient: PatientId)
 (medication: Medication) : PrescriptionId =
 // Cannot accidentally swap patient and doctor IDs
 Prescription.create doctor patient medication

// Attempting to mix identifiers causes compile error
let wrongOrder = prescribeMedication patientId doctorId medication
// COMPILE ERROR: Expected Guid<practitioner> but got Guid<patient>

Real-World Applications in Systems Programming

Our Fidelity Framework uses units of measure throughout its stack for systems-level safety:

Memory Management with Units

[<Measure>] type byte
[<Measure>] type kb = byte * 1024
[<Measure>] type mb = kb * 1024
[<Measure>] type gb = mb * 1024

// Network protocols with bandwidth units
[<Measure>] type second
[<Measure>] type mbps = mb / second

type NetworkBuffer = {
 Capacity: int<mb>
 CurrentUsage: int<byte>
 Bandwidth: float<mbps>
}

// Compile-time prevention of buffer overflows
let allocateBuffer (size: int<mb>) (data: byte[]) =
 let dataSize = data.Length * 1<byte>
 if dataSize > size * 1<mb/byte> then
 // Unit conversion is explicit and checked
 Error "Data exceeds buffer capacity"
 else
 Ok (Buffer.create size)

Hardware Interface Safety

// GPIO pin safety for embedded systems
type [<Measure>] input
type [<Measure>] output
type [<Measure>] pwm

type Pin<[<Measure>] 'mode> = int<'mode>

// Functions that only accept correctly configured pins
let readDigital (pin: Pin<input>) : bool =
 GPIO.read (int pin)

let writeDigital (pin: Pin<output>) (value: bool) : unit =
 GPIO.write (int pin) value

let setPWMDutyCycle (pin: Pin<pwm>) (duty: float<percent>) : unit =
 GPIO.setPWM (int pin) (float duty)

// Compile error if you try to write to an input pin
let inputPin = 7<input>
writeDigital inputPin true // COMPILE ERROR!

Protocol-Level Type Safety

// Network protocol units for Modbus (like the freezer example)
type [<Measure>] holding_register
type [<Measure>] input_register
type [<Measure>] coil
type [<Measure>] discrete_input

// Modbus addresses with their register types
type ModbusAddress<[<Measure>] 'register> = uint16<'register>

// The freezer monitoring system, done right
type FreezerSensor = {
 TempRegister: ModbusAddress<input_register>
 DoorStatus: ModbusAddress<discrete_input>
 SetPoint: ModbusAddress<holding_register>
}

// Read operations that ensure correct register access
let readTemperature (sensor: FreezerSensor) : float<celsius> =
 let rawValue = Modbus.readInputRegister sensor.TempRegister
 float rawValue * 0.1<celsius> // Sensor reports in 0.1°C increments

// Cannot accidentally read a holding register as input
let wrong = Modbus.readInputRegister sensor.SetPoint
// ERROR: Expected uint16<input_register> but got uint16<holding_register>

Memory Safety Without the Performance Tax

One of the harder type safety problems in .NET is the “byref restriction,” the inability to store or return references to memory, which forces defensive copying that can degrade performance in data-intensive applications. Our Fidelity Framework addresses this limitation by combining UMX with our BAREWire capability-based memory management:

// Traditional .NET - forced to copy due to byref restrictions
let updateLargeData (data: float[]) =
 // Can't return a reference, must copy entire array
 let copy = Array.copy data
 for i = 0 to copy.Length - 1 do
 copy.[i] <- copy.[i] * 2.0
 copy // Expensive copy operation

// Fidelity approach - type-safe capabilities with zero copying
[<Measure>] type readonly
[<Measure>] type readwrite
[<Measure>] type exclusive

type MemoryCapability<'T, [<Measure>] 'access> =
 | Capability of BAREWire.Buffer<'T> * AccessToken<'access>

// Functions can only accept appropriately typed capabilities
let readData (cap: MemoryCapability<float, readonly>) =
 match cap with
 | Capability(buffer, _) -> buffer.Read(0)

let modifyData (cap: MemoryCapability<float, readwrite>) =
 match cap with
 | Capability(buffer, _) ->
 for i = 0 to buffer.Length - 1 do
 buffer.[i] <- buffer.[i] * 2.0

// Compile error prevents accidental mutation
let badFunction (cap: MemoryCapability<float, readonly>) =
 modifyData cap // ERROR: Expected readwrite but got readonly

This design prevents both type confusion and performance degradation. Where .NET’s byref restrictions force copying at every boundary, our BAREWire capabilities are designed to pass safely through async boundaries, to be stored in data structures, and to be shared across processes, all while maintaining type safety through UMX:

// Fidelity design target; impossible under standard .NET byref rules
let asyncProcessWithoutCopying (cap: MemoryCapability<float[], readwrite>) = async {
 // Capability can be captured in async - no byref restriction!
 do! Async.Sleep(100)

 // Direct memory access after await - no defensive copying
 modifyData cap

 // Can even return capabilities from async
 return cap
}

// Share memory across process boundaries with type safety
let shareAcrossProcesses (localCap: MemoryCapability<SensorData, exclusive>) =
 // Convert exclusive access to shared read-only for other processes
 let sharedCap = BAREWire.downgrade<exclusive, readonly> localCap

 // Type system ensures other processes can only read
 IPC.share "sensor-data" sharedCap

The integration of UMX with memory capabilities addresses multiple critical requirements:

Type safety: Prevents accidental writes to read-only memory at compile time
Memory safety: Makes use-after-free errors impossible through capability tracking
Performance: Eliminates defensive copying throughout the system
Composability: Capabilities work with async, can be stored, and passed between processes

The Zero-Cost Abstraction Promise

Our Clef units of measure provide their safety at zero runtime cost. The compiler erases all unit information after types have been marshaled to the lowest level of compilation:

// Clef source with units - type safe at compile time
let calculatePower (voltage: float<volt>) (current: float<ampere>) : float<watt> =
 voltage * current

The compilation pipeline shows how units of measure disappear at each stage:

// MLIR from Composer - units erased, but semantics preserved
func.func @calculatePower(%voltage: f64, %current: f64) -> f64 {
 %result = arith.mulf %voltage, %current : f64
 func.return %result : f64
}

; LLVM IR after mlir-opt - further optimized
define double @calculatePower(double %0, double %1) #0 {
 %2 = fmul fast double %0, %1
 ret double %2
}

; x86-64 assembly - pure machine operations
calculatePower:
 mulsd xmm0, xmm1 ; multiply two doubles in SSE registers
 ret ; return result in xmm0

At each stage, more abstraction disappears. Units of measure exist only in the Clef source. By MLIR, they’re gone but the operation remains typed. By assembly, even the operation names vanish. The type safety that prevents unit confusion exists only at compile time, protecting us during development and vanishing completely in the deployed code.

Learning from Near-Disasters

The freezer monitoring incident teaches us several critical lessons:

Types are your specification: The type system should encode your domain knowledge
Make illegal states unrepresentable: If mixing units is wrong, make it impossible
Zero-cost abstractions exist: Safety doesn’t require runtime overhead
Catch errors at compile time: This constructive constraint during design saves time and money, and eliminates entire categories of operational risk

Integration with the Fidelity Framework

Our Fidelity Framework builds on Clef’s units of measure to extend that safety across the stack:

// BAREWire protocol with type-safe channels
type [<Measure>] sensor_data
type [<Measure>] control_command
type [<Measure>] telemetry

type Channel<[<Measure>] 'msgType> = {
 Endpoint: BAREWire.Endpoint
 Protocol: Protocol<'msgType>
}

// Channels can only connect compatible types
let connectChannels
 (sender: Channel<'a>)
 (receiver: Channel<'a>) : Connection<'a> =
 BAREWire.connect sender receiver

// Type error if you try to connect incompatible channels
let sensorChan = createChannel<sensor_data> "tcp://sensor:5555"
let controlChan = createChannel<control_command> "tcp://control:5556"
let invalid = connectChannels sensorChan controlChan
// COMPILE ERROR: Cannot connect Channel<sensor_data> to Channel<control_command>

Beyond Safety: Types as Documentation

Well-designed units of measure serve as living documentation:

// The types tell the whole story
type TurbineMonitoring = {
 RotationSpeed: float<rpm>
 Temperature: float<celsius>
 Pressure: float<pascal>
 Efficiency: float<percent>
 PowerOutput: float<megawatt>
 Runtime: TimeSpan<hour>
}

// Function signatures become self-documenting
let calculateMaintenance
 (runtime: TimeSpan<hour>)
 (avgTemp: float<celsius>)
 (peakPressure: float<pascal>) : TimeSpan<hour> =
 // unit correctness checked at compile time
 let tempFactor = (avgTemp - 85.0<celsius>) / 10.0<celsius>
 let pressureFactor = peakPressure / 100_000.0<pascal>

 let interval = 2000.0<hour> / (1.0 + abs tempFactor + pressureFactor)
 interval - runtime

The Path Forward

As we build increasingly critical systems, from medical devices to autonomous vehicles to financial infrastructure, we can’t afford to treat types as optional. The gene analysis startup nearly lost irreplaceable samples because their monitoring system allowed temperature units to be confused. The cost of catching that class of error at compile time is one annotation; the cost of missing it can be a year of irreplaceable work.

Our Fidelity Framework shows that type safety doesn’t require sacrificing performance. By using Clef’s units of measure and extending them through UMX, we can build systems that are at once:

Safer: Entire classes of errors become impossible
Faster: Zero runtime overhead from type checking
Clearer: Types in source code document intent and constraints
Maintainable: The compiler flags unit mismatches introduced during refactoring

Where We Are Taking This

That company’s freezer monitoring system nearly failed because the monitoring software allowed a dangerous state to exist, with no compiler in the way. The DNA samples survived by luck. With Clef’s units of measure, that error would have been caught at compile time, long before any samples were ever at risk.

In our Fidelity Framework, we are applying this lesson at every level, from low-level memory management to high-level protocol design. When you are building critical systems, “it works for my happy path use case” is not enough. Future systems need the compile-time guarantees a strong type system can provide, and we have found no other representative implementation in the standing literature we have reviewed that provides those guarantees at no additional runtime cost.

The next time you reach for a raw float for temperature, or a plain string for an identifier, the unit and the domain are the part of the design worth encoding in the type. That is the line of defense we are continuing to build out across the framework, from the dimensional type system upward, as the work continues.

Discriminated Unions In Post-Transformer AI

Tue, 17 Jun 2025 00:00:00 +0000

The AI industry stands at an inflection point. As detailed in our “Beyond Transformers” analysis, the convergence of matmul-free architectures and sub-quadratic models will shift how we build and deploy AI systems. While the research community has demonstrated these approaches can match or exceed transformer performance with dramatically lower computational requirements, our investigation at SpeakEZ has uncovered an intriguing gap:

Current tensor-only representations may not optimally capture the heterogeneous computational patterns these models require.

This insight emerged from our practical work on CNN to Topological Object Classification (TopOC) transfer learning, where we recently discovered that heterogeneous representations were essential for maintaining dimensional integrity across different mathematical domains. This article explores our early investigation into how BAREWire’s discriminated union-aware memory layout could address this gap more broadly, though we emphasize this remains a theoretical direction requiring substantial research to verify.

The Post-Transformer Memory Challenge

Our consideration began with examining the building blocks of emerging post-transformer architectures:

Matmul-free models use ternary weights (-1, 0, +1) and replace matrix multiplication with simple addition and subtraction. The UC Santa Cruz research showed these can process billion-parameter models at just 13 watts on FPGAs.

Sub-quadratic architectures like Mamba employ state space models that maintain different computational states across sequence positions. Linear attention variants use kernel approximations that require heterogeneous data structures.

Hybrid architectures combine neural and symbolic components, mixing continuous computations with discrete reasoning steps.

Traditional tensor frameworks force all these patterns into homogeneous arrays, creating potential inefficiency at every level:

# PyTorch's tensor-only approach to representing a Mamba layer
class MambaLayer:
 def __init__(self, d_model, d_state):
 self.A = torch.zeros(d_state) # Diagonal matrix, wastes memory
 self.B = torch.zeros(d_state, d_model)
 self.C = torch.zeros(d_model, d_state)
 self.D = torch.zeros(d_model) # Single parameter per channel
 self.delta = torch.zeros(d_model) # Time step parameter

 # State has to be maintained as tensor
 self.state = torch.zeros(batch_size, d_state)

This homogeneous representation misses an opportunity:

Post-transformer models are inherently heterogeneous, combining different types of computations that might benefit from different memory layouts.

Genesis: CNN to TopOC Transfer Learning

Our exploration of discriminated unions for AI architectures emerged from practical challenges we encountered in our work on CNN to Topological Object Classification (TopOC) transfer learning. In that research, we developed dimensionally-constrained models using the Clef language’s Units of Measure system to maintain representational integrity when bridging fundamentally different mathematical domains, convolutional feature spaces and topological invariants.

This work revealed a critical insight: the difficulty of transfer learning between different representational domains often stems from forcing heterogeneous information through homogeneous tensor pipelines. Our TopOC architecture already employs discriminated union-like patterns out of necessity:

// From our TopOC work - implicit discriminated unions
type SensorData =
 | AccelerometerReading of Tensor<mps2 * Channel * Time>
 | GyroscopeReading of Tensor<radps * Channel * Time>
 | MagnetometerReading of Tensor<T * Channel * Time>

type RepresentationSpace =
 | ConvolutionalFeatures of FeatureTensor
 | TopologicalInvariants of TopologicalTensor
 | PhysicalState of NavigationState

The success of this approach in maintaining both safety and efficiency while translating between radically different mathematical structures suggested a broader principle might be at work. If heterogeneous representations proved essential for CNN-to-TopOC transfer, might they also address the inefficiencies we observe in post-transformer architectures?

Discriminated Unions as Natural Representations

Building on our TopOC insights, we consider that Clef discriminated unions could provide a more natural representation for these heterogeneous computational patterns:

// Potential representation of matmul-free components
type MatmulFreeWeight =
 | Ternary of value: sbyte * scale: float32 // -1, 0, +1 with learned scale
 | Binary of bit: bool * scale: float32 // 0, 1 with scale
 | Quantized of bits: int * value: int * scale: float32

// Possible state space model components with natural structure
type SSMComponent =
 | DiagonalA of values: float32[] // Diagonal transition matrix
 | InputProjection of B: float32[,]
 | OutputProjection of C: float32[,]
 | DeltaParameter of dt: float32 // Single time-step parameter

// Potential sub-quadratic attention variants
type EfficientAttention =
 | LinearAttention of kernel: KernelFunction * features: int
 | FlashAttention of blockSize: int * numBlocks: int
 | SlidingWindow of windowSize: int * stride: int
 | Performer of randomFeatures: float32[,] * orthogonal: bool

We see these representations may serve to match the mathematical structure of post-transformer models more directly.

BAREWire’s Key: Zero-Copy Operations

Our work in developing BAREWire for general use suggests it might serve well to transform these type representations into efficient memory layouts using the BARE protocol’s ability to natively handle discriminated unions. This could be particularly interesting for the mixed computational patterns in post-transformer architectures:

Theoretical Matmul-Free Memory Layout

For matmul-free models, our initial explorations suggest BAREWire might enable direct memory representation of ternary weights with minimal overhead:

// Conceptual BAREWire schema for matmul-free layer
let matmulFreeSchema =
 schema "MatmulFreeLayer"
 |> withType "TernaryWeight" (struct' [
 field "packed_values" (array uint64) // 32 weights per uint64
 field "scale" float32
 ])
 |> withType "Layer" (union [
 variant "Dense" (struct' [
 field "weights" (userType "TernaryWeight")
 field "bias" (array float32)
 ])
 variant "Attention" (struct' [
 field "q_weights" (userType "TernaryWeight")
 field "k_weights" (userType "TernaryWeight")
 field "v_weights" (userType "TernaryWeight")
 ])
 ])

This schema could enable packing 32 ternary values into a single 64-bit word (2 bits per value), achieving significant compression over float32 representation while maintaining zero-copy access. It should be noted that whether this theoretical advantage translates to real-world performance gains remains an open research question, but the implications are intriguing.

Exploring State Space Model Optimization

For sub-quadratic models like Mamba, discriminated unions could also enable more efficient representation of heterogeneous components:

type MambaLayer =
 | StateUpdate of A: DiagonalMatrix * B: Tensor * state: MutableState
 | OutputProjection of C: Tensor * D: float32
 | GateComputation of input_proj: Linear * gate_proj: Linear

// Conceptual zero-copy state updates
let processSequence (layer: Region<MambaLayer, 'r>) (input: Tensor) =
 match BAREWire.readVariant layer with
 | StateUpdate(A, B, state) ->
 // Potential in-place state update without allocation
 for t in 0 .. seqLength - 1 do
 // s_t = A * s_{t-1} + B * x_t
 let x_t = input.[t]

 // Diagonal multiplication could be O(n) not O(n^2)
 DiagonalOps.multiplyInPlace state A
 TensorOps.addScaledInPlace state B x_t

 // Output y_t = C * s_t
 yield computeOutput state

 | _ -> failwith "Invalid layer type"

State space models have natural sparsity patterns and sequential dependencies, and we are exploring whether discriminated unions might represent these more directly than tensor-only frameworks. This speculation arises from examining the mathematical structure of SSMs like Mamba. These models fundamentally differ from transformers in several ways:

Structural Sparsity: The transition matrix A in SSMs is typically diagonal or block-diagonal, meaning only O(n) non-zero elements in what tensor frameworks represent as an n*n matrix. Current implementations waste memory storing zeros or use complex indexing schemes to avoid it.

Selective Mechanisms: Mamba’s key innovation is input-dependent selection - certain computations are gated based on the input content. This creates a branching computational structure where different tokens might follow different computational paths, naturally suited to variant type representation.

Heterogeneous Parameters: SSMs combine fundamentally different types of parameters:

Diagonal matrices (sparse, fixed structure)
Dense projection matrices (full computation needed)
Scalar time-step parameters (single values per channel)
Binary gates (on/off selection mechanisms)

Current tensor frameworks force all these into the same representation, likely missing significant optimization opportunities.

Sequential State Evolution: Unlike transformers that can process all positions in parallel, SSMs must process sequences step-by-step, with each state depending on the previous. This creates a fundamentally different computational pattern - more like a state machine than a parallel matrix operation. Discriminated unions naturally represent state machines with different transition types.

Our hypothesis is that representing these diverse computational patterns as explicit variants could enable compilers and hardware to optimize each pattern separately, rather than forcing everything through generic tensor operations. For instance, diagonal multiplication could be a distinct variant from dense multiplication, allowing specialized hardware paths for each.

Heterogeneous Memory

A particularly intriguing direction in our research involves the potential for discriminated unions to better preserve contextual information in continual learning scenarios. Current approaches force all context into homogeneous tensor representations, in theory losing structural information in the process.

The Signal Preservation Hypothesis

We hypothesize that heterogeneous CPU-GPU memory splits with type-aware layouts could serve to preserve contextual signals that current approaches lose:

// Conceptual context-aware memory representation
type ContextMemory =
 | WorkingMemory of tokens: GPU.Tensor * attention: GPU.Cache
 | EpisodicMemory of events: CPU.SparseGraph * timestamps: Timeline
 | SemanticMemory of facts: CPU.KnowledgeGraph * confidence: float32[]
 | ProceduralMemory of skills: CPU.ProgramSynthesis * usage: Stats

// Potential zero-copy transitions at semantic boundaries
let updateContext (memory: Region<ContextMemory, 'rw>) (newInfo: Information) =
 match newInfo with
 | ImmediateRelevance data ->
 // Hot path: GPU processing
 GPU.updateWorkingMemory memory data

 | LongTermPattern pattern ->
 // Cold path: CPU storage with rich structure
 CPU.integrateEpisodicMemory memory pattern

 | FactualKnowledge fact ->
 // Structured storage: Graph operations
 CPU.updateSemanticGraph memory fact

This approach could enable models to maintain richer representations across different memory hierarchies, though validating this hypothesis would require extensive empirical research.

Advancements in Auto-differentiation

This question represents perhaps the most significant technical challenge in our research direction. Current autodiff systems like PyTorch’s autograd and JAX’s grad are built on the assumption of homogeneous tensor operations with uniform gradient flow. However, our work on Furnace, a component of the Fidelity framework, provides a unique foundation for exploring heterogeneous auto-differentiation.

A Foundation for Heterogeneous Gradients

Furnace brings several key capabilities that position it uniquely for post-transformer architectures:

Nested and mixed-mode differentiation: Unlike conventional frameworks, Furnace supports arbitrarily nested differentiation operations, essential for meta-learning and higher-order optimization in heterogeneous architectures
Clef programming foundation: Built in Clef, Furnace naturally supports the algebraic structures needed for variant type differentiation
PyTorch-familiar idioms with guarantees: While maintaining familiar APIs, Furnace’s core enables reasoning about gradient flow through variant types

Our speculation on extending Furnace for discriminated unions draws from several theoretical directions:

Tagged Gradient Representations

Just as forward computation uses discriminated unions, Furnace could be extended to support tagged gradient variants:

type GradientFlow =
 | DenseGradient of grad: Tensor<float32>
 | SparseGradient of indices: int[] * values: float32[]
 | TernaryGradient of accumulated: float32 * threshold: float32
 | StructuredGradient of pattern: GradientPattern * data: CompactRepresentation

Variant-Aware Chain Rule

The chain rule of calculus remains valid regardless of representation. Furnace’s existing differentiation engine could be extended so each variant type defines its own local gradient computation and composition rules:

Ternary weights: Gradients accumulate until crossing quantization thresholds
Sparse operations: Gradients only flow through non-zero paths
Selective mechanisms: Gradients gate based on forward-pass decisions

Compile-Time Gradient Synthesis

Clef’s type system, combined with Furnace’s architecture, could enable compile-time generation of efficient gradient paths. Since the structure of discriminated unions is known statically, we could potentially generate specialized backprop code for each variant combination, avoiding runtime dispatch overhead while keeping the gradient computation structurally consistent with Furnace’s differentiation rules.

Heterogeneous Adjoints

Drawing from automatic differentiation theory and Furnace’s support for nested differentiation, each variant could maintain its own adjoint representation:

Dense layers: Traditional tensor adjoints (Furnace’s current strength)
Ternary layers: Accumulated gradient buffers with quantization-aware updates
Sparse layers: Coordinate-format gradient storage
Gated layers: Boolean path traces for gradient routing

Lazy Gradient Materialization

Not all gradients need to be materialized as full tensors. Furnace’s foundation could enable lazy evaluation where gradients remain in their most efficient representation until needed for weight updates, a natural extension of Clef’s lazy evaluation capabilities. This concept shares philosophical similarities with DeepSeek’s multi-head latent attention (MLA), which maintains key-value pairs in compressed latent representations until needed for attention computation. Just as MLA achieves memory efficiency by deferring full materialization of attention matrices, our approach could defer full gradient tensor materialization until the actual weight update step.

An intriguing consideration in this approach involves the precision requirements of heterogeneous architectures. Post-transformer models with ternary weights and sparse operations create more complex, discrete loss landscapes that may require higher numerical precision to navigate effectively. This presents both a challenge and an opportunity:

The Precision-Performance Balance: While maintaining higher precision might seem to inhibit performance advantages, our research suggests that Clef’s direct access to extended precision floating-point operations, unencumbered by Python’s C library limitations, could provide a crucial advantage. Where PyTorch and TensorFlow are constrained to 32-bit or 64-bit operations by their underlying BLAS libraries, Furnace could selectively apply 80-bit extended precision (on x86) or 128-bit quad precision where the loss landscape demands it, while maintaining efficient representations elsewhere.

// Conceptual precision-aware gradient accumulation
type PrecisionGradient =
 | StandardPrecision of grad: Tensor<float32>
 | ExtendedPrecision of grad: ExtendedFloat80 // For critical paths
 | LazyAccumulated of thunks: (unit -> float32) list // Deferred computation

// Selective precision based on gradient magnitude and variance
let accumulateWithAdaptivePrecision (gradients: GradientFlow list) =
 match analyzeGradientStability gradients with
 | Stable -> LazyAccumulated (gradients |> List.map toThunk)
 | CriticalRegion -> ExtendedPrecision (accumulateExtended gradients)
 | Standard -> StandardPrecision (accumulate gradients)

The theoretical foundation exists in the community’s work on algebraic automatic differentiation, where sum types (coproducts) have well-defined differentiation rules. The challenge is efficiently implementing these in the context of large-scale neural networks, particularly when balancing precision requirements against computational efficiency. Our hypothesis is that the same type information that enables efficient forward computation could also guide both optimized backward computation and adaptive precision selection, using high precision only where the discrete nature of post-transformer architectures demands it, while maintaining efficiency elsewhere. This selective approach could provide more accurate gradient estimates in the challenging optimization landscapes of quantized networks without uniformly sacrificing performance, but this requires significant research to validate and likely some additional engineering effort to optimize.

Research Directions and Open Questions

Our investigation has identified several key research questions that beg further assessment:

Performance validation: Can discriminated union representations match or exceed the performance of current tensor-only implementations?
Automatic differentiation: How would backpropagation work efficiently across heterogeneous type representations?
Hardware integration: Can existing accelerators be adapted to benefit from variant type dispatch, or would new architectures be required?
Framework compatibility: How could this approach integrate with existing ML ecosystems while providing migration paths?
Memory hierarchy optimization: Can type-aware layouts actually preserve more contextual information across CPU-GPU boundaries?

Beyond Current Architectures

As we look even further forward toward neuromorphic and quantum-classical hybrid models, the potential value of discriminated unions may become even more apparent:

// Potential future neuromorphic representation
type SpikingNeuron =
 | LIF of threshold: float32 * decay: float32 * refractory: TimeSpan
 | Izhikevich of a: float32 * b: float32 * c: float32 * d: float32
 | SpikeResponse of kernel: ResponseKernel * adaptation: AdaptationCurve

// Possible quantum-classical hybrid
type QuantumLayer =
 | ParameterizedGate of angles: float32[] * qubits: int[]
 | Measurement of basis: PauliBasis * classical_post: ClassicalNetwork
 | VariationalCircuit of ansatz: Ansatz * parameters: float32[]

These future architectures may inherently require heterogeneous representations that current tensor-only frameworks cannot express.

A Research Path Worth Exploring

Our investigation at SpeakEZ has identified discriminated union-aware memory layouts as a potentially valuable research direction for post-transformer AI architectures. This exploration emerged naturally from our practical experience with CNN to TopOC transfer learning and investigation of sub-quadratic and matmul-free architectures, where heterogeneous representations proved essential for maintaining dimensional integrity across mathematical domains. While this broader application remains speculative, the convergence of several trends makes this exploration timely:

The shift to heterogeneous post-transformer architectures
The need for more memory-efficient representations
The opportunity for better context preservation in continual learning
The potential for custom hardware optimization
The availability of Furnace’s advanced auto-differentiation capabilities

We emphasize that this article presents early-stage exploration, not validated results. The ideas presented here require rigorous empirical validation, performance benchmarking, and peer review before any significant claims can be made. Our goal is to contribute to the broader research conversation about how the industry might better represent the increasingly heterogeneous computational patterns emerging in modern AI, building on our practical insights.

As the industry moves beyond transformers, we believe exploring alternative memory representations is a valuable research direction. Whether this particular approach proves fruitful remains to be seen, but the investigation itself helps illuminate the assumptions and constraints built into our current frameworks. The combination of our BAREWire memory layout and our Furnace differentiation engine gives us a platform for this exploration, and that is where our interest lies as the work continues.

The technology concepts described here relate to U.S. Patent Application No. 63/786,247 “System and Method for Zero-Copy Inter-Process Communication Using BARE Protocol”, though we note that the specific application to AI architectures will likely remain a research topic of continuing interest.

Clef Programming Language – Clef Programming Language

Counting the Cost of Coordination

Where the cost actually lives

The fix, and who pays for it

Making isolation a property instead of a discipline

Two ends of the same problem

Fearless Concurrency Gets Real

A challenging landscape

Why “the fight” happens

No unsafe in Clef

No skirting deadlock

An analyzer and the structure it rides

The work ahead

Small choices loom large

Where Native Goes, Mobile Follows

The Conventional Path and Its Costs

The Day-One Design Decision

MLIR as the intermediate representation

Nanopass lowering

Module signatures with platform-specific implementations

Memory model designed for arena and region allocation

Coeffect tracking for resource invariants

The Platform Tuple as Compilation Substrate

The UI Strategy: Two Reference Implementations

The HelloWayland pattern: native widgets, direct surface

The WRENHello pattern: system WebView with native backend

When to use which pattern

Per-Platform Walkthrough

Linux desktop

postmarketOS and Phosh

Samsung TV (Tizen)

Android

iOS

Embedded (STM32, RA6M5)

The Module Signatures That Make It Work

Honest Tradeoffs

Per-platform shim code is real and not zero

iOS specifically requires macOS for the build

No hot-reload story comparable to Flutter’s

App Store review may be more cautious on first submission

Strategic Implications

Device fleet longevity

Security surface

Cross-cutting verification across all targets

Where mobile lands on this roadmap

Don't Assume All Proofs Are Bulletproof

Game-Based Security Definitions

The Reductionist Structure

The Bidirectional Consequence

VSH and the Inverted Hierarchy

The Post-Quantum Reductionist Response

Three Categories of Foundation

Implications for Primitive Selection

Terms and Conditions Apply

A Triangle Without Mystery

Corner One: Dimensional Types

Corner Two: Tarau’s Groupoid

Corner Three: Cellular Sheaves

The Triangle

The Engineering Flywheel

Leading Indications

A Runtime Revolution, sort of...

Why This Matters for Cloudflare Workers

BAREWire at the Wire Boundary

How the Runtime Carries the Freight

Streaming Tokens from Containers

What Would Not Change

Where We See the Trust Boundary Moving

Looking Forward

JavaScript You Can Be Proud Of

A Principled Compromise

Building Proofs for the Real World

Beyond Dimensional Consistency

Range Propagation as Proof Machinery

Domain Case: Aerospace Structural Integrity

Flight Envelope Analysis

Domain Case: Financial Risk Constraints

Domain Case: Clinical Dosage Safety

The Common Pattern

The Mechanism Is Not New; the Application Is

No `unsafe` in Clef