Context-Aware Compilation

Modern processors offer substantial parallel execution capacity. A typical server CPU offers dozens of cores, each capable of executing multiple instructions per cycle through SIMD operations. GPUs push this further with thousands of cores organized in warps and thread blocks. Emerging accelerators like NextSilicon’s Maverick or Graphcore’s IPU reorganize computation around different execution models. Yet most code harnesses only a fraction of this capacity, because the right parallel execution strategy follows from what a program does together with what it needs from its environment, and a traditional compiler sees only the first half. Coeffect analysis supplies the second half.

The Parallelism Predicament

Consider a seemingly simple operation that appears in everything from scientific computing to real-time analytics:

let processDataset (data: float[]) =
    data
    |> Array.map (fun x -> complexTransform x)
    |> Array.filter (fun x -> x > threshold)
    |> Array.reduce (+)

How should this compile? The answer depends on context that traditional compilers can’t see:

  • Is complexTransform a pure mathematical function or does it access external resources?
  • What is the size of the data array: does it fit in L3 cache or span gigabytes?
  • Is the target a CPU with strong single-thread performance or a GPU with massive parallelism?
  • Does the computation pattern allow for SIMD vectorization or require scalar operations?

Get these decisions wrong and the “parallel” code runs slower than sequential execution; get them right and it approaches peak hardware throughput. The Fidelity framework uses coeffect analysis to make these decisions at compile time.

Two Paths to Parallelism

Fidelity’s approach rests on a choice between two compilation strategies: interaction nets for pure parallelism and delimited continuations for controlled concurrency. The coeffects a program carries determine which strategy applies to a given region of code.

Interaction Nets: When Everything Can Happen at Once

Interaction nets represent computation as a graph where nodes interact through local rewrite rules. When code has no sequential dependencies or external effects, this model admits maximum parallelism:

// Pure computation - perfect for interaction nets
let matrixMultiply (a: Matrix) (b: Matrix) =
    Array2D.init a.Rows b.Cols (fun i j ->
        seq { for k in 0 .. a.Cols - 1 -> a.[i,k] * b.[k,j] }
        |> Seq.sum)

// Compiles to interaction net:
// - Every element computation is independent
// - Maps directly to GPU thread blocks
// - No synchronization needed until final result
 
  graph TD
    subgraph "Interaction Net Compilation"
        A[Matrix A] --> MUL1[Multiply i,j,0]
        B[Matrix B] --> MUL1
        A --> MUL2[Multiply i,j,1]
        B --> MUL2
        A --> MUL3[Multiply i,j,k]
        B --> MUL3
        MUL1 --> SUM[Sum Reduction]
        MUL2 --> SUM
        MUL3 --> SUM
        SUM --> R[Result i,j]
    end

Delimited Continuations: When Order Matters

Delimited continuations capture “the rest of the computation” at specific points, enabling precise control over execution order and resource management:

// Computation with external dependencies - needs delimited continuations
let enrichDataPoints (points: DataPoint[]) = async {
    let mutable enriched = []

    for point in points do
        let! weather = WeatherService.getConditions point.Location
        let! traffic = TrafficService.getDensity point.Location

        enriched <- { point with
                        Weather = weather
                        TrafficDensity = traffic } :: enriched

    return List.rev enriched
}

// Compiles to delimited continuations:
// - Must maintain order for external service calls
// - Can optimize service call batching
// - Handles failure/retry gracefully
 

The choice between interaction nets and delimited continuations fundamentally determines what optimizations are possible and which hardware can efficiently execute the code.

Coeffects

Coeffects track what code needs from its environment, and that requirement is the information needed to choose between parallel execution strategies:

// Coeffect inference discovers the nature of computation
type ContextRequirement =
    | Pure                           // No external dependencies → Interaction nets
    | MemoryAccess of AccessPattern  // Data access pattern → Guides parallelization
    | ResourceAccess of Resource Set // External resources → Delimited continuations
    | Temporal of HistoryDepth       // Needs past values → Streaming architecture
 

The Fidelity compiler will be designed to automatically infer these requirements:

// Example 1: Scientific computation
let runSimulation (grid: float[,]) =
    let nextGeneration =
        grid |> Array2D.mapi (fun i j cell ->
            let neighbors = getNeighbors grid i j
            updateCell cell neighbors)

    if converged grid nextGeneration then
        nextGeneration
    else
        runSimulation nextGeneration

// Coeffect analysis discovers:
// - Pure computation (no external resources)
// - Regular memory access pattern (stencil)
// - Recursive structure (iterative refinement)
// → Compile to interaction nets on GPU
 

Real Hardware, Real Decisions

Let’s see how coeffect analysis guides compilation for different processors:

Multi-Core CPU: Cache-Aware Parallelism

// Data processing with mixed patterns
let analyzeTimeSeries (data: TimePoint[]) =
    data
    |> Array.windowed 100
    |> Array.Parallel.map (fun window ->
        let stats = computeStatistics window
        let anomalies = detectAnomalies window stats
        (stats, anomalies))
    |> consolidateResults

// Coeffect analysis:
// - Sliding window → temporal coeffect
// - Pure computation within windows
// - Window size fits in L2 cache
//
// Compilation strategy:
// - Use CPU SIMD for statistics
// - Parallelize across cores with cache-aware chunking
// - No GPU transfer overhead worth it
 

GPU: Massive Data Parallelism

// Image processing pipeline
let processImages (images: Image[]) =
    images
    |> Array.map (fun img ->
        img
        |> applyConvolution kernel1
        |> applyConvolution kernel2
        |> normalizeHistogram)

// Coeffect analysis:
// - Pure computation
// - Regular 2D access pattern
// - High arithmetic intensity
//
// Compilation strategy:
// - Interaction nets mapping to GPU warps
// - Texture memory for convolution kernels
// - Fused kernel execution
 

Custom Accelerators: Adaptive Execution

Modern accelerators with novel architectures will be able to accept different workload patterns. Coeffects enable this novel adaptation to maximize their potential:

// ML inference with dynamic characteristics
let runInference (model: Model) (input: Tensor) =
    let layers = model.Layers

    let rec forward (input: Tensor) (layers: Layer list) =
        match layers with
        | [] -> input
        | Linear weights :: rest ->
            // Coeffect: Matrix multiplication pattern
            let output = input |> matmul weights |> activate
            forward output rest
        | Attention heads :: rest ->
            // Coeffect: All-to-all communication pattern
            let output = multiHeadAttention input heads
            forward output rest
        | Convolution kernel :: rest ->
            // Coeffect: Local spatial pattern
            let output = convolveSpatial input kernel
            forward output rest

    forward input layers

// Coeffect analysis per layer type:
// - Linear → Pure matrix ops → Configure as systolic array
// - Attention → All-to-all → Configure as crossbar
// - Convolution → Spatial → Configure as 2D mesh
//
// Maverick reconfigures between layers
 

Separation of Concerns

Under this design, developers write intent-focused code while the compiler determines the strategy for parallel execution from the inferred coeffects:

  graph LR
    subgraph "Developer Writes"
        CODE[Clean Clef Code<br/>Business Logic Only]
    end

    subgraph "Coeffect Analysis"
        CODE --> INFER[Infer Context<br/>Requirements]
        INFER --> PURE{Pure<br/>Computation?}
        INFER --> RESOURCE{Resource<br/>Access?}
        INFER --> TEMPORAL{Temporal<br/>Dependencies?}
    end

    subgraph "Compilation Strategy"
        PURE -->|Yes| INET[Interaction Nets<br/>Maximum Parallelism]
        RESOURCE -->|Yes| DCONT[Delimited Continuations<br/>Controlled Execution]
        TEMPORAL -->|Yes| STREAM[Streaming Architecture<br/>Pipeline Parallelism]

        INET --> GPU[GPU/TPU<br/>Thousands of Threads]
        DCONT --> CPU[CPU<br/>Smart Scheduling]
        STREAM --> CUSTOM[Custom Accelerator<br/>Adaptive Pipeline]
    end

Beyond AI: Parallelism Everywhere

While AI workloads grab headlines, coeffect-driven compilation benefits any parallel computation:

Financial Modeling

let monteCarloSimulation (initialState: MarketState) (scenarios: int) =
    [|1..scenarios|]
    |> Array.Parallel.map (fun i ->
        let rng = Random(i)
        simulateScenario initialState rng)
    |> Array.map calculateValue
    |> computeRiskMetrics

// Pure computation with independent scenarios
// → Interaction nets on GPU for massive parallelism
 

Real-Time Signal Processing

let processAudioStream (stream: AudioSample seq) =
    stream
    |> Seq.windowed 1024
    |> Seq.map (fun window ->
        let spectrum = FastFourierTransform window
        let features = extractFeatures spectrum
        classifySound features)

// Temporal coeffects with real-time constraints
// → CPU SIMD with predictable latency
 

Bioinformatics

let alignSequences (reference: DNA) (samples: DNA[]) =
    samples
    |> Array.Parallel.map (fun sample ->
        findBestAlignment reference sample)
    |> filterHighQualityAlignments

// Memory-intensive pure computation
// → NUMA-aware CPU parallelism
 

Observability Falls Out of the Same Analysis

Coeffect analysis carries a second use: automatic observability. Since the compiler already tracks context requirements, telemetry can be inserted at the boundaries that analysis already identifies:

// The source code remains clean
let processOrder order =
    validateOrder order
    |> Result.bind calculatePricing
    |> Result.bind checkInventory
    |> Result.bind createShipment

// Compiler knows:
// - validateOrder is pure → Time CPU cycles
// - calculatePricing accesses pricing service → Track service latency
// - checkInventory queries database → Monitor query performance
// - createShipment calls external API → Record API metrics
//
// Telemetry inserted at natural boundaries with zero overhead
 

This telemetry falls out of the same analysis that drives optimization. Fidelity’s Program Semantic Graph, and the static resolution of interaction nets and delimited continuations, keep all code paths known and traceable, with no opaque allocations or convenience downcasts to obscure data and control flow. The boundary-level telemetry rides on that determinism: because the compute graph tracks it, the observation points are already there to read.

The same determinism reduces the fixed costs of day-over-day development cycles, not only runtime cost. The transparency these designs carry also lowers the ongoing burden of production monitoring and maintenance.

Separate Analysis, Accurate Workloads

Fidelity’s architecture keeps coeffect analysis separate from your code structure:

// Your AST stays clean
type PSGNode = {
    Id: NodeId
    Kind: PSGNodeKind
    Symbol: FSharpSymbol option
    SourceRange: range
}

// Coeffects live in external analysis maps
type CoeffectAnalysis = {
    ComputationPatterns: Map<NodeId, ComputationPattern>
    MemoryAccessPatterns: Map<NodeId, AccessPattern>
    ResourceRequirements: Map<NodeId, Set<Resource>>
    TemporalDependencies: Map<NodeId, HistoryDepth>
}

// Compilation uses both for intelligent decisions
let selectBackend (node: PSGNode) (coeffects: CoeffectAnalysis) =
    match Map.tryFind node.Id coeffects.ComputationPatterns with
    | Some PureDataParallel -> InteractionNetBackend
    | Some ResourceDependent -> DelimitedContinuationBackend
    | Some StreamingComputation -> PipelineParallelBackend
    | _ -> DefaultSequentialBackend

Academic Rigor Yields Real World Performance

Coeffects deliver measurable performance wins:

  • 10-100x speedups by choosing the right parallel execution model
  • Near-linear scaling on multi-core systems through intelligent work distribution
  • Optimal hardware utilization by matching computation patterns to processor strengths
  • Zero abstraction penalty with optimal decisions are made at compile time

The choice between interaction nets and delimited continuations follows a clear rule: what your code needs to accomplish on a given server architecture settles it. By making this choice automatically based on coeffect analysis, Fidelity targets the right workload for a given chipset every time.

As we enter an era of increasingly heterogeneous hardware (CPUs, GPUs, TPUs, DPUs, and exotic accelerators), this context-aware compilation becomes essential. Your code expresses intent, this approach to intelligent analysis discovers requirements, and the compiler generates optimal execution strategies. That’s the power of coeffects: turning academic insight into real-world performance.

Advanced Implementation: Hybrid BitNet with Compressed KV Cache

To illustrate the transformative power of coeffect-guided compilation, consider a cutting-edge hybrid architecture that combines BitNet’s ternary operations with DeepSeek-style compressed key-value caching. This exemplifies how coeffects enable optimal workload distribution across heterogeneous hardware.

The Architectural Innovation

Traditional approaches run entire models on GPUs, missing a fundamental insight: different layers have vastly different computational characteristics. By combining coeffect analysis with BAREWire’s zero-copy technology, Fidelity can automatically split workloads optimally:

// Coeffect analysis discovers the natural CPU/GPU boundary
type HybridDeepSeekBitNet = {
    // CPU-optimized layers (ternary operations)
    BitNetAttention: {
        Operations: Add/Subtract only
        MemoryPattern: Sequential streaming
        Coeffect: Pure, LowArithmeticIntensity
        -> Target: CPU with AVX-512
    }

    // GPU-optimized components (parallel decompression)
    CompressedKVCache: {
        Operations: Parallel decompression
        MemoryPattern: Random access
        Coeffect: HighParallelism, MemoryIntensive
        -> Target: GPU with warp-level primitives
    }
}

Zero-Copy Memory Architecture

BAREWire’s unified memory abstraction eliminates the traditional CPU-GPU transfer bottleneck:

module HybridInference =
    // Shared memory pool - no copies needed!
    let kvCache = BAREWire.createUnified<byte> {
        Size = 16<GB>
        Layout = CompressedKVLayout
        AccessPattern = CPUWrite_GPURead
    }

    // CPU processes BitNet layers efficiently
    let processBitNetLayer (input: Tensor) (weights: TernaryWeights) =
        // Just adds and subtracts - perfect for CPU!
        let output = CPU.ternaryMatmul input weights

        // Write directly to shared KV cache
        // GPU can read without any memory transfer
        kvCache.WriteCompressedScores output.AttentionScores
        output

    // GPU handles parallel KV decompression
    let retrieveContext (query: Query) =
        GPU.kernel {
            // Read directly from CPU-written memory
            let! compressed = kvCache.ReadCompressedBlock query.Range
            // Decompress in parallel across GPU warps
            let! decompressed = parallelDecompress compressed
            return reshapeToContext decompressed
        }

Quaternary Encoding for Sparse Attention

The hybrid architecture benefits from sophisticated encoding that represents both ternary values and sparsity:

type QuaternaryValue =
    | Neg      // -1
    | Zero     // 0
    | Pos      // +1
    | NoValue  // Sparse/undefined

// Efficient packing enables CPU SIMD operations
type CompressedAttentionBlock = {
    Mask: uint64           // Bit mask for value presence
    PackedValues: byte[]   // 4 quaternary values per byte
    Metadata: BlockStats   // For adaptive processing
}

// CPU can efficiently process sparse patterns
let processSparseTernary (block: CompressedAttentionBlock) =
    if block.Mask = 0UL then
        Vector.zero  // Skip entirely
    else
        // SIMD process only present values
        CPU.simdTernaryOps block.PackedValues block.Mask

Performance Implications

This hybrid approach achieves remarkable efficiency by letting each processor do what it excels at:

ComponentTraditional (GPU-only)Hybrid (CPU+GPU)Improvement
BitNet LayersUnderutilizes GPU cores512 ops/cycle on CPU8x throughput
KV CompressionSequential on GPUParallel decompression4x faster
Memory TransfersConstant CPU-GPU copiesZero-copy via BAREWireEliminated
Power Efficiency300W GPU always active50W CPU + GPU when needed3-6x better

Dynamic Adaptation Through Coeffects

The coeffect system enables runtime adaptation based on workload characteristics:

// Coeffects monitor and adapt the CPU/GPU split dynamically
let adaptiveHybridInference (model: HybridModel) (stream: TokenStream) =
    coeffectAsync {
        // Initial split based on static analysis
        let! initial = processTokens stream

        // Runtime telemetry through coeffects
        observe "cpu_ternary_throughput" adjustCPULoad
        observe "gpu_decompression_latency" adjustGPULoad
        observe "memory_bandwidth_usage" optimizeLayout

        // Coeffects guide dynamic rebalancing
        match workloadPattern with
        | LongContext ->
            // Shift more KV processing to GPU
            CompileStrategy.IncreaseGPUDecompression
        | RapidTokenGeneration ->
            // Maximize CPU ternary throughput
            CompileStrategy.OptimizeCPUPipeline
        | Interactive ->
            // Balance for low latency
            CompileStrategy.MinimizeHandoffs
    }

In this hybrid BitNet and compressed-KV arrangement, the coeffects a region carries drive where it runs: CPUs take the ternary operations, GPUs take parallel decompression, and the analysis of what each component needs from its environment decides the placement. That is the same purity-and-requirement reading the rest of this document describes, applied across processors instead of within one.