Source-Level Dependencies

Source-Level Dependency Resolution

This article was originally published on the SpeakEZ Technologies blog as part of our early design work on the Fidelity Framework. It has been updated to reflect the Clef language naming and current project structure.

The Composer compiler represents a fundamental shift in how Clef code gets compiled to native executables. Unlike traditional F# compilation that relies on pre-compiled assemblies and the .NET runtime, Composer compiles Clef directly to native code through MLIR and connected “back ends”, right now primarily LLVM, creating truly dependency-free executables. This architectural choice creates an interesting challenge: how do we handle library dependencies when we can’t rely on traditional assembly resolution?

Beyond Assembly-Based Dependencies

Traditional F# development uses a well-established pattern where open statements resolve to compiled assemblies in NuGet packages. When you write open System.Collections.Generic, the F# compiler knows exactly where to find the pre-compiled code. This works for .NET applications but breaks down completely when your goal is zero-dependency native executables.

In the Fidelity framework ecosystem, CCS (Clef Compiler Services) provides the native type universe that replaces the BCL. When a developer writes open Fidelity.Platform in their Composer application, the compiler needs to discover, parse, and include the relevant source files from the library ecosystem — including any Farscape-generated binding libraries.

Consider this simple example that illustrates the complexity:

module Examples.HelloWorld

open Fidelity.Platform

let hello() =
    Console.Write "Enter your name: "
    let name = Console.ReadLine()
    Console.WriteLine $"Hello, {name}!"

[<EntryPoint>]
let main argv =
    hello()
    0

This innocent-looking program depends on a range of functions across multiple modules, each with their own internal dependencies and platform-specific implementations. CCS and the Composer compiler must discover all these dependencies, parse only what’s needed, and weave them together into a coherent compilation unit.

Farscape for C/C++ Library Binding

A key aspect of our dependency resolution strategy is how we handle C/C++ libraries. The Farscape CLI generates [<FidelityExtern>] attributed Clef binding declarations from C headers, producing source-based packages that flow through the same dependency resolution pipeline as any other Fidelity library.

Farscape generates up to three layers of output for each library binding:

Layer 1 (Binding Declarations): [<FidelityExtern>] attributed function declarations that carry library name and symbol metadata through the entire compilation pipeline. These are regenerated from C headers on each farscape project run.
Layer 2 (Idiomatic Wrappers): Safe Clef wrapper functions with error handling, resource management, and type conversions. Also regenerated automatically by Farscape’s WrapperCodeGenerator.
Overlay (Optional): NTU dimensional type annotations scaffolded once from Moya TOML annotations, then maintained as developer-owned code. This layer is generated at the developer’s discretion via --generate-overlay.

// Layer 1: FidelityExtern binding declaration (generated by Farscape)
module Fidelity.libc.IO.Platform

[<FidelityExtern("libc", "write")>]
let write (fd: int32) (buf: nativeptr<byte>) (count: unativeint) : int32 =
    Unchecked.defaultof<int32>

// Layer 2: Idiomatic wrapper (generated by Farscape)
module Fidelity.libc.IO

let writeBytes (fd: int32) (data: nativeptr<byte>) (length: int) : Result<int, IOError> =
    let result = Platform.write fd data (unativeint length)
    if result >= 0l then Ok (int result)
    else Error (IOError.fromErrno result)

The [<FidelityExtern>] attribute is a quotation carrier — CCS recognizes it during compilation, emits an opaque extern node in the PSG (Program Semantic Graph) with the library and symbol metadata, and the Unchecked.defaultof<T> body is never processed. This extern node flows through Baker saturation as a leaf and arrives at Alex for MLIR emission with its binding metadata intact.

Moya Project System

Farscape’s Moya project system organizes library decompositions through .moya.toml files. A single Moya project can reference multiple C headers — for example, unistd.h and fcntl.h for IO operations — with Farscape merging and deduplicating declarations across headers automatically:

[library]
name = "libc"
headers = ["/usr/include/unistd.h", "/usr/include/fcntl.h"]
include_paths = ["/usr/include"]

[output]
mode = "fidelity"
directory = "./output"

[[namespaces]]
name = "Fidelity.libc.IO"
library = "libc"
prefixes = ["read", "write", "open", "close", "lseek", "pipe", "dup"]

These generated binding packages become first-class citizens in the Fidelity ecosystem, discoverable and usable through the same open statement and .fidproj dependency mechanism as any other source-based library.

Dependency Resolution Architecture

After extensive analysis, we’ve developed a layered approach to resolving source-level dependencies that leverages CCS and the Fidelity compilation pipeline. The solution operates through several complementary mechanisms.

Package Discovery and Resolution

ClefPak, our package management system, resolves dependencies declared in .fidproj files. Packages are distributed as source through the clefpak.dev registry:

# Example .fidproj file
[package]
name = "my_embedded_app"
version = "0.1.0"
authors = ["Developer <[email protected]>"]

[dependencies]
"Fidelity.libc.IO" = { version = "0.1.0", binding = "dynamic" }
"Fidelity.libc.Memory" = { version = "0.1.0", binding = "dynamic" }

[build]
output_kind = "console"

[platform]
default = "x86_64-linux"

When ClefPak resolves dependencies, it fetches source packages from clefpak.dev (or local paths during development), constructs the full dependency graph including transitive dependencies, and determines the compilation order.

CCS Compilation

Where traditional F# uses FCS (F# Compiler Services) targeting the CLR, Fidelity uses CCS — a fork that provides native-first type semantics. CCS handles source-level dependency resolution natively:

Native Type Universe (NTU): All types resolve to native representations. string is a UTF-8 fat pointer, not System.String. option<'T> has value semantics, not heap-allocated.
[<FidelityExtern>] Recognition: When CCS encounters a binding declaration with this attribute, it emits an opaque extern node in the PSG rather than trying to compile the Unchecked.defaultof<T> body.
SRTP Preservation: Statically resolved type parameters survive through compilation, enabling Baker and Alex to make specialized code generation decisions.

Reachability Analysis

The power of source-based distribution becomes most apparent with reachability analysis. Traditional F# compilation includes entire assemblies even when you only use a single function. Our source-level approach enables surgical precision.

Starting from entry points like [<EntryPoint>] functions, the compiler traces through symbol usage to build a minimal dependency set. For our HelloWorld example, this analysis might discover that we only need:

Console.Write and Console.ReadLine from the platform library
A handful of supporting functions for string formatting and I/O
The write and read libc bindings that those platform functions depend on

Everything else — socket operations, threading primitives, file system calls — never enters the compilation graph. This intelligent reachability applies equally to Farscape-generated bindings: only the C/C++ functions actually called in the program receive MLIR function declarations in the final output. This embodies Bjarne Stroustrup’s famous quote, “only pay for what you use.”

The Compilation Pipeline

Source-level dependency resolution integrates with Composer’s nanopass compilation pipeline. The full process follows these steps:

Resolve dependencies declared in .fidproj using ClefPak
Discover source files from resolved packages (clefpak.dev and local)
Parse all required source files using CCS
Construct the Program Semantic Graph (PSG) from CCS output
Analyze reachability from entry points to prune unused code
Elaborate intrinsics — [<FidelityExtern>] extern nodes, syscall expansion
Saturate with Baker — expand HOFs, match expressions, sequence operations into primitive sub-graphs
Resolve platform bindings — the PlatformBindingResolution nanopass resolves each extern node to either Syscall (freestanding) or LibcCall/ExternCall (console) based on build configuration
Witness with Alex — traverse the enriched PSG, emit MLIR with binding metadata
Lower MLIR through progressive dialect transformations to LLVM IR
Optimize with LLVM — LTO across module boundaries for statically bound libraries
Link using collected library references — -lc, -lwayland-client, etc. from ExternCall metadata

Each step maintains diagnostic information, ensuring that compilation errors provide clear guidance about which library and source file contains the problematic code.

Binding Strategy Resolution

A critical nanopass in this pipeline is PlatformBindingResolution, which runs before Alex begins witnessing. This nanopass resolves every extern node to a concrete binding strategy:

// PlatformBindingResolution resolves all binding decisions before witnessing
type ResolvedBinding =
    | Syscall of syscallNum: int64 * constraints: string
    | LibcCall of funcName: string
    | ExternCall of library: string * symbol: string

// Alex receives pre-resolved bindings — no configuration lookup needed
let witnessExternNode (node: PSGNode) : MLIR<Val> = mlir {
    match node.ResolvedBinding with
    | ExternCall (library, symbol) ->
        // Emit external func decl + call with MLIR attributes:
        //   fidelity.binding_strategy = "dynamic"
        //   fidelity.library_name = "libc"
        return! emitExternCall library symbol node.Signature
    | Syscall (num, constraints) ->
        return! emitSyscallOps num constraints
    | LibcCall name ->
        return! emitLibcCallOps name
}

This separation ensures that witnesses remain pure transformations from PSG semantics to MLIR. Alex never queries build configuration — it emits what the pre-resolved extern node tells it to. Binding strategy changes (switching a library from dynamic to static linking) never require modifications to Alex’s witnessing logic.

BAREWire Integration: Memory Mapping Across Boundaries

One of the most exciting aspects of our architecture is the integration with BAREWire, our library for binary serialization, memory mapping, and IPC. When Farscape processes C headers, it generates not only Clef binding declarations for the C/C++ API but also BAREWire memory layout descriptors for relevant structures.

This enables several critical capabilities:

Zero-copy operations between Clef and C/C++ code
Efficient IPC between processes built with Fidelity
Consistent memory layouts across language boundaries
Safe serialization of complex data structures

BAREWire’s integration with Farscape-generated bindings creates a seamless experience for developers, who can write normal Clef code without worrying about the complexities of memory mapping and native interop.

Platform-Aware Compilation

One of the most sophisticated aspects of our dependency resolution is platform awareness. Modern software must handle diverse deployment targets, from embedded ARM controllers to desktop Linux to cloud servers. The .fidproj configuration drives platform-specific decisions:

[build]
output_kind = "console"    # or "freestanding", "embedded", "library"

[dependencies]
libc = { binding = "dynamic" }                    # System library — always dynamic
crypto = { version = "1.2.0", binding = "static" } # Security — static for LTO

[profiles.release]
binding.overrides = { crypto = "static" }

When compiling for a freestanding target, CCS intrinsics like Sys.write resolve to direct syscalls rather than libc calls. When targeting console mode, the same code links dynamically against the system’s libc. The binding strategy flows from .fidproj configuration through PlatformBindingResolution to Alex’s MLIR emission — the developer’s source code never changes.

The Developer Experience

A critical aspect of our approach is keeping the complexity minimal for developers. While the architecture is sophisticated, the developer experience aspires to remain straightforward through IDE integration and the Fidelity toolchain.

Integrated Development Environment

We’re working on a Fidelity extension for Visual Studio Code that bridges the gap between traditional F# development and our native compilation approach. When working with Fidelity projects, the extension:

Monitors changes to both source files and .fidproj files
Resolves dependencies through ClefPak’s package graph
Directs CCS for IntelliSense and error checking against the native type universe
Provides LSP and highlighting support for intermediate representations (.mlir, .ll)

The extension integrates with existing F# editor tools like Ionide where possible. The end goal is a hybrid environment that supports Fidelity framework projects while feeling like idiomatic Clef development workflow.

Practical Workflow: From Development to Deployment

The integrated workflow enables developers to move from concept to deployment with minimal friction:

Create a new Fidelity project with cpk new my_project
Edit Clef source files with full IntelliSense support via CCS
Resolve dependencies automatically through ClefPak and clefpak.dev
Generate C/C++ bindings with farscape project --project ./lib.moya.toml
Build your application with cpk build --target x86_64-linux
Deploy the resulting native binary directly to the target platform

Throughout this process, the developer rarely needs to manually specify file paths, manage include directives, or configure compiler settings. The framework manages the complexity of source-level resolution, Farscape-generated bindings, and platform-specific compilation, allowing the developer to focus on the goals of their Clef code.

Looking to the Future: clefpak.dev

While our current implementation focuses on local library resolution, we’re actively developing the clefpak.dev package registry that extends source-based distribution to a full ecosystem. Inspired by Rust’s Cargo system, ClefPak provides a streamlined way to discover, distribute, and consume Fidelity packages:

# Create new project
cpk new my_project

# Add dependencies
cpk add "Fidelity.libc.IO"
cpk add "Fidelity.libc.Memory"

# Build for a specific platform
cpk build --target x86_64-linux --release

Packages are distributed as .fidpkg archives — compressed source archives containing Clef code, metadata, and any Farscape-generated binding layers. The registry supports both centralized and federated models, with organizational registries for enterprise and air-gapped environments.

The integration between ClefPak, Farscape, and Composer creates a seamless development experience from package discovery to deployment:

Discover libraries through the clefpak.dev registry
Generate C/C++ bindings with Farscape from library headers
Add source-based dependencies to your .fidproj
Build your application with Composer — CCS compiles all sources together
Deploy native binaries to your target platform

This ecosystem approach enables the Fidelity Framework to grow organically while maintaining the core principles of zero-cost abstractions, type safety, and platform adaptability. By building on the foundation of source-level dependency resolution, we’re creating a development environment that combines the best aspects of F# with the performance and flexibility of native compilation.