Frontier Bridge

Memory-hierarchy planning for giant MoE models on hardware you already own.

Download the runbook for your hardware class. Every number in it is measured or absent — and when the numbers don't work, you get an honest refusal, not a hopeful guide.

v0.1-dev — first verified row landed 2026-07-05: a 284B MoE running agent-capable on a single workstation, reproduced 2x (matrix · roadmap)

Quickstart · Runbooks · Demo · How it works · Architecture · Status · Honest limits · Matrix · FAQ · Contributing

Can a single high-end machine run frontier-scale open models — GLM-5.2 (744B total / 40B active, upstream claims), DeepSeek V4 class — if VRAM, unified memory, system RAM, and fast NVMe are planned as one hierarchy? That is the question this project exists to measure. The answer ships in two forms: a planning CLI (profile hardware → plan the tiers → launch command or honest refusal) and runbooks — distributable playbooks per hardware class whose every number is mechanically traceable to a hash-pinned, twice-reproduced benchmark file.

git clone https://github.com/Brianletort/Frontier-Bridge.git
cd Frontier-Bridge && ./scripts/bootstrap.sh   # venv, install, then `frontier doctor`:
                                               # what this machine is, what's missing
                                               # (with the fix), which runbook applies
source .venv/bin/activate

# profile this machine (hardware_profiles/local/ is gitignored)
frontier detect -o hardware_profiles/local/my_machine.yaml
PROFILE=$(awk '/^profile_id:/{print $2}' hardware_profiles/local/my_machine.yaml)

frontier catalog models                # the curated catalog: GLM-5.2, DeepSeek V4 Flash, ...
frontier plan deepseek-v4-flash --quant q2_imatrix \
    --hardware "$PROFILE" --workload coding_agent --ctx 32768

No supported detect path for your OS yet? frontier catalog hardware lists committed reference profiles (rtx6000_96gb_64ram, gb10_128gb, m5_max_128gb) you can plan against immediately.

"What can I actually run on my hardware?" clusters into hardware classes — a GB10, an M5 Max 128 GB, and a 16 GB-VRAM/128 GB-DDR laptop are three different answers to the same question, each shared by everyone who owns a machine in that class. A runbook answers it once per class: diagnosis, a curated model menu with planner verdicts (refusals listed on purpose), exact launch recipes, and measured expectations.

One command finds yours:

frontier runbook match          # profiles this machine, lists the runbooks that apply

What makes these different from the internet's pile of "run model X on hardware Y" posts is the provenance rule, enforced by CI: a runbook's numbers are folded from committed benchresult/v1 files or they do not appear (RFC 0003). frontier runbook verify rejects any performance claim without a backing result file, and any menu verdict that disagrees with its committed plan. Prose is authored; numbers never are.

The model menu draws from a deliberately small catalog — top open-weight models per size class, enterprise-usable licenses required, admission criteria written down in RFC 0004 so curation is a rule, not a popularity contest.

Runbooks live in runbooks/ (YAML source + rendered markdown). Two ship today — unified-128gb, backed by the first verified rows, and gpu-laptop-128gb (draft: planner-verified menu, performance honestly unmeasured until its hardware is benched) — with the RTX 6000 class next. Contributing a runbook for a class we don't cover is one of the most valuable PRs this project accepts.

The loop below is the real CLI on the M5 Max — recorded with VHS, reproducible from the committed tape. The full YAML these commands produce is in See it work:

detect and plan run in seconds with nothing downloaded. The full loop — download a pinned GGUF, frontier run, frontier bench, reproduce twice, fold into the matrix — is the benchmark playbook. Model artifacts are 107–467 GB; the planner tells you before you download whether your machine is worth the bandwidth. (On Apple Silicon the planner defaults to the ds4 engine; pass --engine llama_cpp if that's what you have installed.)

Profile → plan → refusal, on real hardware — an M5 Max, sold as 128 GB (GiB), which detect measures as 137.4 decimal GB of unified memory. These are schema and planning outputs, not benchmark-verified results; usability ratings wait for the benchmark pipeline, per the results-integrity rule.

1. Profile the machine. frontier detect measures what it can and records null for the rest — no spec-sheet guessing:

profile_id: apple_m5_max_137gb_detected
nodes:
- id: unified0
  kind: memory
  class: unified
  capacity_gb: 137.4
- id: ssd0
  kind: storage
  class: internal_ssd
  measured:
    seq_read_gbps: 14.22          # measured on this machine, not quoted
    bench_tool: frontier-detect python bounded read (512MB, uncached, qd1)

2. Plan a model.

frontier plan deepseek-v4-flash --quant q2_imatrix --engine llama_cpp \
    --hardware apple_m5_max_137gb_detected --workload coding_agent --ctx 32768

Planner output, not a benchmark-verified result (shown with llama.cpp; ds4 is the default engine on Apple Silicon). recommended means a measured memory fit with headroom, not proven usability — and the l2 tier is overflow topology, present even when the artifact fits resident:

verdict: recommended
placement:
  resident: { dense_core: unified0, router: unified0, shared_experts: unified0 }
  tiered:
    routed_experts:
      l0: { node: unified0, budget_gb: 116.9, policy: layer_aware_lru }
      l2: { node: ssd0, mode: stream_on_miss }
phases:
  prefill: { notes: throughput-bound; expert misses tolerable }
  decode:  { prefetch: static_hotlist, notes: miss-sensitive; protect the active path }
runtime:
  engine: llama_cpp
  launch: llama-server -m <GGUF_PATH> -c 32768 --host 127.0.0.1 --port 8080 -ngl 999
risks:
- agent_workloads_are_decode_latency_sensitive

3. Get refused when the numbers don't work. Ask for more context than the model supports and the planner will not pretend:

# frontier plan glm-5.2 --hardware rtx6000_96gb_64ram --workload long_context --ctx 2000000
verdict: not_recommended
reasons:
- 'context_budget_exceeds_claimed_max: 2000000 > 1048576'

That refusal is the product working as designed. A planner you can trust when it says yes is one that says no out loud.

The memory hierarchy from computer-organization class is back — orders of magnitude bigger at every level, with the cache line replaced by the expert slice, and this time no hardware manages it for you. The planner is the missing cache controller. The full formalization is in docs/memory_hierarchy.md:

Sparse MoE is what makes the new hierarchy workable. A 744B-total model activates ~40B parameters per token (upstream claims) — so the question worth testing is not "does 744B fit in memory" but "can the active path stay fast while inactive experts tier across RAM and SSD?" That is a planning problem, and it is the problem this project works on.

Everything is a graph of resources and links, measured where possible, null where not — never assumed. A hardware profile has no hardcoded vram/ram/ssd fields — it has memory, compute, storage, and network nodes joined by links with measured bandwidth. An RTX 6000 box, a GB10 with coherent unified memory, and a Mac Studio are the same schema with different topology. So is a multi-GPU rack node — which is what makes scaling up a schema change, not a rewrite. And because tiers are bandwidth classes rather than device categories, Thunderbolt SSDs, eGPU enclosures, and NAS boxes slot in wherever their measured links put them (RFC 0002). Design details in RFC 0001.

flowchart LR
    detect["frontier detect<br/>measure this machine"] --> hw[("hwprofile/v1<br/>resource graph")]
    models[("modelprofile/v1<br/>curated catalog, hash-pinned")] --> plan
    hw --> plan["frontier plan<br/>fit check + tiered placement"]
    plan --> planfile[("plan/v1<br/>placement + launch command")]
    planfile --> run["frontier run<br/>verify sha256, launch, health-check"]
    run --> bench["frontier bench<br/>TTFT, decode tps, p95"]
    bench --> result[("benchresult/v1<br/>pinned + reproduced")]
    result --> matrix["frontier results matrix<br/>compatibility matrix"]
    result --> runbookFold["fold: numbers, never authored"]
    runbookFold --> runbookFile[("runbook/v1<br/>per hardware class")]
    runbookFile --> rendered["frontier runbook render<br/>downloadable playbook"]
    plan -. "verdict: not_recommended" .-> refusal["honest refusal<br/>with reasons"]

Frontier Bridge is not another inference runtime. Plans emit exact launch commands for llama.cpp, ds4 (an SSD-expert-streaming runtime for Apple Metal), MLX, vLLM, and SGLang — launch-command wrappers today, not deep integrations; TensorRT-LLM and KTransformers are planned (adapter status) — and the harness measures the result. Prefill and decode are planned separately from day one: prefill is throughput-bound and tolerates expert-cache misses; decode is miss-sensitive and gets latency targets and prefetch policy of its own. Protecting the interactive path is where usefulness lives.

Want the long read — the planner's decision path, the worked worst-case streaming math, topology diagrams for all three reference machines? docs/architecture.md.

People who care about this problem rarely own one machine. The fleet layer (RFC 0005) registers the machines you own and answers the question one level up — frontier fleet plan <model> runs the planner against every registered profile and ranks the verdicts; fleet detect and fleet bench operate the same CLI remotely over SSH (Tailscale names work unchanged) and pull the artifacts back for review.

Two machines can even be one topology: a Thunderbolt cable between two 128 GB laptops creates a combined resource graph whose RAM fits models neither fits alone, pipeline-split at a layer boundary. Whether that is usable is an open measurement question with a written protocol and decision rule — the dual-node spike — and it becomes a supported topology only if the numbers earn it.

The measurement pipeline is built and tested; the first verified rows are the current milestone (roadmap). What works today:

Next: bring up the GB10 and the discrete-GPU laptops (fleet runbook), publish their verified rows and runbooks, run the dual-node Thunderbolt spike (protocol), tag v0.1.0, then the CUDA SSD expert-streaming spike (protocol). Full phased plan: docs/roadmap.md.

• We do not make any frontier model run on consumer hardware. The goal is to make the best possible use of available infrastructure — turning models from impossible into runnable, and from runnable into useful, is the hypothesis the benchmarks exist to test.
• Every number in this repo carries its provenance: measured applies to hardware telemetry we ran ourselves, claimed to upstream model metadata, and verified only to benchmark rows backed by pinned hashes and two reproductions (GOVERNANCE.md). Anything we can't support is null — we never guess.

More hard questions — SSD endurance, Q2 quality, external drives and eGPUs — get straight answers in the FAQ.

The v0.1 targets: GLM-5.2 (Q2/Q4 routed GGUF) and DeepSeek V4 Flash (Q2/Q4 imatrix GGUF) across four hardware classes. Usability ratings, in order: unrated → runs → usable → interactive → agent_capable, plus not_recommended as a first-class result. No row moves past unrated until a hash-pinned benchresult/v1 file backs it, reproduced twice. Context ranges are targets, not results:

First verified rows landed 2026-07-05: a 284B-parameter MoE running as a usable coding agent on a single Apple workstation, every number pinned and reproduced twice (results/verified/).

Additional MoE families (Qwen, MiniMax, Kimi, and newer GLM/DeepSeek releases) follow the same profile format. The live scoreboard — generated from committed benchresult/v1 files, never hand-edited — is docs/compatibility_matrix.md.

The operating thesis is simple: own the steady state, rent the surge. Run your steady-state AI workloads on infrastructure you own — a workstation, a lab server, a private rack — and rent cloud capacity for the peaks. Frontier Bridge makes the owned side of that equation credible: measured proof of what hardware you own can actually run.

AI runs across a continuum of infrastructure — laptops, workstations, lab servers, private racks, colocation, and hyperscale platforms — and the aim is for the most capable open models to be practical at every rung someone can afford. Because the same resource-graph schema describes a workstation and a rack node, the project's profiles, plans, and benchmarks carry across that whole range. Full mission and strategy: MISSION.md; the upward path is sketched in docs/enterprise_bridge.md.

Goals (v0.1). Verified benchmark rows for GLM-5.2 and DeepSeek V4 Flash on three reference machines — RTX 6000 96 GB, GB10 128 GB, M5 Max 128 GB — and an open schema standard anyone can submit results against.

Principles (from MISSION.md):

1. Credibility before code. Honest limits and refusal behavior matter more than early features.
2. Measurement before optimization. Nothing is verified without pinned hashes and two reproductions.
3. Open standard before novel kernels. The schemas and benchmark format are the durable contribution.
4. Not another runtime. We generate exact launch commands for existing engines and measure the result.

The most valuable early contributions are hardware profiles from machines we don't have (RTX 5090 + 128 GB DDR and Strix Halo are open gaps), benchmark results that reproduce, and runbooks for hardware classes we don't cover — a runbook is schema-validated and CI-checked (RFC 0003), so contributions are mechanically verifiable, not wiki edits. Start with CONTRIBUTING.md; the results-integrity rule is in GOVERNANCE.md. Issue templates exist for volunteering a profile and reporting hardware quirks, and Discussions is open — first thread: which machine should we profile next?

schemas/              JSON Schema files for the v1 schemas
rfcs/                 design RFCs (schemas are ratified here)
hardware_profiles/    committed hwprofile/v1 YAML (detected or manual provenance)
model_profiles/       committed modelprofile/v1 YAML per model/quant (curated per RFC 0004)
plans/                example plan/v1 outputs
results/              benchmark results (community and maintainer-verified)
runbooks/             runbook/v1 per hardware class + rendered markdown (RFC 0003)
fleet/                fleet registry example (real registries live in gitignored fleet/local/)
runtime_adapters/     per-runtime adapter status
notebooks/            reproducible figures generated from committed profiles
docs/                 roadmap, playbooks, spike protocols, enterprise bridge, launch checklist
src/frontier_bridge/  the CLI and library
tests/                pytest suite

Apache 2.0. See LICENSE, NOTICE, and IP_NOTICE.md.