GitHub Repos advanced 2 min read Sep 16, 2026 · Updated Sep 19, 2026
Public Preview Sign in free for the full digest →

Run 744B MoE Models From Disk With Colibri

“A C runtime streams MoE experts from disk so a documented 744B model does not need every expert resident in accelerator memory.”

Run 744B MoE Models From Disk With Colibri
1 Views
0 Likes
0 Bookmarks
Source · github.com

““I must say, this is incredible. I’d never thought it’d be possible to run these massive models on consumer-grade compute.” — Hacker News commenter”

You know that feeling when a model fits the task you want to test but cannot fit in your RAM or VRAM? Colibri targets that gap for MoE models, where each token only needs a routed subset of experts. Instead of requiring every expert to stay in accelerator memory, it reads the selected experts from a storage hierarchy. You trade simpler memory planning for local storage requirements, hardware-sensitive speed, and backend validation work.

aillmmoelocal-inferencecopen-sourcegpu

Think of the model as a library: Colibri keeps the main reference desk open in RAM or VRAM and fetches only the books the next request needs from NVMe. For each token, it runs the dense trunk, routes to a subset of MoE experts, then loads those experts through an LRU cache with pinned entries and lookahead. You set `COLI_MODEL` to a supported converted model and call `coli chat`, `coli web`, or `coli serve`. The C CPU engine runs without engine dependencies; optional CUDA, HIP, Metal, and Vulkan paths can handle supported accelerator configurations.

01
Disk-streamed routed experts — you can test supported MoE models without keeping every expert in VRAM or RAM.
02
VRAM, RAM, and NVMe tiers — you can place weights across the hardware you already own instead of treating GPU memory as the only usable store.
03
LRU cache, pinned experts, and lookahead — you can keep frequently selected experts close to compute and reduce repeat storage reads.
04
Pure C CPU engine — you can build the core with GCC or Clang plus OpenMP, without engine dependencies.
05
Optional CUDA, HIP, Metal, and Vulkan paths — you can test supported accelerator paths while retaining a CPU default.
06
Chat, web, server, doctor, and tune commands — you can inspect setup, run an interactive session, or expose a local service from the same tool.
Who it’s for

If you already run local models and have fast storage but limited RAM or VRAM, Colibri gives you a concrete MoE runtime to evaluate. It also suits systems engineers who want to study cache placement, expert routing, and reproducible performance evidence. Skip it if you need a low-risk serving platform across untested GPU configurations: open issues report output correctness problems on specific Vulkan and GB10 paths.

Worth exploring

Treat Colibri as a beta-stage, specialist runtime worth testing on an exact model and hardware match. v1.11.0 adds a ninth model family, closes five bugs, and expands CI coverage, while the repository still reports 131 open issues and two open GPU-path correctness reports. Its value lies in its storage-tiered MoE design, not in a blanket production-readiness claim.

Developer playbook
Tech stack, code snippet, sentiment, alternatives.
PM playbook
Adoption angles, user fit, positioning.
CEO playbook
Traction signals, ROI, build vs buy.
Deep-dive insight
Full long-form analysis, no fluff.
Easy mode
Core idea, fast — when you need the gist.
Pro mode
Technical nuance, edge cases, tradeoffs.
Read the full digest
Go beyond the preview

Deep-dive insight, Easy and Pro modes, plus action playbooks — the full breakdown is one tap away.

Underrated tools. Unfiltered takes.

Read the full digest in the Snaplyze app for deep-dive insight, Easy and Pro modes, and the playbooks you can actually use.

Install Snaplyze →