colibri
nicheBrilliantly engineered local inference of a 744B MoE model, but the hardware appetite and glacial speed confine it to a niche of enthusiasts.
What it is
A pure-C inference engine for the 744B-parameter GLM-5.2 Mixture-of-Experts language model, designed to run on consumer hardware (25GB RAM, NVMe SSD) by keeping only the dense layers in memory and streaming the routed experts from disk on demand.
How it differs from vanilla Claude
A vanilla capable base agent (Claude) cannot run GLM-5.2 at all — it is a cloud API model. colibri enables local execution of this frontier-class model on a machine that costs a fraction of a GPU server, but at speeds of 0.05–2.8 tokens/second.
Skill, plugin, or workflow shift?
standalone-app — it is a compiled binary invoked from the command line; not a plugin or library. The repository provides a self-contained runtime, a one-time converter, and an optional HTTP server.
Devil's advocate — is this just complexity?
The primary counterargument is that the task itself — running a 744B model locally — is of dubious value given the extreme speed penalty. For the vast majority of users, using a cloud API (Claude, GPT-4) is faster, cheaper, and more reliable. The 370GB disk requirement and ~30s load time are substantial barriers. Moreover, colibri is tied exclusively to GLM-5.2; there are no alternatives. The engineering is undeniably impressive, but the practical utility is minimal: you get frontier-quality answers at a small fraction of the speed of even a local 7B model. Is it worth the complexity? For most, no.
What would make it better
Support for more model architectures (e.g., DeepSeek-V3, Llama) to broaden its value beyond GLM. A distributed mode to pool RAM/disk across multiple machines for faster expert loading. A high-level API for integration into other tools. Automatic selection of which experts to pin based on usage statistics. Faster cold-start by pre-fetching experts for likely early tokens.
The honest case for it
If you need to run a 744B model completely offline on commodity hardware — for privacy, censorship avoidance, or education — colibri is the only option. Its C codebase is a model of minimalism and performance optimization. For the small audience that values sovereignty over speed, it is a marvel of systems engineering.
Who it's for
Audience fit
Depth and leverage for a technical engineer who wants to understand it and level up their workflow — not just offload work.
Value for someone who wants a more capable tool without the technical depth — accessible, does-it-for-you.
Deep technical engineering requiring C compilation, large model downloads, and tolerance for sub-1 tok/s speeds; not for users seeking turnkey solutions.