Run 700B AI Models on a Computer You Already Own
3 open-source engines now run models from 284B to 2.8T parameters on ordinary hardware — no data centre, no monthly subscription, and your files never leave the machine.
Run 700B AI Models on a Computer You Already Own
Until recently, running a frontier-scale open model meant renting a data-centre GPU. That changed in 2026 with a simple idea: treat your SSD, RAM and VRAM as one memory hierarchy instead of demanding the model fit in VRAM. Mixture-of-Experts models make this practical, because only a fraction of the model is active for any one token.
For anyone in India paying per-month for AI tools, this is the interesting part: the running cost is your electricity bill, and nothing you type leaves the room.
1. Colibrì — the widest hardware support
Colibrì is a pure C engine with zero dependencies that runs eight model families, from 7B up to 2.8 trillion parameters, by streaming experts from disk. It is Apache-2.0 licensed and deliberately built to work on hardware you already have rather than hardware you should buy.
- Same commands everywhere:
coli chat,coli serve,coli web - Vision support through GLM-5.3-Flash (321B)
- Written in C with no engine dependencies, so it builds almost anywhere
- Explicitly an experiment platform — expect speed to vary, not correctness
2. DwarfStar (ds4) — best on Apple Silicon
DwarfStar is a narrow, carefully-tuned engine written by Salvatore Sanfilippo (antirez, of Redis). It targets DeepSeek V4 Flash first, and also runs GLM 5.2, GLM 5.3 and GLM 5.3 Flash, with DeepSeek V4 PRO on very large machines.
It also has a serious server story. On 8×L40S — cards vLLM no longer supports for new models — the project reports roughly 120 tokens per second of aggregate generation and 2000 tokens per second of prefill across multiple sessions. That is an old GPU server turned into a company-wide LLM endpoint.
| Backend | What it needs |
|---|---|
| Metal (Mac) | 96 GB+ unified memory for the fast path; less works via SSD streaming |
| NVIDIA CUDA | Multi-GPU supported, including older Ada-generation cards |
| ROCm | Strix Halo systems, e.g. the Framework Desktop |
3. Be honest about what your machine can do
Neither engine makes a small laptop into a workstation. What they change is the floor — the point below which running a big model was simply impossible. SSD streaming trades speed for feasibility, and on a slow drive that trade is steep.
Pros
- No subscription — the cost is electricity
- Nothing you type leaves your machine
- Works offline once the weights are downloaded
- Open weights mean nobody can deprecate your model
Cons
- Weights are large — tens to hundreds of gigabytes to download
- Slower than a hosted frontier API on the same task
- Setup is a terminal job, not an app install
- Quality still trails the best paid models on hard reasoning
How to decide in two minutes
Check your RAM
96 GB or more of unified memory on a Mac, or a multi-GPU box: try DwarfStar first.
Check your disk
An NVMe SSD makes streaming usable. A spinning disk mostly does not.
Start small
Run a 7B model (OLMoE) end to end before downloading a 744B one.
Measure, then judge
Time a real task on your own machine. Benchmarks from other hardware will not match yours.
Do I need a GPU to run these?
Not necessarily. Both engines can stream from disk and use CPU and RAM, though a GPU makes a large difference to speed. Apple Silicon uses unified memory rather than a separate GPU card.
Is this legal and free?
Yes. Colibrì is Apache-2.0 and DwarfStar is MIT. The model weights carry their own licences — check the licence of the specific model you download, since a few restrict commercial use.
Will this replace ChatGPT for me?
For private work, drafting and code on a capable machine, often yes. For the hardest reasoning tasks, hosted frontier models are still ahead. Many people run both.
How much disk space do I need?
Plan for tens of gigabytes for a small model and hundreds for the largest ones. The model file is the bulk of it, and you need free space beyond that for the OS to breathe.
Save this summary as an image or share it.
AICreatorHub Team
The AICreatorHub editorial team is a group of hands-on AI practitioners, writers and developers based in India. We test AI tools and models ourselves, track official releases from OpenAI, Anthropic, Google, Meta and xAI, and translate them into simple, India-first guides in English and Hindi. Every article is written for real Indian use cases — pricing in rupees, free-tier tips and practical, tested steps — so you get accurate, up-to-date and genuinely useful AI information.