FAQ

Blunt answers to the questions a skeptic actually asks. Short version up front: VRAMPilot is a llama.cpp wrapper. What it does that the popular tools do not: recover from out-of-memory at runtime instead of crashing, remember the configuration that actually worked, and survive an OOM in the middle of a generation.

Is this just a GUI for llama.cpp?

Mostly, yes — and we say it before you do. VRAMPilot is a UX/automation layer on top of llama.cpp. llama.cpp does the actual work: loading the model, GPU offload, tensor split, serving. VRAMPilot runs no inference op itself — it launches llama-server and manages the flags. It is not a new inference engine and not a research result. What it adds on top: the runtime OOM-recovery loop, the persistence of what actually booted, the in-inference watchdog, and honest reporting of what was traded away. Those are the only parts worth caring about.

How is it different from LM Studio, Ollama or Jan? They already auto-fit models.

They do, and VRAMPilot does not claim otherwise. VRAM estimators and auto-context-sizing already exist — Jan's "Fit to Hardware", Ollama's VRAM-tiered context defaults, LM Studio's pre-load estimator. Those are table stakes. The difference is when the fitting happens: those tools prevent OOM at load time. None of them recover at runtime. When a configuration overflows anyway — an explicit context was forced, a long prompt pushed the KV cache over, VRAM was less free than estimated — they crash, or silently spill into system RAM and slow down without telling you. There is no detect → back-off → retry loop in any of them (probed by name in 2026, see the comparison page). That gap is the whole reason VRAMPilot exists.

What exactly is "OOM-recovery"?

When the server fails to come up — or boots but cannot generate a token — the launcher reads the error log, classifies it as out-of-memory, backs off one step, and retries. The order preserves the most capability: KV-cache precision first (keeps your full context, lossy on long reasoning), then shrink context, then offload more layers to CPU. If nothing fits, it says it hit the floor instead of pretending. In the validated run, a deliberately impossible configuration at 262144 context recovered to 131072 — 4x more context than a context-only back-off would have kept — and served a real reply.

What does it remember about my machine?

Every configuration that actually booted, persisted per (machine fingerprint, model header hash, requested context) in a local SQLite file — append-only: a configuration that stops working is kept as history, never overwritten. The next launch boots the known-good configuration directly; a driver or GPU change invalidates it and triggers a replan. Why bother: a failed attempt costs a full model load, measured at ~220 s per attempt for a 9.5 GB MoE on the gate machine's disk. Nothing leaves your machine; vrampilot configs list shows everything it remembers, and --no-cache skips it entirely.

What happens if VRAM runs out mid-generation?

That is a crash none of the tools we probed covers (by name, 2026 — see the comparison page): you are deep into a long generation, another application grabs VRAM, and the server dies. On NVIDIA, the watchdog measures free VRAM with real nvidia-smi reads, soft-alerts on a bad trend, and at the auto-calibrated floor does a controlled restart at a degraded configuration, then persists the survivor so the next launch starts there. Validated run under real pressure: floor crossed at 102 MiB free → recovered in 223.9 s at context 8192 → 4096 while the pressure stayed → served a real completion. The honest cost, stated by the tool itself: the generation in flight is lost. Recovery is real, not invisible. Counter-test: 0 soft alerts, 0 interventions across normal generations. On Vulkan/Apple, free VRAM is an estimate, so the watchdog honestly downgrades itself to process and health watching — and tells you so.

Do I need to install llama.cpp myself?

No. The first run fetches a pinned llama.cpp build (b9592) for your OS and GPU, with mandatory SHA256 verification: a mismatch deletes the file and aborts, hard. Measured on the gate machine: 7.6 s from cold to a real served completion. Defaults: Vulkan on Windows/Linux (one binary, any GPU vendor), Metal on macOS, CUDA opt-in. If you prefer your own llama-server build, you can still point to it.

Does it actually work? What is validated?

Measured on real hardware, not asserted. It runs and serves a real completion on 3 GPU vendors: an NVIDIA RTX 3070 (CUDA), an AMD Radeon 780M iGPU (Vulkan), and an Apple M1 (Metal). OOM-recovery was demonstrated end-to-end on NVIDIA and AMD: fed a deliberately impossible configuration, it recovered and served a real reply. Persistence and the watchdog were validated end-to-end on an NVIDIA RTX 4070 Laptop GPU, 8 GB — including a 9.5 GB MoE at 32k context via expert-offload. Raw logs are committed next to every claim.

Can it run a model bigger than my GPU's VRAM?

Within limits, yes — that is what offloading is for: the validated runs include a 9.5 GB file on an 8 GB GPU, fitted via MoE expert-offload. But there is a physical floor: a model too big even at the floor configuration cannot run, and no software beats physics. CPU offload also costs speed — VRAMPilot tells you what it traded, but slow is slow. And on a tiny machine, an absurd request can boot and produce garbage; normal use caps context to the model's training limit.

What's the moat? Couldn't LM Studio or Ollama just copy this?

Honestly — yes, they could. It is an engineering moat, not a patent or a data moat. It has already started at the prevention layer: recent llama.cpp builds ship a load-time auto-fit (-fit), so load-time prevention is now upstream table stakes. What remains unserved is the runtime recovery loop, the persistence, and the honest lossiness reporting. The bet is to ship now, go deep on the multi-axis back-off, and validate across vendors. If the incumbents add it, that is a win for everyone running local models.

What does it NOT do?

Said plainly: it is not a new runtime — llama.cpp does the inference, and if llama.cpp cannot run your model, neither can this. It does not make your GPU bigger — a model that does not fit even at the floor cannot run. It does not make recovery free — a watchdog restart loses the generation in flight, and a degraded configuration is degraded (the report names the loss). Apple recovery is not fully demonstrated: run-and-serve works on M1 and the mechanism is in place, but a clean forced-OOM demo was not possible on the small CI runner. It is a modest, useful utility built solo in a sprint — not a revolution.

Why should I trust the "honest lossiness" reporting?

Because it is the point, and because it is checkable. The tool shows the back-off trail and names each tradeoff; you see exactly which configuration booted. This website applies the same rule to itself: every figure on it is wrapped in claim markup pointing to the committed validation file that contains it, and the site build fails if a figure and its source disagree.

Is it free?

Yes, the tool is free to use. The code repository is private at the moment, with the intent to open it. It builds on llama.cpp, which is open source.