// blog

Why models over 4 GB failed in the browser, and the two fixes

By Ozgun Ay · 4 September 2026 ·5 min read

wllamawebassemblywebgpullama.cpp

For a long time AgentOp could load a 2.6 GB model in a browser tab and could not load a 5 GB one. The failure looked like a single bug. It was two, sitting on top of each other, and they happened to share a number: 4 GB.

This is what both were and how each was fixed, because the second one is easy to miss and cost more time than the first.

The setup

AgentOp runs llama.cpp compiled to WebAssembly, through a fork of wllama. A GGUF file is downloaded from Hugging Face, cached by the browser, and executed with WebGPU offload where the machine has a usable GPU. The current pin is wllama 3.6.1 on llama.cpp build b10735-1b89a43.

Model sizes in the registry run from about 0.5 GB to 7.5 GB. Anything at or below roughly 3.5 GB worked. Above that, nothing did.

Ceiling one: wasm32 has a 4 GB address space

A WebAssembly module compiled for wasm32 addresses memory with 32-bit pointers. That caps its linear memory at 4 GiB, and that cap is the whole address space, not just the model weights: llama.cpp also needs room for the KV cache, the compute buffers and its own allocator overhead inside the same space.

In practice, models over about 3.5 GB had nowhere to live.

The fix is a Memory64 (wasm64) build. The 64-bit variant raises the ceiling to 16 GiB, which is the maximum the WebAssembly JavaScript API currently permits. Both shipped builds are wasm64:

a JSPI build, used where the browser supports the JavaScript Promise Integration API (Chrome does), and
an Asyncify build as the fallback for browsers that do not.

The two builds are not interchangeable at runtime, and the glue code is versioned with the binary. Four vendored artifacts have to move together or you get silent corruption rather than a clean error: the JS bundle, the single-thread .wasm, and the compat .js plus .wasm pair.

That change alone did not fix the download.

Ceiling two: the JavaScript heap is also capped near 4 GB

With wasm64 in place, a 4.9 GB model still failed — but now during the download, before llama.cpp ever saw a byte.

The downloader read the response stream and accumulated chunks so it could report progress:

const reader = response.body.getReader();
const chunks = [];
let loaded = 0;

while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    chunks.push(value);
    loaded += value.byteLength;
    progressCb({ loaded, total });
}

const blob = new Blob(chunks, { type: 'application/octet-stream' });

Every chunk stays in the JavaScript heap until the Blob is constructed at the end. The V8 heap has its own limit in the same neighbourhood as the wasm32 one, so the array of chunks ran out of room at almost exactly the size where the first ceiling used to bite. Same symptom, different subsystem — which is what made it look like the wasm64 change had simply not worked.

The fix is to never hold the file in the heap at all. Pipe the response through a TransformStream that counts bytes as they pass, then let Response.blob() write the result into the browser’s disk-backed blob storage:

// Count bytes as they stream past, then let Response.blob() write them
// straight to the browser's (disk-backed) blob storage. Collecting the
// chunks in an array instead would hold the whole model in the JS heap,
// which is capped near 4GB — every model above roughly 3.5GB failed to
// download on a cache miss.
let loaded = 0;
const counter = new TransformStream({
    transform(chunk, controller) {
        loaded += chunk.byteLength;
        progressCb({ loaded, total });
        controller.enqueue(chunk);
    },
});
const counted = new Response(response.body.pipeThrough(counter), {
    headers: { 'Content-Type': 'application/octet-stream' },
});
const blob = await counted.blob();

Progress reporting is preserved, the peak heap usage is one chunk, and the browser decides where the bytes actually live.

Two other things that bite at this size

OPFS does not work from file://. wllama’s built-in model cache uses the Origin Private File System, which requires a real origin. A downloaded standalone agent opened by double-clicking has the origin null, so the cache silently never hit and every run re-downloaded several gigabytes. The cache backend was replaced with the Cache API, which does work on file://, with a no-op fallback when even that is unavailable.

Threads need COOP/COEP headers. A multi-threaded WebAssembly build needs SharedArrayBuffer, which needs the page to be cross-origin isolated. A hosted page can set those headers; a file on disk cannot. So the hosted run page uses the pthread pool and the standalone file is single-threaded on the CPU side. GPU offload works in both, which is what keeps the standalone file usable.

What it looks like now

On an NVIDIA RTX 4090 in Chrome, Qwen 3 4B (Q4_K_M, a 2.6 GB download) generates at about 57 tokens/second once the model is warm. The 8B-class models that were previously impossible to load now load.

The registry currently offers 27 models across 10 families, from roughly 0.5 GB to 7.5 GB, all Q4_K_M GGUF files pulled from Hugging Face and cached by the browser after the first run.

The general lesson

When two independent limits share a number, fixing one of them looks like fixing nothing. The wasm64 work was correct and necessary and changed no observable behaviour, because the download failed first. It took explicitly asking “which subsystem is out of memory?” rather than “why does the big model fail?” to see the second ceiling at all.