We compressed VoiceChat by 74% and made it run in real time on one card.

A voice model that can act on a request has two jobs: keep up with the caller, and do the right thing. We made a smaller build of VoiceChat and checked that it still does the right thing: the same tool calls, and the same words. Then we changed how the model is served, so that it keeps up.

NVIDIA’s file stores every number in 32 bits. Nine tenths of them fit in 16 bits exactly, which takes the file to 24.2 GB without changing a single value. The TWX4 build is 11.5 GB: 74% smaller than the published file and 53% smaller than that 16-bit copy.

Smaller only counts if the model still acts. We spoke the same 100 requests to both builds: 80 that call for a tool and 20 that call for none. A tool call has two parts, which tool and what to pass it, so we scored both.

A gap of three or four calls in 80 is within chance, so the two are level. They made the same first call on 89 of the 100 requests.

We recorded the original answering those 100 requests. Then we played each conversation back through our build and, every time the original said a word, asked which word our build would say next.

The first row is the ceiling. Run in 16 bits, the way a server runs it, the published model matches its own recording on 99.3% of words.

A voice model makes sound in frames of 80 milliseconds, so it has 80 ms to finish each one. NVIDIA’s open code, run as it ships with five tools in the prompt, takes 114 ms and falls further behind as the call goes on. With our serving changes, the same published model on the same prompt takes 72 ms.

The server runs every build from unpacked weights. So on the card, the TWX4 build takes the same time per frame and the same memory as the published model.

What gets smaller is the file you download, store and move to where the model runs. It is 11.5 GB: a quarter of the published file, and half of its 16-bit copy.

Every figure here sets the TWX4 build beside VoiceChat as NVIDIA publishes it, on the same inputs.

Size. Files on disk.

Tool calls. 100 spoken requests in six synthetic voices on a clean line, one request and its answer each. Forty were asked against five operations tools, forty against five everyday tools, and twenty call for no tool. Both builds ran NVIDIA’s own system prompt and tool template, and the score is the first call each one made.

Same words. The original’s 100 recorded conversations, played back through each build and compared at 1,661 words and 1,848 tool-call tokens.

Speed. One NVIDIA 96 GB card running NVIDIA’s open pipeline, one conversation at a time. Times are the average frame. For both builds, a request’s slowest frame took 86 to 89 ms, and 3 of 19 requests had one near 350 ms. A host needs an audio buffer for those.

Repeated answers. Across 100 answers offline and 20 through the streaming path, an answer repeated itself twice for the original and once for the TWX4 build.