Benchmark

We shrank PersonaPlex by 62% to run at 20ms a frame.

PersonaPlex is NVIDIA’s speech model. It listens and talks at the same time, so it has 80 milliseconds to produce each frame of audio. As published it is 16.7 GB and spends about 40 ms on each frame on one laptop. Our TWX4 build is 6.3 GB and spends 20 ms. Given the original’s own conversations, it chooses the same next word 96% of the time.

See the matching eval card for the result and limits in one screen.

The setup

One voice model, four builds, one laptop.

A model that talks while it listens cannot fall behind. Every 80 milliseconds it owes the conversation another frame of audio. For a voice model, size and speed are the product, not a cost line. We took PersonaPlex as NVIDIA publishes it, made three smaller builds, and ran all four on the same laptop.

BuildWeightsWhat it is
PersonaPlex as published16.7 GBNVIDIA’s release, and the reference for everything below
Standard 8-bit conversion8.2 GBThe conversion the runtime ships with
Standard conversion at our size6.2 GBThe same converter, taken down to the size of our build
Tileward TWX46.3 GBOur build of the same model
Size and speed

6.3 GB against 16.7, and 20 ms of each frame against about 40.

The TWX4 build is 62% smaller than the published model. The published model spends about half of each 80 ms frame computing. The TWX4 build spends a quarter.

BuildWeightsCompute for each 80 ms frame
PersonaPlex as published16.7 GB37 to 42 ms
Standard 8-bit conversion8.2 GB23 ms
Standard conversion at our size6.2 GB19 ms
Tileward TWX46.3 GB20 ms

Speed follows size: the standard conversion at our size is as fast as ours. What separates those two is how closely each follows the original.

Same words

At the same size, TWX4 stays closer to the original than the standard conversion.

Smaller and faster only counts if the model still says what the original would. We recorded the original holding four conversations. Then we played them back through each build and, every time a word was due, asked which word the build would say next.

BuildWeightsSame next word as the original
PersonaPlex as published, replayed in this runtime16.7 GB99.2%
Standard 8-bit conversion8.2 GB98.1%
Standard conversion at our size6.2 GB92.8%
Tileward TWX46.3 GB96.0%

The first row is the ceiling. The standard 8-bit conversion comes closest to it, at 98.1%, for 1.9 GB more and 3 ms more a frame than the TWX4 build.

Speaking freely

A speech recogniser follows it as well as the original.

Then we let each build hold 20 conversations of its own, and had a speech recogniser write down what it heard. It heard 4.4% of the TWX4 build’s words differently from the words the build meant to say, against 7.0% for the published model. That gap is within the margin of this test, so the builds are level. No build went silent or repeated itself.

Deployment meaning

A voice model that leaves three quarters of every frame free.

On hardware you own, a voice model shares the machine with everything around it. A build that uses a quarter of each frame leaves room for the application, for more than one call, or for a smaller machine.

We measured the second of those. Four conversations at once took 34 to 39 ms a frame on the TWX4 build and 49 to 61 ms on the published model. Both are inside the 80.

How it was measured

Every build on the same laptop, the same runtime and the same conversations.

Both tests compare each build with the model as NVIDIA publishes it.

The machine. One MacBook Pro with an M4 Max. Every build ran in the same runtime, Apple’s MLX, from its own weights. The published model ran from the published weights, and each smaller build from its compressed ones.

Size. Files on disk.

Compute. The model alone on one conversation, with nothing else running on the machine. The audio codec runs beside it and is not part of this figure. The published model and the TWX4 build ran three sessions each: 36.9, 39.7 and 42.2 ms, and 19.9, 20.0 and 20.1 ms. Each standard conversion ran one.

Same words. Four conversations recorded from the published model: 106 seconds, with 375 moments where a word is due. A point either way is within chance.

Speaking freely. Four spoken questions, five runs each, about half a minute a conversation. Whisper medium wrote down what each build said. The 95% ranges are 3.0 to 5.8% for the TWX4 build and 4.3 to 10.2% for the published model.

Bring your own workload

Bring the voice model you run and the machine you run it on, and we will measure the same three things. Talk to an engineer, compare the voice model eval card, or read what we serve and what it costs.