int8 is the default for microcontroller inference and it is a good default. Ternary goes four to five times further on size and removes multiplication entirely. This is when that trade is worth making, and when it is not.
The two schemes side by side
| int8 | Ternary (1.58-bit) | |
|---|---|---|
| Values per weight | 256 | 3: −α, 0, +α |
| Bits per weight | 8 | log₂(3) ≈ 1.58, ~2 packed |
| Inner-loop operation | integer multiply-accumulate | sign flip, skip or copy — no multiply |
| Post-training quantization | works well | does not work — must train ternary |
| Hardware support | everywhere, often with DSP instructions | none; it is plain integer work |
| Sparsity | incidental | structural — the zero level is free skipping |
Measured, same network, same task
Our own 6,567-parameter TCN on UCI HAR, both paths exported and run through the real C engine:
| Weight bytes | Accuracy | |
|---|---|---|
| int8, 5 seeds | 7,044 | 0.9095 ± 0.0077 |
| ternary-32, deployable, 1 seed | 4,765 | 0.9053 |
| ternary-32, float before quantization, 5 seeds | — | 0.9170 ± 0.0151 |
Read the middle row carefully. 0.9053 is one seed of the deployable static-quantized path, measured against a five-seed int8 mean. It is nominally below int8. What is earned is a deployable, C-verified artifact at 4,765 bytes with exact 2947/2947 parity — 1.48× smaller than int8. It is not an accuracy result, and five seeds of the deployable path have not been run.
One more number worth having: static quantization of the ternary model costs about 1.17 percentage points — float 0.9170 down to 0.9053 deployed. If you see a ternary accuracy figure that came from a dynamic-quantization convenience path rather than the static artifact you would actually ship, it is about 1.2 points optimistic.
Why you cannot post-train quantize to ternary
int8 tolerates rounding a trained float model afterwards, because 256 levels is enough to land close to the original value. Three levels is not; naive rounding destroys the network. Ternary models are trained ternary from the start with a straight-through estimator — the forward pass quantizes, the backward pass updates a hidden full-precision copy as if quantization were the identity. Over training the network learns weights that survive being snapped to three levels, because those are the only values it has ever seen in its own forward pass.
Practical consequence: adopting ternary is a training decision, not a deployment decision. You cannot try it at the end of the project.
When to choose which
| Choose | When |
|---|---|
| int8 | You have an already-trained model; your part has a DSP or SIMD instruction set to exploit; flash is not the binding constraint; you want the mainstream toolchain. |
| Ternary | Flash is the binding constraint; the core has no floating-point unit and no useful SIMD; you are training from scratch anyway; you want the zero level's structural sparsity. |
| Neither, yet | You have not measured which resource actually binds. Do that first — it is an afternoon, and it usually changes the answer. |
The part that is not a trade-off
At very small scale, ternary is not purely a cost. The ternary inductive bias substitutes for capacity when there is very little of it: at 60,000 parameters the routed-ternary block beats a parameter-matched FP32 transformer by about 22% on TinyStories perplexity. The effect reverses above a million parameters, where the float model wins by about 11%. So ternary is a small-model technique that happens also to save flash, rather than a compression technique that happens to work on small models — and both of those single-seed numbers are directions, not results.
Questions
Is ternary quantization better than int8 for microcontrollers?
Smaller, yes — about 4× fewer weight bytes and no multiplication in the inner loop. Better depends on your constraint. int8 works as post-training quantization on an existing model and exploits DSP instructions; ternary must be trained ternary from the start and cannot be retrofitted at the end of a project.
Can I convert my existing model to ternary weights?
No. Three levels per weight is too coarse for naive rounding to survive. Ternary models are trained ternary using a straight-through estimator, so the network learns weights that are robust to being snapped to three levels.
How much accuracy does ternary cost?
On our UCI HAR network, static quantization of the ternary model costs about 1.17 percentage points — 0.9170 float against 0.9053 deployed, though that deployed figure is one seed against a five-seed int8 mean of 0.9095.