"How much flash does a TinyML model need" is usually answered with the weight count, which is the smallest of the three numbers that matter. Here is the full arithmetic, so you can budget a part before you train anything.
The three things that consume flash
| Component | What it is | Typical size |
|---|---|---|
| Weights | The parameters, quantized | params × bytes-per-weight |
| Runtime | Interpreter, operator kernels, dispatch tables | 0 to ~40 KB |
| Model description | Graph structure, tensor metadata | a few KB, or zero if compiled in |
The runtime is the one people forget, and it is often the largest. For the same 6,567-parameter network we measured:
| Bytes | Compiled fixed-shape engine | TensorFlow Lite for Microcontrollers |
|---|---|---|
| Weights | 7,044 | 32,800 |
| Runtime in flash | 8,400 | 38,912 |
| Total | 15,444 | 71,712 |
The interpreter costs more flash than the model does. That is not a criticism of TensorFlow Lite — an interpreter is what lets you swap models without reflashing, which is a genuinely valuable property. It is a criticism of budgeting from the weight count alone.
Bytes per weight, by quantization
| Scheme | Bits/weight | 10K params | Multiplies? |
|---|---|---|---|
| FP32 | 32 | 40,000 B | yes, floating point |
| int8 | 8 | 10,000 B | yes, integer |
| int4 | 4 | 5,000 B | yes, integer |
| Ternary, packed base-3 | ~1.6 | ~2,000 B | no — sign flip, skip or copy |
Real example rather than a formula: our ternary-32 model packs to 4,765 bytes of weights, against 7,044 for the int8 version of the same network and 18,840 for the TFLite FlatBuffer in the same run.
RAM is the constraint that actually bites
Flash is cheap and getting cheaper. SRAM is neither. A part with 512 KB of flash and 20 KB of SRAM is completely ordinary, and it is SRAM that decides whether your model runs. Two things consume it:
- Activations. The intermediate tensors. With an interpreter you pay for a tensor arena sized to the largest concurrent set.
- Attention caches, if you have attention. A key-value cache grows with context length, and this is what puts transformers over the SRAM wall on this class of part more often than any other single factor.
Measured, for the Atome LM language model line, peak RAM including stack high-water on a real Cortex-M3 build: 14.5 KB for the nano model, 27.5 KB for small, 52 KB for the classifier, 104 KB for tinystories, 205 KB for mid, 411 KB for the full 944K production model. The full table, mapped against real parts, is here.
A budgeting procedure
- Write down the flash and SRAM you can actually spare after the rest of the firmware. Not the part's totals.
- Subtract the runtime. Zero for a compiled engine, tens of kilobytes for an interpreter. Do this before choosing a model size, not after.
- Divide the remaining flash by bytes-per-weight to get your parameter ceiling.
- Check the parameter ceiling against SRAM, not flash. This is where most plans die.
- Only now pick an architecture, and prefer one whose state does not grow with input length.
Doing it in that order takes an afternoon. Doing it in the other order takes a respin.
Questions
How much flash does a TinyML model need?
Weights plus runtime, and the runtime is usually the bigger half. For the same 6,567-parameter network we measured 15,444 bytes total for a compiled fixed-shape engine against 71,712 for TensorFlow Lite for Microcontrollers — of which 38,912 bytes is the interpreter.
How many bytes does one weight take?
Four for FP32, one for int8, half for int4, and about 0.2 for ternary packed four trits to the byte in base 3. Our ternary-32 model packs to 4,765 bytes of weights against 7,044 for the int8 version of the same network.
Is flash or RAM the real constraint on a microcontroller?
RAM, almost always. Flash is cheap and a part with 512 KB of flash and 20 KB of SRAM is completely ordinary. Activations and any attention key-value cache consume SRAM, and a cache that grows with context length is the most common reason a transformer does not fit.