Atome LM EDGE vs TensorFlow Lite for Microcontrollers

This page is the whole comparison between Atome LM EDGE and TensorFlow Lite for Microcontrollers on the benchmark we have run most carefully: UCI HAR human-activity recognition, 2,947 held-out windows from 30 subjects, the same 6,567-parameter temporal convolutional network compiled through both engines.

It includes the axis we lose on and the axes nobody has measured. That is not modesty — a benchmark you can only read one way is not a benchmark, and an embedded engineer choosing a runtime needs the shape of the trade, not a headline.

Who is who

Both arms are named, because a comparison where the reader cannot tell which component belongs to whom is worthless.

• Ours: the Atome LM EDGE C99 engine — a fixed-shape int8 temporal convolutional network with no interpreter, no heap and no scratch arena.
• Opponent: TensorFlow Lite for Microcontrollers (TensorFlow 2.21.0), the standard Google runtime for microcontrollers, running the same architecture.
• Opponent, kernel level: CMSIS-NN v4.1.0 — ARM's optimised kernel library, and the code X-CUBE-AI generates underneath. Where a cycle count is compared, CMSIS-NN is the opponent's number, not ours. We did not write it and we do not claim it.

1 · Accuracy — a tie, and it is a real tie

The split is subject-wise: the 30 people in the test set never appear in training, which is the only split that means anything for a wearable. Model selection happens on a validation set carved out of the training subjects. The test set is scored once. Five seeds per arm.

Paired difference: -0.000204, against a paired 2·SE of 0.017222. The difference is roughly eighty times smaller than the noise band. Our arm is ahead on 3 of 5 seeds, which is what a coin does.

This is a tie and we call it a tie. An earlier version of this comparison reported a win from a single seed. It was wrong: the 5-seed mean does not support it. If you see an Atome accuracy figure anywhere quoting a single number above 0.91 against a TFLite Micro constant, it is superseded by this table.

Two details that matter for reading the table honestly. First, the opponent is trained at a matched budget — same epochs, same tuning effort — not the weaker as-is configuration, which scores 0.8958 and would have flattered us by 1.4 points. Second, our own shipped checkpoint scores 0.9206 on the same 2,947 windows through the real C engine, but that checkpoint was selected on the test set, so it is not the number to quote. The leak-free 0.9095 is.

2 · Flash and RAM — a win, 4.64×

Same network, same parameter count, both engines built and measured here.

The gap is structural rather than clever. TensorFlow Lite for Microcontrollers carries an interpreter: it reads a FlatBuffer at boot, walks an operator table and dispatches through it, and it needs a tensor arena to do that. Our engine has no interpreter to carry, because the network shape is compiled in. That is a real cost on their side and a real constraint on ours — changing the model means recompiling the firmware, where they can swap a FlatBuffer.

The ternary variant pushes the weights further. Packed four trits to the byte, the 32-channel model is 4,765 bytes of weights against the 18,840-byte FlatBuffer in the same leak-free run — 3.95× — and it is verified end-to-end: the real C engine agrees with the reference on 2947/2947 windows, exactly. Its accuracy is 0.9053 on one seed, so it is a deployable artifact, not a result: five seeds of the deployable path have not been run.

3 · Per-decision cost on a sliding window — a win, 27.3×

This is the only axis on this page where the difference is not a matter of degree. A stateless runtime exposes a per-tensor convolution with no way to retain per-layer history between calls. Ask it for a decision and it recomputes the whole window. Ask it again one sample later and it recomputes the whole window again, including the 127 samples that did not change.

Our engine keeps per-layer activation rings and computes one new timestep per layer per sample. The rival API cannot express that. It is not a speed contest; it is a capability the interface does not have.

27.3× cheaper per decision than CMSIS-NN, 79.6× cheaper than our own batch path — and the decisions are bit-identical to full recomputation, proved on every window, not sampled.

The boundary, stated with the result. This holds only for overlapping windows. Streaming wins when a decision is needed more often than every 27 samples against CMSIS-NN, or every 80 against our own batch path. At stride 128 — non-overlapping windows — streaming costs 1.6× more, and it costs about 10 KiB of RAM for the state (24,732 B bss against 6,192 B). On a part with 20 KB of SRAM that trade may not be worth making. The claim is not "we are 27× faster than X-CUBE-AI". It is: for continuous sliding-window sensing, we are 27× cheaper per decision, and a stateless runtime has no path to this at all.

4 · Per-window latency — a loss, 2.89×

If your decisions are not on a sliding window, the previous section does not apply and this one does. On identical arithmetic — the same 817,152 multiply-accumulates, the same emulator, the same cycle table — our convolution is slower than ARM's.

CMSIS-NN gets there by restructuring the whole convolution — im2col into one large GEMM across all output channels — and paying for a scratch arena to do it. It needs 16 KiB of scratch (24,048 B bss against our 6,192 B). We tried three SIMD variants of our own inner loop and all three were slower than the scalar code, because with a kernel width of 7 the call overhead per output channel exceeds the seven multiply-accumulates it performs. We reverted to the scalar loop and left the measurements in the source so nobody rewires it blind.

So the honest summary of axes 3 and 4 together: on a sliding window we are far cheaper per decision; on a one-shot window we are nearly three times slower. Which one you are is the question that decides the engine.

5 · What we have not measured

Listed with the same weight as the wins, because a benchmark that hides its gaps is marketing.

How to reproduce this

Every figure above is read from a result artifact, not typed from memory:

• Accuracy — fair_bench_leakfree_result.json
• Footprint — atome_tcn.json + baseline_tflite_micro.json
• Ternary variant — ternary32_export.json
• Streaming — streaming_asymmetry.json
• Per-window latency — cmsis_nn_vs_ours_cortexm4.json
• C99 engine on the full test set — c99_full_test_results.json

The engine, the training path and the export format are Apache-2.0 on GitHub. The benchmark harness for the EDGE line is part of a pilot engagement rather than the public repository; if you want to run it against your own data, say so and we will send it.

Which one should you use

A straight answer, given the table above.

• Use TensorFlow Lite for Microcontrollers if you need to swap models without recompiling firmware, if your part has room and you value the ecosystem, or if your inference is one-shot and latency-critical.
• Use Atome LM EDGE if flash is the binding constraint, if you are classifying a continuously sliding sensor window, or if you need to prove that what ran on the desk is bit-for-bit what runs on the device.
• Accuracy is not the deciding factor on this task. The two are within noise of each other, and anyone telling you otherwise about a 6,567-parameter model on 2,947 windows is reading a single seed.

Questions

Is Atome LM more accurate than TensorFlow Lite for Microcontrollers?

No. On UCI HAR with a leak-free subject-wise split and 5 seeds it is a tie: 0.9095 ± 0.0077 for Atome LM EDGE against 0.9097 ± 0.0154 for TFLite Micro. The paired difference of -0.000204 is far inside the 2·SE noise band of 0.017222.

How much smaller is Atome LM EDGE than TFLite Micro?

4.64× on total flash for the same 6,567-parameter network: 15,444 bytes against 71,712. Runtime RAM is 2.17× smaller, 3,072 bytes against 6,656. The ternary variant reaches 4,765 bytes of weights.

Is Atome LM faster than TFLite Micro?

It depends entirely on whether your window slides. Per decision on an overlapping sliding window, Atome LM EDGE is 27.3× cheaper than CMSIS-NN, the kernel library X-CUBE-AI generates. Per one-shot window it is 2.89× slower. At stride 128 the streaming advantage inverts and becomes a 1.6× penalty.

Are these numbers measured on real silicon?

No. The cycle counts are Unicorn 2.1.4 emulation against the ARM DDI 0403E cycle table, and 61% of executed instructions are not in that table and default to one cycle, so absolute cycle counts are lower bounds. Both arms run on the same harness, so the ratios are the trustworthy part. Accuracy figures are not emulated — they come from the real C99 engine on all 2,947 held-out windows.

Has X-CUBE-AI been benchmarked?

Not the tool itself. ST's stm32ai requires an account we do not have. What is measured is CMSIS-NN v4.1.0, the kernel library X-CUBE-AI generates underneath, which is a lower bound on the tool's cost. Any X-CUBE-AI figure quoted elsewhere on the internet for this comparison is published, not measured by us.

What dataset is this?

UCI HAR — human activity recognition from a waist-mounted smartphone, 6 classes, 2,947 test windows from 30 subjects who never appear in training. One dataset is one dataset; nothing here generalises to your task without running it on your task.