Embedded Hardware
One bit per parameter is what puts a real model on the part you already ship.
Extraordinary Intelligence Density
At any fixed memory budget, one-bit weights carry more parameters per megabyte than any wider format — the same silicon holds a more capable model.
No floating-point unit
One-bit weights turn the matrix multiply into additions and sign flips, which brings integer-only parts back into scope for work that used to need an accelerator.
Less memory traffic
Most of the energy in inference goes on moving weights rather than on the arithmetic itself. Sixteen times fewer bits to move is the saving that shows up on a battery.
No network in the loop
Inference happens on the part. Nothing leaves the device, nothing waits on a round trip, and nothing stops working when the link does.
Lower bill of materials
Less RAM, less flash and a smaller part carry straight through to unit cost — once across a design, then again on every unit built from it.
Thermal headroom
Lower sustained power keeps sealed enclosures and passive cooling on the table, which is often what decides whether the product can be built at all.
The budget is fixed long before the model arrives
An embedded part is chosen for cost, power and thermals, and its memory is fixed at design time — often for a product life of seven to ten years — with most of it already claimed by the system and the application the device exists to run. The model gets the leftovers. The usual answer is to take a full-precision model and cut it down — prune it, distil it, shrink the vocabulary — until it fits. What fits at the end is a smaller model that has lost most of what made it worth deploying.
Training at one bit per parameter changes which model you are choosing between. The constraint is in from the first step, so the model learns to hold what it knows inside the format — which is why the quality survives where after-the-fact compression loses it. The same architecture, at the same parameter count, needs a sixteenth of the memory and does its weight arithmetic in integers: a model that had to sit in external DRAM behind an accelerator sits in memory already on the board and runs on the core already in the part.
Bandwidth is the speed limit
When a model generates text the processor is mostly waiting: every token requires streaming the model's entire weights out of memory, and it is memory bandwidth, not arithmetic, that sets the pace. Tokens per second is, to a first approximation, bytes moved per token. A one-bit model moves an order of magnitude fewer bytes than a full-precision one — and it speeds up in direct proportion, with the biggest gains on the biggest models.
The same bytes govern the power draw. Moving data out of DRAM costs far more energy than computing on it, so a model that streams a tenth of the traffic spends a corresponding fraction of the energy — before counting that the arithmetic left over is addition rather than multiplication. On a battery that is the difference between a feature that is always on and one that users learn to switch off; in a sealed enclosure it is the difference between sustained performance and throttling.
- MEMORY · WEIGHT FOOTPRINT
- ~10×smaller than FP16 — roughly 1.6 bits stored instead of 16
- SPEED · CPU INFERENCE
- UP TO 6×faster than standard frameworks, measured on commodity x86 CPUs
- ENERGY · PER TOKEN
- UP TO 23×less energy than comparable conventional models
The memory multiplier is format arithmetic: roughly 1.6-bit ternary weights against FP16. CPU speed reaches 6.17× on x86 in published bitnet.cpp benchmarks against standard frameworks; the energy comparison reaches 23× for a 2-billion-parameter ternary model against openly published models of comparable size. Sources: 1-bit AI Infra, arXiv:2410.16144, and the BitNet b1.58 2B4T technical report, arXiv:2504.12285. No accelerator was involved.
The CPU you already ship is enough
Optimised lookup-table kernels make one-bit inference fast on the ordinary cores an embedded product already has: measured speedups of 1.4–5.1× on ARM and up to 6.2× on x86, with energy cut by 55–70% on ARM, against the same models run through standard frameworks. There is no dependency on a neural accelerator, a GPU, or any one vendor's AI stack — and where the part does have an NPU, it stays free for the vision and signal work NPUs are good at, while the language model runs on cores that would otherwise be idle.
That independence compounds across a catalogue. A CPU-side runtime carries the same model across product generations and silicon vendors — the old part with no accelerator, the current one, the next one — so the capability becomes one asset deployed everywhere rather than a porting project per chip, and the model never ties the roadmap to a supplier. Where a design does go to an FPGA or custom silicon, a multiplication-free network is the cheap one to lay down: adders in place of multipliers, more parallel compute in less area at lower power.
The headroom is real: the same kernels have run a hundred-billion-parameter one-bit model on a single CPU at human reading speed.
1 bit vs FP4 vs 16 bits
| Per weight | Full precision | FP4 | One bit |
|---|---|---|---|
| Bits stored | 16 | 4 | 1 |
| 100M parameters | 200 MB | 50 MB | 12.5 MB |
| 1B parameters | 2 GB | 500 MB | 125 MB |
| Weight arithmetic | Multiply–accumulate | Multiply–accumulate | Add and subtract |
| Floating-point unit | Required | Required | Not required |
Weights alone, at sixteen, four and one bit per parameter. FP4 formats carry a shared scale for every block of weights, so four bits is the stored figure rather than the effective one — in practice it lands a little above. The same honesty applies on our side: ternary models, which we also train, pack to roughly 1.6 bits stored — a tenth of full precision rather than a sixteenth — and the arithmetic rows read the same either way. Activations, the runtime and your own application sit on top of every column.
Frontier intelligence on your chips & SOMs.
TALK TO US ↗