Hyper Efficiency
Sixteen bits of precision cut to one, with the model still doing its job.
Our Research
We research 1-bit AI models
What do we research?
By default, most AI models are trained in full precision. This means each parameter within the model has 16 bits. This works, but we ask the question: does every parameter really need 16 bits?
We research 1-bit AI models, where each parameter is either a 0 or 1*. We also study tenary quantization and other model compression and quantisation techniques.
0.4127−1.08360.05920.7314−0.2648
−0.90210.3376−0.48851.20490.0917
0.6438−0.12500.8763−0.56010.3092
−0.27640.9915−0.73280.1486−1.0472
100111100010100111100000111010000011101001100111011011001111010110011010000011001011111000100001010000000010011111110101110111000100001010011001101100011110000101000110110010101000100000000101000010011000011001001011010001110010110101010111011010101101000111010100010001010001010011010001111010110000010010001110110100111001101110010000110000100001100100101001111001111010101110101111110011100101010101101001010000110010000111101000100111001101101110110101100000001011000100101100
Why?
16x lower RAM use
A model that needed 32GB fits in 2GB — small enough for a phone.
Lightning Fast
Multiplies become additions, so each token takes far less arithmetic.
Ultra low power
Less memory traffic, less energy per token, no data centre required.
Need a 1-bit model?
GET IN TOUCH ↗