Ternary-Bonsai-2-27B: A 27B Model in 6GB
A 27B model squeezed into 5.95GB at 1.72 bits per weight, claiming 98.2% of full-precision performance on its own benchmark suite - and what that compression actually costs.
Quantization
A 27B model squeezed into 5.95GB at 1.72 bits per weight, claiming 98.2% of full-precision performance on its own benchmark suite - and what that compression actually costs.