vram.wiki
My space Add your setup
Back to directory
Coding · 2026-09-27

Ternary Bonsai 2 27B on an RTX 4090: speed at a quality price

The same project's measured run of a ternary-quantised 27B model: about 92 tokens/s generation with no speculative decoding, from 6.7 GB of weights, at a documented cost in agentic coding ability.

A second measured combination from the same open-source project (awesome-local-ai) puts Ternary Bonsai 2 27B on the same RTX 4090.

What Bonsai 2 is

Qwen 3.8 27B re-quantised to ternary weights (~1.72 bits per weight, 6.7 GB), served by a llama.cpp fork because its GGUF types sit past upstream's supported range.

Measured

  • ~92 tok/s generation with no speculative decoding at all
  • ~3,016 tok/s prefill
  • From 6.7 GB of weights instead of 16.7.

Pointing the Qwen MTP draft head at it was measured a net loss.

The trade, quoted in full

The project's own cited figures put Bonsai 2 at 52.8 on Terminal-Bench 2.1 and 60.8 on SWE-bench Verified, against 69.7 and 80.6 for full-precision Qwen 3.8 27B. The speed is real and so is the gap, which the project quotes in full rather than hiding in an average.

VERIFIABLE SOURCE

Published from an open-source project repository · reviewed before publication

View repository

Correct or update this entry

SETUP HISTORY

Snapshots over time

2 versions
CURRENT2026-09-27Ternary Bonsai 2 27B on an RTX 4090: speed at a quality price

The same project's measured run of a ternary-quantised 27B model: about 92 tokens/s generation with no speculative decoding, from 6.7 GB of weights, at a documented cost in agentic coding ability.

Coding1 machineTernary Bonsai 2 27Bllama.cpp
ChangedDescription
Compare with revision 1
DescriptionPreviousA second measured combination from the same open-source project (awesome-local-ai) puts Ternary Bonsai 2 27B on the same RTX 4090. Bonsai 2 is Qwen 3.8 27B re-quantised to ternary weights (about 1.72 bits per weight, 6.7 GB), served here by a llama.cpp fork because its GGUF types sit past upstream's supported range. The author reports about 92 tokens/s generation with no speculative decoding at all, and roughly 3,016 tokens/s prefill, from 6.7 GB of weights instead of 16.7. This row runs on measurements, not marketing: pointing the Qwen MTP draft head at it was measured a net loss, and the project states the price plainly. Its own cited figures put Bonsai 2 at 52.8 on Terminal-Bench 2.1 and 60.8 on SWE-bench Verified, against 69.7 and 80.6 for full-precision Qwen 3.8 27B. The speed is real and so is the gap, which the project quotes in full rather than hiding in an average.This snapshotA second measured combination from the same open-source project (`awesome-local-ai`) puts Ternary Bonsai 2 27B on the same RTX 4090. ## What Bonsai 2 is Qwen 3.8 27B re-quantised to ternary weights (~1.72 bits per weight, 6.7 GB), served by a llama.cpp fork because its GGUF types sit past upstream's supported range. ## Measured - **~92 tok/s generation** with no speculative decoding at all - **~3,016 tok/s prefill** - From 6.7 GB of weights instead of 16.7. Pointing the Qwen MTP draft head at it was measured a **net loss**. ## The trade, quoted in full The project's own cited figures put Bonsai 2 at 52.8 on Terminal-Bench 2.1 and 60.8 on SWE-bench Verified, against 69.7 and 80.6 for full-precision Qwen 3.8 27B. The speed is real and so is the gap, which the project quotes in full rather than hiding in an average.

This is the currently published snapshot.

REVISION 12026-09-27Ternary Bonsai 2 27B on an RTX 4090: speed at a quality price

The same project's measured run of a ternary-quantised 27B model: about 92 tokens/s generation with no speculative decoding, from 6.7 GB of weights, at a documented cost in agentic coding ability.

Coding1 machineTernary Bonsai 2 27Bllama.cpp

Archived published snapshot — kept for context and comparison.

Other setups on this hardware

Coding · 2026-09-27Qwen 3.8 27B on a single RTX 4090, measured (llama.cpp)
All setups on this hardware

Read next

CodingCoding a SaaS product on an M1 Max, one well-scoped feature at a timeCoding98% local, on a self-extended Pi Agent 'software factory'CodingThe first local model this person actually trusts with real work

Discussion

0 comments

No comments yet. Questions and corrections on this setup are welcome.

Leave a comment

Comments are reviewed before they appear. Keep it civil, cite your sources, and stay on topic.