Back to directory
Coding · 2026-10-02

Qwen 3.8 27B on a MacBook Pro M4 Max 36GB: context scaling, TTFT and decode through Ollama

Context-scaling benchmark of Qwen 3.8 27B in NVFP4/MLX through Ollama on a 36 GB M4 Max for agentic coding, measuring TTFT, prefill and decode from 4K to 115K.

Tested Qwen 3.8 27B (NVFP4/MLX, 128K configured context, medium thinking) through Ollama 0.34.4 on a MacBook Pro M4 Max with 36 GB unified memory, for agentic coding. The point of the test was not "40 tok/s" but how long you actually wait before generation at real coding contexts.

Settings

OLLAMA_FLASH_ATTENTION=1
OLLAMA_KV_CACHE_TYPE=q8_0
OLLAMA_NUM_PARALLEL=1
OLLAMA_KEEP_ALIVE=5m

MLX vs GGUF

Model formatDecode
Q4_K_M / GGUF~10.7 tok/s
NVFP4 / MLX~40-41 tok/s

A ~3.8x difference in decode throughput; the author waited around 18 minutes for one non-MLX landing-page task.

Context scaling (active input)

Active inputTime before generationPromptDecode
~4K23.6 s198 tok/s46.6 tok/s
~8K46.8 s178 tok/s41.4 tok/s
~16K1m 30s176 tok/s30.8 tok/s
~30K3m 15s157 tok/s31.9 tok/s

Stress test with retrieval markers

ContextIntegrityTime before generationPromptDecode
~32K4/4 PASS3m 29s152 tok/s29 tok/s
~64K4/4 PASS7m 55s134 tok/s24 tok/s
~96K4/4 PASS14m 55s107 tok/s20 tok/s
~115K4/4 PASS21m 01s87 tok/s14 tok/s

All four retrieval markers still passed at ~115K, so the runtime really does handle the long context; it is just not pleasant to use. Memory pressure rose too: by 64K available memory fell from 89% to 27% with swap rising 529 MB to 2.5 GB, and at 96K to 24% available with 2.9 GB swap.

Takeaway

Decode speed alone is misleading: prompt processing, time to first token and decode are three separate numbers, and memory pressure matters too.

VERIFIABLE SOURCE

Reported anonymously by an r/LocalLLM contributor · 98 upvotes at capture

View source
SETUP HISTORY

Snapshots over time

3 versions
CURRENT2026-10-02Qwen 3.8 27B on a MacBook Pro M4 Max 36GB: context scaling, TTFT and decode through Ollama

Context-scaling benchmark of Qwen 3.8 27B in NVFP4/MLX through Ollama on a 36 GB M4 Max for agentic coding, measuring TTFT, prefill and decode from 4K to 115K.

Coding1 machineQwen 3.8 27BOllama
ChangedDescription
Compare with revision 2
DescriptionPreviousTested Qwen 3.8 27B (NVFP4/MLX, 128K configured context) through Ollama 0.34.4 on a MacBook Pro M4 Max with 36 GB unified memory, for agentic coding. ## Numbers - Warm decode: **~40-41 tok/s** with TTFT near **3.5 s**, prefill ~180-190 tok/s at small context - At ~32K active input: **3m 29s** before generation - At ~115K: **21m 01s**, decode down to **14 tok/s**, though 4/4 retrieval markers still passed - The Q4_K_M/GGUF build managed only **~10.7 tok/s** against ~40-41 for NVFP4/MLX (a 3.8x gap) - At 96K, memory fell to 24% available with 2.9 GB swap ## Limits One machine, one request at a time (`OLLAMA_NUM_PARALLEL=1`); large contexts stayed functional but were not pleasant to use.This snapshotTested Qwen 3.8 27B (NVFP4/MLX, 128K configured context, medium thinking) through **Ollama 0.34.4** on a MacBook Pro M4 Max with 36 GB unified memory, for agentic coding. The point of the test was not "40 tok/s" but how long you actually wait before generation at real coding contexts. ## Settings ```shell OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 OLLAMA_NUM_PARALLEL=1 OLLAMA_KEEP_ALIVE=5m ``` ## MLX vs GGUF | Model format | Decode | | --- | --- | | Q4_K_M / GGUF | ~10.7 tok/s | | NVFP4 / MLX | ~40-41 tok/s | A ~3.8x difference in decode throughput; the author waited around 18 minutes for one non-MLX landing-page task. ## Context scaling (active input) | Active input | Time before generation | Prompt | Decode | | --- | --- | --- | --- | | ~4K | 23.6 s | 198 tok/s | 46.6 tok/s | | ~8K | 46.8 s | 178 tok/s | 41.4 tok/s | | ~16K | 1m 30s | 176 tok/s | 30.8 tok/s | | ~30K | 3m 15s | 157 tok/s | 31.9 tok/s | ## Stress test with retrieval markers | Context | Integrity | Time before generation | Prompt | Decode | | --- | --- | --- | --- | --- | | ~32K | 4/4 PASS | 3m 29s | 152 tok/s | 29 tok/s | | ~64K | 4/4 PASS | 7m 55s | 134 tok/s | 24 tok/s | | ~96K | 4/4 PASS | 14m 55s | 107 tok/s | 20 tok/s | | ~115K | 4/4 PASS | 21m 01s | 87 tok/s | 14 tok/s | All four retrieval markers still passed at ~115K, so the runtime really does handle the long context; it is just not pleasant to use. Memory pressure rose too: by 64K available memory fell from 89% to 27% with swap rising 529 MB to 2.5 GB, and at 96K to 24% available with 2.9 GB swap. ## Takeaway Decode speed alone is misleading: prompt processing, time to first token and decode are three separate numbers, and memory pressure matters too.

This is the currently published snapshot.

REVISION 22026-10-02Qwen 3.8 27B on a MacBook Pro M4 Max 36GB: context scaling, TTFT and decode through Ollama

Context-scaling benchmark of Qwen 3.8 27B in NVFP4/MLX through Ollama on a 36 GB M4 Max for agentic coding, measuring TTFT, prefill and decode from 4K to 115K.

Coding1 machineQwen 3.8 27BOllama
ChangedDescription
Compare with revision 1
DescriptionPreviousTested Qwen 3.8 27B (NVFP4/MLX, 128K configured context) through Ollama 0.34.4 on a MacBook Pro M4 Max with 36 GB unified memory for agentic coding. Warm decode was about 40-41 tok/s with TTFT near 3.5 s and prefill around 180-190 tok/s at small context, but at ~32K active input the wait before generation rose to 3m 29s, and at ~115K it reached 21m 01s with decode down to 14 tok/s, even though 4/4 retrieval markers still passed. The Q4_K_M/GGUF build managed only ~10.7 tok/s against ~40-41 for NVFP4/MLX (a 3.8x gap), and at 96K memory fell to 24% available with 2.9 GB swap. Limits: one machine, one request at a time (OLLAMA_NUM_PARALLEL=1), and large contexts stayed functional but were not pleasant to use.This snapshotTested Qwen 3.8 27B (NVFP4/MLX, 128K configured context) through Ollama 0.34.4 on a MacBook Pro M4 Max with 36 GB unified memory, for agentic coding. ## Numbers - Warm decode: **~40-41 tok/s** with TTFT near **3.5 s**, prefill ~180-190 tok/s at small context - At ~32K active input: **3m 29s** before generation - At ~115K: **21m 01s**, decode down to **14 tok/s**, though 4/4 retrieval markers still passed - The Q4_K_M/GGUF build managed only **~10.7 tok/s** against ~40-41 for NVFP4/MLX (a 3.8x gap) - At 96K, memory fell to 24% available with 2.9 GB swap ## Limits One machine, one request at a time (`OLLAMA_NUM_PARALLEL=1`); large contexts stayed functional but were not pleasant to use.

Archived published snapshot — kept for context and comparison.

REVISION 12026-10-02Qwen 3.8 27B on a MacBook Pro M4 Max 36GB: context scaling, TTFT and decode through Ollama

Context-scaling benchmark of Qwen 3.8 27B in NVFP4/MLX through Ollama on a 36 GB M4 Max for agentic coding, measuring TTFT, prefill and decode from 4K to 115K.

Coding1 machineQwen 3.8 27BOllama

Archived published snapshot — kept for context and comparison.

Discussion

0 comments

No comments yet. Questions and corrections on this setup are welcome.

Leave a comment

Comments are reviewed before they appear. Keep it civil, cite your sources, and stay on topic.