vram.wiki
My space Add your setup
Back to directory
Business automation · 2026-10-17

ISP NOC automation on two vLLM servers with 4× RTX 3090 each

A small ISP (~7,600 customers) serves its agentic operations work on two identical machines, each with 4× RTX 3090 (250W-capped) and vLLM v0.30.0; llama-benchy measures ~5993 tok/s prefill and ~199 tok/s generation.

A small ISP serving about 7,600 customers runs two identical machines to serve its local agentic work. They are wired into the operations stack: VPP and MikroTik routers, RADIUS, client billing, DPDK metrics, LibreNMS reporting and more.

Hardware

Two identical machines, each:

  • CPU: AMD EPYC 7642, 48 cores / 96 threads
  • RAM: 512GB DDR4 ECC 2999 MT/s
  • GPU: 4× RTX 3090 on PCIe Gen4 x16, with a P2P driver mod
  • Power: each GPU capped at 250W

Stack

  • Runtime: vLLM v0.30.0
  • Model: Swift-1.5-Qwen3.8-Flash-Next-W4A16

Benchmarks (llama-benchy, --no-cache --exact-tg --concurrency 1)

  • Prefill (--pp 32768 --tg 1): 5992.84 ± 16.34 tok/s
  • Generation (--pp 1 --tg 2048): 199.00 ± 11.43 tok/s
  • Mixed (--pp 32768 --tg 2048): prefill 6016.62 ± 25.43, generation 191.00 ± 12.68 tok/s

The team reports that local replaced the cloud for this work.

VERIFIABLE SOURCE

Reported anonymously by an r/LocalLLM contributor · 0 upvotes at capture

View source

Correct or update this entry

Is this your setup? Claim it to keep the snapshot current

SETUP HISTORY

Snapshots over time

2 versions
CURRENT2026-10-17ISP NOC automation on two vLLM servers with 4× RTX 3090 each

A small ISP (~7,600 customers) serves its agentic operations work on two identical machines, each with 4× RTX 3090 (250W-capped) and vLLM v0.30.0; llama-benchy measures ~5993 tok/s prefill and ~199 tok/s generation.

Business automation2 machinesQwen 3.8 Flash NextvLLM
ChangedTitleDescriptionHardwareRuntimeTools / workflow
Compare with revision 1
TitlePreviousISP NOC automation on two 4×RTX 3090 EPYC serversThis snapshotISP NOC automation on two vLLM servers with 4× RTX 3090 each
DescriptionPreviousAn internet service provider runs **two identical servers** to automate network operations with a local model. The work covers VPP and MikroTik routers, RADIUS, LibreNMS reporting, client tickets and network analysis, plus billing and reporting around them. ## Hardware Two identical servers: - **CPU:** AMD EPYC 7642 - **RAM:** 512GB DDR4-2999 - **GPU:** 4× RTX 3090, 96GB VRAM total, each on PCIe Gen4 x16 - **Storage:** 2× 3.2TB enterprise PCIe Gen3 NVMe ## Model and stack - **Model:** Qwen 3.8 Flash Next - **Harness:** custom, in-house - **Context:** 262144 tokens ## Reported performance - Prefill: **5500+ tok/s** - Generation: **150+ tok/s** Local replaced the cloud for this work, according to the submitter.This snapshotA small ISP serving about 7,600 customers runs **two identical machines** to serve its local agentic work. They are wired into the operations stack: VPP and MikroTik routers, RADIUS, client billing, DPDK metrics, LibreNMS reporting and more. ## Hardware Two identical machines, each: - **CPU:** AMD EPYC 7642, 48 cores / 96 threads - **RAM:** 512GB DDR4 ECC 2999 MT/s - **GPU:** 4× RTX 3090 on PCIe Gen4 x16, with a P2P driver mod - **Power:** each GPU capped at 250W ## Stack - **Runtime:** vLLM v0.30.0 - **Model:** Swift-1.5-Qwen3.8-Flash-Next-W4A16 ## Benchmarks (llama-benchy, `--no-cache --exact-tg --concurrency 1`) - **Prefill** (`--pp 32768 --tg 1`): **5992.84 ± 16.34 tok/s** - **Generation** (`--pp 1 --tg 2048`): **199.00 ± 11.43 tok/s** - **Mixed** (`--pp 32768 --tg 2048`): prefill **6016.62 ± 25.43**, generation **191.00 ± 12.68 tok/s** The team reports that local replaced the cloud for this work.
RuntimePreviousNot specifiedThis snapshotvLLM
Tools / workflowPreviousCustom, in-house harnessThis snapshotNot specified

This is the currently published snapshot.

REVISION 12026-10-17ISP NOC automation on two 4×RTX 3090 EPYC servers

An ISP network team automates routers, RADIUS, monitoring, billing and tickets with Qwen 3.8 Flash Next on two identical EPYC servers, each with 4× RTX 3090, and reports that local replaced the cloud entirely.

Business automation2 machinesQwen 3.8 Flash NextRuntime unspecified

Archived published snapshot — kept for context and comparison.

Read next

DocumentsGrading 500 student exams with a local modelBusiness automationCoaching elite athletes with a 2x DGX Spark clusterBusiness automationPR review and support triage on two budget RTX 3060s

Discussion

0 comments

No comments yet. Questions and corrections on this setup are welcome.

Leave a comment

Comments are reviewed before they appear. Keep it civil, cite your sources, and stay on topic.