VRAM calculator | NeuronPlant

NP-VRAM/GPU memory

How much VRAM does this context actually take?

Weights, KV cache, and concurrency, from the model's published layer and KV-head counts. A planning estimate for a NeuronPlant delivery. Not an exact runtime measurement.

Mode

Model

36 layers · 32 Q heads · 8 KV heads · head 128 · 8.2B

Weight precision

KV cache precision

Hardware

96 GB GDDR7 ECC · 600 W. Workstation Edition. 1,792 GB/s. Published MIG: 4×24 GB, 2×48 GB, or 1×96 GB. Slices do not share weights.

Context length

Concurrent sequences

Safety margin

Estimated VRAM requirement

Qwen3 8B · AWQ 4-bit

Planning estimate. Not a benchmark and not a deployment guarantee.

Model weights3.8 GiB

KV cache1.1 GiB

Runtime / CUDA1.8 GiB

Safety reserve1.0 GiB

Estimated minimum: 6.7 GiB
Recommended: 7.8 GiB
Memory class: 16 GB
Concurrent sequences: 1
Requested context: 8K
Model maximum context: 128K

RTX PRO 6000 Blackwell Workstation

Works

Concurrent sequences share one weight load. A replica loads the weights again. This job is 12 copies per GPU.

Published MIG slices hold 4 replicas. A slice cannot borrow memory from the next slice.

AWQ 4-bit weight memory is parameters × bytes. AWQ, GPTQ, and INT packs also store scales and zero-points, so a live process can sit above this floor. No measured checkpoint size is stored for this pair yet.

Runtime allowance is 1.5 GiB + 8% of weight memory + 0.2 GiB per extra sequence. vLLM, CUDA graphs, paged attention, tensor parallel, and fragmentation move the real number. Memory class is a bin, not a GPU recommendation.

KV bytes/token 147,456 · https://huggingface.co/Qwen/Qwen3-8B

NP / Intake

Tell us what you need to achieve with AI.

One accountable delivery partner from AI requirement to a delivered factory: site, power, cooling, compute, fabric, structured cabling & passive infrastructure, storage, platform, procurement, and handover. You do not need a vendor list or a hall name first. The hall is a supplier.

Start a Project →