Use case
High-performance accelerator training infrastructure.
A tightly coupled training cluster: fabric, storage bandwidth, and cooling sized to the job, not the idle rack. GPU is the common case. TPU and other ASICs when that is the domain.
Architecture
This path is one instance of the factory data pipeline: data to workload, landed on the accelerator plant, then operate. NeuronPlant engineers the cell and the vendor coordination. We do not write every model. We do not construct the hall.
NP-NET / NVIDIA switch planes
East-West · North-South · Management. Families, not SKUs.
East-West
01
NVIDIA Quantum-X / Spectrum-X
RDMA/RoCE or InfiniBand. GPU-to-GPU, DGX-to-DGX, scale-out.
800 Gb/s XDR / 400 Gb/s NDR / 800GbE class
Collectives, NCCL, scale-out between nodes or NVL racks. The model runtime lives here. This is not user traffic and not BMC.
02
Quantum or Spectrum-X
Own data plane when dedicated. Off the compute rails.
400 Gb/s IB or 800GbE
Compute to high-performance storage: datasets, checkpoints, RAG index path. Do not fold it into a generic East-West compute line if dedicated. Do not auto-label it North-South or East-West without that context. Decide with the storage machine, not with leftover ports.
North-South
03
Spectrum-X Ethernet
Users, apps, API gateway/LB, enterprise/core, firewalls.
100G-class typical
Application-facing Ethernet. NFS homes, logs, ingest, people. Must not share queues with training I/O. Not just another rail next to the GPU rails.
Management
04
Dedicated 1/10G / Cat6A
BMC, management, operational control. Own patch field.
1/10G class
Lights-out, firmware, serial, leak, PDU management if that job exists. In-band plant services sit here, not on application North-South. If this dies inside a compute bundle, the hall is dark and you cannot see why.
DWG-NET-01 · laying out
NP-STO / What to look for
Throughput numbers below for training come from the NVIDIA SuperPOD with NVIDIA DGX B300 reference architectures. RAG and NIM notes follow how those stacks actually move bytes, plus the NVIDIA NIM support matrix for profile sizing.
Training / post-training
Sustained sequential GB/s and checkpoint burst, not IOPS marketing.
Shared parallel filesystem for datasets. Separate checkpoint target that can absorb a simultaneous flush from the job. NVIDIA SuperPOD B300 RA states high-performance storage I/O per node must exceed 40 GB/s (Ethernet RA) and, on the Quantum-X800 RA, 80 GB/s.
Typical: HPS on IB (QM9700) or RoCE (SN5600) as the storage/data fabric, plus a quieter NFS/user tier on SN5610 as North-South / front-end. Do not fold HPS into a generic East-West compute line if dedicated.
Avoid: A single enterprise NAS asked to be both home directories and the training scratch.
RAG / enterprise retrieval
Random read IOPS, metadata, and a clean split between documents and the vector index.
Object or file store for source documents. NVMe-backed or in-memory vector index. Ingest path sized to nightly rebuilds, not to training checkpoints.
Typical: Object + vector engine. North-South / front-end Ethernet is usually enough if the index is local or on a low-latency tier. That is not the East-West compute fabric.
Avoid: Putting the vector store on the same parallel FS as a training job.
Inference / NIM serving
Fast model-weight load, local NVMe or a warm cache, isolation from training I/O, and a KV / prefix cache plan on large serving.
NVIDIA NIM selects a profile from its support matrix (precision, tensor-parallel size, LoRA). Weights load from local NVMe. That is not the whole inference memory story. On a large workload, KV cache needs a hierarchy: GPU HBM, host overflow, and sometimes a shared prefix cache across the serving plane. That tier is not a storage array and not a checkpoint pool.
Typical: Node-local NVMe for NIM images and weights. KV in HBM first. Host memory and a serving-plane prefix cache when context and concurrency fill the GPU.
Avoid: Loading 70B-class weights over a contended NFS home share. Treating KV as another LUN on the training filesystem.
Research / HPC mix
Scratch that can be purged, project space that cannot, and a scheduler that knows the difference.
Burst scratch on the HPS fabric. Project datasets on a capacity tier. Home and logs on the user-storage Ethernet path, as in the NVIDIA SuperPOD split.
Typical: Two storage systems: HPS + user storage. NVIDIA documents this split explicitly.
Avoid: One quota, one protocol, every lab on the same queue.
NP-CBL / Fabric planes
East-West · North-South · Management. Cabling builds the plants.
DWG-CBL-01 · laying out
EW / East-West
Hall-internal scale-out. Compute fabric first. Storage/data fabric on its own plane when dedicated. Not user traffic.
01 / East-West Compute Fabric
MPO/MTP trunks, 400/800G optics, rail-aligned
GPU-to-GPU, DGX-to-DGX, scale-out. RDMA/RoCE or InfiniBand. NVIDIA Spectrum-X or Quantum-X as the high-speed EW compute fabric. NCCL, all-reduce. One polarity map. One tray.
02 / Storage / Data Fabric
Separate MPO or LC plant onto the storage leaf when dedicated
Compute to high-performance storage: datasets and checkpoints. Own internal data plane when physically separate. Do not fold it into a generic East-West compute line. Do not auto-label it North-South or East-West without that context. NVIDIA SuperPOD keeps this off the compute rails.
NS / North-South
Users, applications, API gateway/load balancer, enterprise/core, firewalls. Application-facing Ethernet. Not another GPU rail.
03 / North-South / Front-End Fabric
LC or lower-count MPO, 100G-class typical
Users, apps, API gateway/LB, enterprise/core, application-facing Ethernet. Not just another rail next to the GPU rails. Must not share a tray or a cassette with training I/O.
MGMT / Management
BMC, management, operational control. Never classify as application North-South.
04 / Management / OOB Fabric
Cat6A or dedicated 1/10G optics. Own patch field.
BMC, management, operational control. Lights-out, firmware, serial, leak sensors. Never application North-South. If this plant dies inside a compute bundle, you cannot even see why the hall is dark.
Fiber plant, MPO, labeling and test are spec lines, not a fit-out. Structured Cabling & Passive Infrastructure.
Infrastructure
Design considerations
NP / Intake
One accountable delivery partner from AI requirement to a delivered factory: site, power, cooling, compute, fabric, structured cabling & passive infrastructure, storage, platform, procurement, and handover. You do not need a vendor list or a hall name first. The hall is a supplier.