256 Concurrent AI Agents on a Single Exxact Valence Workstation with GB300

Most local LLM benchmarks report tokens per second. That number answers how fast a system generates text. It doesn't answer the question for scaling/sizing hardware for an agentic AI fleet: How large of model + how many agents of that model can you run on a single workstation based on the NVIDIA DGX Station GB300?

TLDR: We ran 256 concurrent AI agents on a single desk-side Exxact Valence Workstation with GB300 based on NVIDIA DGX Station. We observed no slowdown; the 256th agent was as reliable as the first and we could’ve pushed it further, but the limit wasn’t the number of concurrent, it was something else: context length.

Benchmark Setup: Exxact Valence Workstation with GB300 on 256x Qwen3 235B-A22B

The NVIDIA DGX Station GB300 platform is essentially the NVIDIA DGX Spark imagined as a extreme desktop workstation. It features an NVIDIA Grace 72-core CPU with 496GB LPDDR5 paired with an NVIDIA GB300 with 252GB HBM3e memory, totaling to 748GB of coherent memory similar to the DGX Spark.

We used is the Qwen3-235B-A22B NVFP4 quantization, served on vLLM 0.13.0 through NVIDIA's NGC container. At load, vLLM reported a GPU KV cache size of 1,034,736 tokens. That figure is the basis for every capacity number later in this post.

vllm serve /model \
  --served-model-name nvidia/Qwen3-235B-A22B-NVFP4 \
  --tensor-parallel-size 1 \
  --max-model-len 40960 \
  --gpu-memory-utilization 0.90 \
  --quantization modelopt_fp4 \
  --kv-cache-dtype fp8_e4m3 \
  --tool-call-parser hermes \
  --enable-auto-tool-choice \
  --trust-remote-code

The agent workload came from our own local-agent-benchmark suite, the same one used in the DGX Spark agent benchmark. It runs 17 structured tests across six categories, scoring agent behavior rather than raw generation speed. Each agent runs all 17 tests as an independent process with its own conversation and tool state.

In our benchmark for the Exxact Valence Workstation with GB300 we tested these 3 things:

• Reliability & Throughput as we increase in number of concurrent Agents
• Context Length vs Number of Agents for DGX Station
• Sustained Load for DGX Station verifying power draw and heat

How 256 AI Agents Performs on Exxact Valence with GB300 (NVIDIA DGX Station)

First, we verify whether or not an agentic AI fleet of agents degrades as concurrency increases on the Exxact Valence with GB300. At 128 and 256 concurrent agents, the real agent workload delivered roughly 70% of matched synthetic throughput, showing that agent execution scales predictably.

Note most synthetic LLM benchmarks don’t reflect the agentic AI workflows. Real agents are not single-turn completions. They maintain state, call tools, read tool results, recover from errors, and repeat that loop until the task is done. A tokens-per-second benchmark captures onyl part of that picture.

With that being said, the Exxact Valence with GB300 is capable of massively high throughput, delivering over 3,400 tok/s on 256 concurrent AI models. We then ran the full test suite at increasing agent counts, from 1 up to 256, each agent running independently against the same server to see if there is degradation of AI quality as we increase the number of agents.

Amazing! Pass rate stays in a tight band. Across 256 agents in concurrency, the pass rate never leaves a 0.824–0.897 range. Zero agents failed at any concurrency tested, and throughput scales faster than latency degrades.

But this is not news to us here at Exxact. The limiting factor is not the number of parallel agents we can theoretically put on a single DGX Station… It’s the number of agents plus the context length each agent can have.

Qwen3-235B has a maximum context length of 40,960 tokens (exceeding this returns an HTTP 400 context-length error). Let’s review the math: with a KV cache of 1M tokens and 40K tokens of context, we max out at about 24 concurrent sessions; somewhere between 24, and 32 we had a sharp decline. Any additional agents introduced into our system at a 40K-token context length lose efficiency and plateau at about 115 tok/s.

To size the number of Agents an NVIDIA DGX Station can fit, context length is the metric we need to define:

• KV Cache in Tokens = Context Length per Agent in Tokens × Number of Agents
• Short-context agents (triage/routing/tool dispatch) can scale into the hundreds. (5-10K context)
• Long-context agents (coding/large working sets) hit a ceiling in the low dozens. (30-40K context)

That’s ~10x swing in concurrency on the same hardware, driven primarily by session size. This is because 256 agentic AI models in parallel, GPU work is 50:50 processing input and producing output, as opposed to common LLM benchmarks that have an 80:20 generation to prefill. Agentic AI resubmits its context at every turn, so the context length is much higher for multiple concurrent (or even singular) agentic AIs.

Testing Exxact Valence with GB300 (NVIDIA DGX Station) on Sustained Load

Our next tests we verify thermals validation and check for any throttle flags. Since the NVIDIA DGX Station GB300 is configured to sit in the office instead of the data center, we ran 64 agents continuously for 65 minutes, six full passes of the test suite. Over the timeframe, we observed no slow downs or any performance drops.

On thermals, we tracked SM clock speed and throttle flags directly. Across 2,269 samples over the full hour:

• SM clock: Held at a flat 2,070 MHz minimum and maximum, against a 2,070 MHz cap. This did not budge. Great!
• Throttle flags set: Had zero alerts. Great!
• Temperature: 57°C peak, 54.5°C average, plateauing. Very manageable temperatures.
• Power: 788.6W average under sustained load, 855.4W peak. Idle draw is 213W. Surprisingly low power draw for a system of this caliber.
• Mean time-to-first-token across the full hour was 74.3ms and stayed flat:
• Zero requests spent in the queue over the full 65 minutes.

Agents per megawatt or AA-AgentPerf efficiency is becoming a common way to compare agent-serving hardware, using agents per megawatt (number of simultaneous agents sustained per megawatt at a given latency). Using that same math: this is about 81,000 agents per megawatt at a 74.3ms mean time-to-first token (64 agents at 788.6W sustained)

Reproduce our Benchmark

Running an Agentic AI workload is very different from standard LLM benchmarks, so we encourage you to run this same benchmark test on your own! Because agents repeatedly resend their conversation history and tool outputs, their real bottleneck is often context processing, not raw output speed.

The full test suite is open source, and reproducing a comparable number takes four steps.

1. Serve the model. Full flag-by-flag explanation in the vLLM backend guide:

docker run -d --name bench --gpus all --ipc=host --network host \
  -v /path/to/qwen3-235b-a22b-nvfp4:/model \
  nvcr.io/nvidia/vllm:26.01-py3 \
  vllm serve /model --served-model-name nvidia/Qwen3-235B-A22B-NVFP4 \
    --tensor-parallel-size 1 --max-model-len 40960 \
    --gpu-memory-utilization 0.90 --quantization modelopt_fp4 \
    --kv-cache-dtype fp8_e4m3 --tool-call-parser hermes \
    --enable-auto-tool-choice --trust-remote-code

2. Note the KV pool size that vLLM prints at load. This run reported 1,034,736 tokens, and every capacity figure in this post divides that number by context-per-session.

3. Run the agent sweep using the concurrency runner:

git clone https://github.com/Exxact-Software/local-agent-benchmark
cd local-agent-benchmark
export BENCH_BASE_URL=http://127.0.0.1:8000
python3 scripts/swarm.py --model nvidia/Qwen3-235B-A22B-NVFP4 \
  --sweep 1,4,8,16,32,64,128,256 --skip-hop

4. Compare results against the published figures in docs/results/dgx-station-gb300.md in the repository. The scoring methodology is documented there too.

Before trusting a result, confirm the tool-call parser is set correctly by running a single agent first, since a mismatched parser can make a working model look broken. Also check that results.json reports suite_version: 1.1.0, since scores from earlier scorer versions (before the reasoning-token fix) aren't comparable, as documented in the changelog.

Frequently Asked Questions on Exxact Valence with GB300 (NVIDIA DGX Station)

What limits how many agents I can run?

Model size plays a big role in consuming VRAM, but context length per agent is the true limiting factor. The NVIDIA DGX Station can run extremely large models or man models; agent count and reliability held up far past what most fleets need, but the KV cache pool and the model's context window are what run out first.

What happens if an agent's context exceeds the model's context window?

It fails immediately. If the prompt & tool output exceed the model’s context window, the request is rejected with an HTTP 400 “context length” error. Design your agents to keep context under the limit—e.g., summarize/compact older turns or store details externally and retrieve only what you need. Reducing context length will degrade quality, relevance, and accuracy, so proceed with caution.

Does running more agents at once reduce accuracy or reliability?

Pass rate stayed within a narrow band from 1 to 256 concurrent agents, and zero agents failed at any concurrency level tested.

Why does a tokens-per-second benchmark overstate what an agent fleet can actually do?

Agents resubmit their full conversation history on every turn, so a large share of GPU work goes to re-reading context (prefill) rather than generating new tokens. Standard synthetic benchmarks assume a much lighter prefill load, which is why agents only achieved about 70% of the synthetic throughput figure at matched concurrency.

Does this NVIDIA DGX Station throttle or overheat under sustained multi-agent load?

Across a 65-minute sustained run at 64 concurrent agents, SM clock speed never dropped from its cap and zero throttle flags were recorded, with temperatures plateauing well under thermal limits.

How many agents should I actually plan for on the NVIDIA DGX Station?

Number of agents depends heavily on session length. Short-session agents like triage, routing, or tool dispatch can scale into the hundreds. Agents that accumulate long context, like coding assistants, will saturate the same hardware at a few dozen concurrent sessions. Size for context per agent first, then check that figure against the KV pool your own server reports.

Final Thoughts

Don't size an agent deployment off a tokens-per-second figure, or the biggest LLM you can fit. Size it off context-per-agent multiplied by the number of agents that will be active at a given time. Check that total against the KV cache pool your server reports at startup.

An Exxact Valence Workstation with GB300 will comfortably hold more concurrent agents than most SMB deployments will need, at interactive latency, without throttling with short-session agentic AI with low context length. For agentic AI deployments that maintain long context, deploy fewer models to maintain high efficiency and quality. Number of Agents + Context Length is the key to planning your agentic AI hardware.

The system we used in this blog is available as the Exxact Valence with GB300 based on NVIDIA DGX Station. You can configure and price one directly, browse the full NVIDIA DGX systems range, or contact our engineers today to get help configuring a system to fit your own workload.

We ran a similar local agentic AI test on the smaller NVIDIA DGX Spark: Benchmarking Local AI Agents on NVIDIA DGX Spark, which could handle nine small models one at a time. We also talked about what workloads get the most from NVIDIA DGX Station.