How We Ran 10x Million-Token Context AI Agents in Parallel on a Single Exxact Valence Workstation on GB300

We recently benchmarked 256 concurrent AI agents on an Exxact Valence Workstation powered by NVIDIA DGX Station GB300 and came away with three conclusions: reliability doesn't degrade under load, the real ceiling is context rather than agent count, and agent capacity follows a clean formula: KV cache pool divided by context per session.

In that last benchmark blog, we ran only 1 model: Qwen 3.5 284B 40K context so we decided to run a different model: DeepSeek-V4-Flash-0731.

TLDR; we found some interesting findings. Multi agentic AI reliability finding held and got stronger, but our original KV cache calculation was off. We assumed Exxact Valence with GB300 had a fixed amount of KV cache but based on each model, that number is dynamic.

We got a single Exxact Valence Workstation built on the NVIDIA DGX Station to serves ten simultaneous million-token sessions.

Benchmark Setup: DeepSeek-V4-Flash-0731 with vLLM on Exxact Valence with GB300

We picked DeepSeek-V4-Flash-0731 because it differs from Qwen3-235B-A22B across the factors most likely to affect capacity and throughput:

vllm serve /model \
  --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 \
  --tensor-parallel-size 1 \
  --max-model-len 40960 \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8_e4m3 \
  --tool-call-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --trust-remote-code

We used the same tests we ran: 17 structures tests across six categories, scoring agent behavior instead of raw tokens per second speed. Theses evaluate these three things:

• Reliability & Throughput as we increase in number of concurrent Agents
• Context Length vs Number of Agents for DGX Station
• Sustained Load for DGX Station verifying power and heat

How Model Behavior Changes the Prefill/Generation Mix

Agentic workloads repeatedly submit conversation history and tool results, but that does not produce one universal prefill-to-generation ratio. The mix also depends on how much output the model generates and how quickly each turn completes.

At 128 agents, Qwen3 produced a nearly even 52/48 prefill-to-generation split, while DeepSeek operated at 81/19. Their generation throughput was similar—2,184 versus 2,136 tok/s—but DeepSeek reached 8,885 prefill tok/s compared with Qwen3's 2,360 tok/s. DeepSeek's shorter responses completed turns faster, causing conversation context to be resubmitted more frequently.

Takeaway: the prefill/generation split is shaped by model verbosity and turn behavior, not simply by whether a workload is agentic. A verbose reasoning model can appear balanced, while a terse model can be heavily prefill-weighted. In both tests, real agents still delivered less throughput than matched synthetic workloads—72% for Qwen3 and 59% for DeepSeek—so synthetic tokens-per-second should not be treated as a direct measure of agent capacity.

In our 256-agent DGX Station benchmark, Qwen3's near-50/50 split reflected that model and workload configuration. The DeepSeek result shows why the ratio should be measured for each deployment rather than reused as a general assumption.

Dense and Sparse Attention Require Different Capacity Models

KV cache pool ÷ context tokens per session = estimated concurrent sessions is a common estimate for concurrent-session capacity. However, this estimate is only accurate for dense-attention models, where each session's full context remains resident in the KV cache.

Sparse or sliding-window attention, which DeepSeek V4 uses, changes the relationship because the backend may not retain every context token at the same data storage cost in the KV pool.

For Qwen3, 48 concurrent sessions at 40k token context window would require roughly 1.44 million KV-cache tokens against a 997,753-token pool. Its throughput peaked at 24 sessions, then dropped sharply at 32 as the scheduler crossed the dense-attention capacity boundary.

DeepSeek continued scaling through 48 sessions and reached 345.7 output tok/s. Its sparse sliding-window backend does not keep the entire 40k-token context resident at the same per-token cost, allowing the system to support at least 45% more sessions than the simple formula predicted.

Takeaway: use pool ÷ context per session for dense-attention models. For sparse or windowed-attention models, treat it as a conservative starting point and validate capacity empirically.

Ten Concurrent Million-Token AI Agents on One Workstation

A single Exxact Valence Workstation built on the NVIDIA DGX Station GB300 can hold an estimated ten concurrent AI agent sessions, each configured with more than one million tokens of context. That is enough context for every agent to keep an entire codebase, document corpus, or other large working set live at the same time—on a system that fits under a desk and plugs into a wall socket.

This capability appeared when we configured DeepSeek-V4-Flash-0731 for its native 1,048,576-token context window. vLLM reported an 11,086,409-token KV-cache pool, equivalent to 10.6 full-context sessions:

Increasing --max-model-len from 40,960 to 1,048,576 expanded the reported KV pool by roughly 11×, from 997,753 to 11,086,409 tokens. Although estimated full-context concurrency declined from 24.4 to 10.6 sessions as each session became larger, the system’s total addressable live context grew from roughly one million to 11.1 million tokens.

DeepSeek’s sparse sliding-window attention is the deciding factor. Unlike dense models, it does not retain every context token at the same KV-cache cost, so wider context windows can expand usable capacity. Although DeepSeek and Qwen3 had similar KV memory at matched settings, session capacity varied substantially by attention architecture and context configuration.

Takeaway: one Exxact Valence with GB300 can provide the KV-cache capacity for approximately ten simultaneous million-token AI agent sessions when configured with DeepSeek-V4-Flash-0731. The 10.6-session figure is a vLLM capacity estimate, not a measured ten-agent performance result, so production sizing should still be validated with the exact model, context length, and workload you plan to deploy.

Reproduce our Benchmark

Running an Agentic AI workload is very different from standard LLM benchmarks, so we encourage you to run this same benchmark test on your own! Because agents repeatedly resend their conversation history and tool outputs, their real bottleneck is often context processing, not raw output speed. You can your results with ours too.

The full test suite is open source, and reproducing a comparable number takes four steps.

1. Serve the model. Full flag-by-flag explanation in the vLLM backend guide:

docker run -d --name bench --gpus all --ipc=host --network host \
  -v /path/to/deepseek-v4-flash-0731:/model \
  nvcr.io/nvidia/vllm:26.06-py3 \
  vllm serve /model --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 \
    --tensor-parallel-size 1 --max-model-len 40960 \
    --gpu-memory-utilization 0.90 --kv-cache-dtype fp8_e4m3 \
    --tool-call-parser deepseek_v4 --enable-auto-tool-choice --trust-remote-code

2. Run the agent sweep using the concurrency runner:

export BENCH_BASE_URL=http://127.0.0.1:8000
python3 scripts/swarm.py --model deepseek-ai/DeepSeek-V4-Flash-0731 \
  --sweep 1,16,64,128 --skip-hop

Conclusion

The headline result is simple: one Exxact Valence Workstation with NVIDIA GB300 can provide enough KV-cache capacity for approximately ten concurrent AI agents, each configured with one million tokens of context. At that setting, vLLM reported 11.1 million tokens of live-context capacity—putting codebases, document collections, and other large working sets within reach of multiple local agents at once.

Our second benchmark also reinforced an important sizing lesson: agent capacity is determined by the complete deployment, not the GPU alone. Reliability remained stable across two architecturally different models, but throughput, KV-cache behavior, and session capacity changed with the model, attention architecture, and context configuration. The ten-agent result is therefore a strong capacity estimate for this DeepSeek configuration—not a universal promise for every workload.

We ran these tests on an Exxact Valence built on NVIDIA DGX Station with one GB300 Grace Blackwell Ultra, 252 GB of HBM3e, and 496 GB of LPDDR5X. When sizing your system, start with the model, context length, and required concurrency—then validate the configuration against your workload.

Configure an Exxact system for Agentic AI, like the Exxact Valence with GB300 tested, or talk to our engineers about sizing a system for your AI agents. These system is extremely popular so temper expectations on lead time and availability!

Both models were benchmarked on a single Exxact Valence Workstation built on NVIDIA DGX Station GB300 using our open-source suite at Exxact-Software/local-agent-benchmark. The concurrency runner, the scoring methodology, the serving configurations for both models, and the full result set are all published, so you can reproduce these numbers, or find where they don't hold, on your own hardware.