Size CPU and RAM for a vLLM Deployment

Use vLLM's physical-core floor, explicit memory controls, and startup cache reports to size CPU and RAM before production load testing.

Size CPU and RAM for a vLLM deployment by using the documented worker-to-core floor, recording every CPU-side memory control you will enable, and checking the GPU KV-cache headroom reported at startup. For llm deploy planning, treat physical CPU capacity, system RAM, and GPU memory as separate constraints; changing one budget does not automatically resize the others.

A practical CPU, RAM, and GPU budget for llm deploy

The vLLM optimization guidance gives a minimum of 2 + N physical cores for N GPU workers. The two fixed cores cover the API server and engine core, while the remaining allocation provides one core per GPU worker.

physical-core minimum = 2 + N
minimum vCPUs with hyperthreading = 2 x (2 + N)

Here, N means GPU workers, not simply the number of GPUs advertised by a hardware vendor. When hyperthreading is enabled, the guidance treats one vCPU as one hyperthread, or one-half of a physical core, and therefore gives the second formula above.

Treat these values as minimums, not proof that a host will meet its workload target. Before approving the machine, record:

The actual physical-core count, rather than only the provider's vCPU figure.
The number of GPU workers that will run on the host.
Whether hyperthreading is enabled.
Which cores and memory remain available after the serving process starts.

This creates a hardware floor without inventing a CPU capacity figure that the documentation does not provide.

Build a RAM worksheet instead of inventing a minimum

The cited vLLM documentation does not publish one universal total-RAM minimum. Use a planning worksheet instead, and mark it clearly as a budgeting method rather than an official vLLM formula:

planning RAM budget = measured runtime baseline
                    + enabled CPU offload
                    + applicable CPU caches
                    + host headroom

Account for each configurable component separately:

CPU offload: The vLLM serve CLI reference defines --cpu-offload-gb as the amount of CPU memory used for offloading per GPU, in GiB; its default is 0, meaning no offloading. If you enable it, the same reference warns that the CPU-GPU interconnect must be fast because part of the model moves from CPU memory to GPU memory during each forward pass.
CPU caches: The vLLM memory-conservation guide documents a default 4 GiB CPU-backend KV cache controlled by VLLM_CPU_KVCACHE_SPACE. It also documents a default 4 GiB multimodal processor cache controlled by mm_processor_cache_gb. These apply to different modes; do not automatically add both defaults to every text-only deployment.
Model loading: The memory guide also states that, with tensor parallelism, every process reads the whole checkpoint before splitting it into chunks. Disk-loading time therefore grows with tensor-parallel size. Track this loading path separately from steady-state RAM consumption.

Record the configured per-GPU offload amount, the applicable cache sizes, and the host margin independently. That gives you a RAM worksheet you can update without pretending that an unrelated cache default is a universal host requirement.

Set the serving limits explicitly

The vLLM serve CLI reference defines --max-model-len as the context length covering both prompt and output. If it is unset, vLLM derives the value from the model configuration; -1 or auto selects the largest model length that fits in GPU memory. The same reference defines --max-num-seqs as the maximum sequences processed in one iteration and --max-num-batched-tokens as the maximum tokens processed in one iteration.

--gpu-memory-utilization is the fraction assigned to the model executor, with a documented range from 0 to 1 and a default of 0.92. When --kv-cache-memory-bytes is set, it takes precedence over --gpu-memory-utilization for KV-cache allocation. Choose one allocation policy deliberately rather than changing both while assuming they govern the same memory.

Select devices through CUDA_VISIBLE_DEVICES; the memory-conservation guide directs users to control device selection this way rather than by calling torch device functions before vLLM starts. A starting command template is:

CUDA_VISIBLE_DEVICES=YOUR_VALUE vllm serve YOUR_VALUE \
  --max-model-len YOUR_VALUE \
  --max-num-seqs YOUR_VALUE \
  --max-num-batched-tokens YOUR_VALUE \
  --gpu-memory-utilization YOUR_VALUE \
  --port 8000

Replace each YOUR_VALUE from your model, device plan, workload contract, and memory policy. The explicit context and concurrency values make the starting configuration reviewable; an automatic maximum can change the relationship between accepted work and available KV-cache capacity.

Check the result under representative load

At startup, inspect the GPU KV cache size line and the reported maximum concurrency. The parallelism and scaling guide explains that this line reports how many tokens the GPU KV cache can hold at once. Correlate that result with CPU RAM consumption, GPU memory, and the request shape you actually intend to serve.

The vLLM benchmark guide positions vllm bench serve for feature evaluation and regression testing and recommends GuideLLM for production-server benchmarking. Its documented online example is:

vllm bench serve \
  --backend vllm \
  --model NousResearch/Hermes-3-Llama-3.1-8B \
  --endpoint /v1/completions \
  --dataset-name sharegpt

This is the documented example, not a capacity result for your server. For a custom workload, the guide requires JSONL data with a prompt field and selection through --dataset-name custom:

{"prompt": "YOUR_VALUE"}

The benchmark report includes request throughput and output-token and total-token throughput. In the benchmark metric definitions, TTFT runs from sending a request to receiving its first streamed output; ITL measures gaps between consecutive streamed outputs; and TPOT is calculated once per request, excluding the first token, before aggregation. With speculative decoding, one streamed output may contain several tokens, so ITL and TPOT can diverge.

Do not compare every run as though it were independent. The benchmark documentation warns that repeating vllm bench serve against the same server can reuse prompts in its prefix cache and inflate throughput. It also notes that metric terminology is not standardized across benchmarking tools, so record the tool, configuration, dataset, request limits, and metric definitions alongside every result.

Where sizing usually goes wrong

When KV-cache space runs out, vLLM can preempt requests and recompute them later. The optimization guidance identifies increasing gpu_memory_utilization, or decreasing max_num_seqs or max_num_batched_tokens, as remedies. Treat frequent preemption as a reason to revisit workload limits, not merely as a reason to raise GPU utilization.

Before sign-off, check these common errors:

Applying the physical-core rule directly to a vCPU count without accounting for hyperthreading.
Treating the CPU-backend KV cache and multimodal processor cache as mandatory defaults for every model.
Enabling CPU offload without validating the CPU-GPU interconnect under load.
Combining --kv-cache-memory-bytes with an assumed --gpu-memory-utilization effect.
Using automatic context selection without recording the resulting startup capacity.
Accepting repeated benchmark results without accounting for prefix-cache reuse.
Comparing reports that use different metric names or workload limits.

These steps are compiled from the official documentation linked in the text; nothing was run or measured for this article, so check the current version before use.