AI DeploySelf-hosted LLMGet it deployed

Test RunPod Storage for a Self-Hosted LLM

RunPod network volumes outlive Pod deletion; check persistence, serve a model with vLLM, and verify the endpoint using documented steps.

Test RunPod storage for a self hosted llm by checking the selected disk’s lifecycle, confirming the path used for retained files, and then serving and verifying a model. Keep vLLM request metrics separate from storage speed: the cited documentation provides no NVMe, HDD, or network-volume throughput command. The procedure below is a validation plan, not a report of hands-on results.

Choose the disk lifecycle you need to test

Start with the event your data must survive. RunPod’s storage options define these storage behaviors:

Storage option Documented lifecycle Encryption
Container disk Temporary storage created when the Pod launches and cleared when the Pod stops Cannot be encrypted
Volume disk Data in /workspace survives stops and restarts but is deleted when the Pod is terminated Can be encrypted
Network volume Exists independently of any Pod, can be attached to multiple Pods, and remains after the Pod is deleted Cannot be encrypted

Volume disk size can only be increased, not decreased. If your requirement is persistence through Pod deletion, the network volume is the relevant option; if it is limited to stop-and-restart persistence, the volume disk provides that lifecycle.

Do not treat any of these disks as archival storage. RunPod states that it is not designed for long-term cloud storage and recommends backing up critical data to a local machine or a dedicated cloud storage provider.

Prepare and attach a network volume

In the console workflow documented by RunPod’s network-volume guide, select a data center, enter a name, and specify the size in GB. For Pods, network volumes are available only in Secure Cloud.

Attach the network volume when you create the Pod. It cannot be detached later, and an attached network volume replaces the default volume disk at /workspace. Its size can be increased but not decreased; for volumes beyond 4 TB, contact RunPod support.

Storage charges continue accruing while the Pod is stopped. RunPod’s published warning, checked on 2026-10-01 and subject to change, says that if the balance remains at $0 and the charges cannot be covered, the network volume may eventually be terminated and its data cannot be recovered.

Use this sequence to check persistence without confusing storage layers:

  • Place a uniquely named test file under /workspace on the attached network volume.
  • Stop and restart the Pod, then confirm that the file remains.
  • Terminate the Pod, attach the same network volume to another Pod, and check the file again.
  • Perform write checks without simultaneous workers writing to the volume, because RunPod warns that concurrent writes may corrupt data.

This establishes whether the data survived each lifecycle event. It does not establish storage speed.

Install vLLM in a fresh environment

The vLLM quickstart specifies Linux and Python 3.10 to 3.13. Its GPU installation guide requires an NVIDIA GPU with compute capability 7.5 or higher and notes that the pre-built package contains pre-compiled C++ and CUDA 12.9 binaries.

Run the quickstart’s recommended fresh-environment commands:

uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm --torch-backend=auto

With --torch-backend=auto, uv inspects the installed CUDA driver version and selects the appropriate PyTorch index automatically.

Before starting the server, place any model or cache artifacts that must persist in the attached /workspace. The cited quickstart does not define a cache-directory flag, so consult the current vLLM project documentation for the chosen model’s loading method. Merely attaching a network volume does not prove that vLLM loaded model files from it.

Expose and verify the model endpoint

The quickstart starts a model with vllm serve. For RunPod, use the documented address flags so the service does not remain bound only to localhost:

vllm serve Qwen/Qwen2.5-1.5B-Instruct --host 0.0.0.0 --port 8000

A vLLM server process hosts one model at a time. The default endpoint is http://localhost:8000, but RunPod’s port-exposure guide requires a proxied service to bind to 0.0.0.0, not localhost or 127.0.0.1.

In the Pod configuration, add the port to the comma-separated Expose HTTP Ports (Max 10) field. RunPod documents this HTTP proxy pattern:

https://[POD_ID]-[INTERNAL_PORT].proxy.runpod.net

Here, the internal port is 8000. A proxied HTTP service is publicly reachable, so implement authentication inside the application; the Pod ID does not replace application access controls. The proxy also passes through Cloudflare, which closes connections after 100 seconds with a 524 error if the service has not responded.

Check the server locally first:

curl http://localhost:8000/v1/models

Then request /v1/models through the proxy URL supplied in the Pod configuration. A successful local request verifies the vLLM process; a successful external request also verifies that the service is bound correctly and exposed through the proxy.

Test a self hosted llm with vLLM

After the model endpoint works, the vLLM benchmark CLI documentation provides a serving benchmark based on ShareGPT. Download the documented dataset:

wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json

Run the documented online benchmark against the running server:

vllm bench serve \
  --backend vllm \
  --model NousResearch/Hermes-3-Llama-3.1-8B \
  --endpoint /v1/completions \
  --dataset-name sharegpt

The report includes request throughput and output-token and total-token throughput. Time to first token, or TTFT, runs from sending a request to receiving its first streamed output. Inter-token latency, or ITL, records the gaps between consecutive streamed outputs. Time per output token, or TPOT, is calculated once per request, excluding the first token, and then aggregated. With speculative decoding, one streamed output may contain several tokens, so ITL and TPOT can diverge.

--request-rate controls requests per second; inf selects maximum-throughput behavior and is the default, sending all requests immediately. --max-concurrency caps outstanding requests, while leaving it unset makes concurrency unlimited. Repeating the benchmark against the same server can reuse prompts in the prefix cache and inflate throughput.

These results describe the vLLM endpoint. They are not NVMe, HDD, or network-volume throughput measurements.

Where the check usually goes wrong

Use failures to identify the layer that needs attention:

  • If the test file disappears, compare the Pod event with the selected storage option’s lifecycle.
  • If the local model endpoint fails, inspect the vLLM server command before judging storage.
  • If the local endpoint works but the proxy does not, revisit the bind address and exposed HTTP port.
  • If benchmark results change between runs, inspect the request controls and possible prefix-cache reuse.
  • If you need storage speed, do not substitute TTFT, ITL, TPOT, or token throughput for a disk test.

These steps are compiled from the official documentation linked in the text and should be checked against the current version before use.

Sources