GPU Rental Costs for Self-Hosted LLM Workloads

Compare rented GPU costs by checking accelerator memory, power, availability, commitment, storage, bandwidth, and utilization before deployment.

GPU rental costs for a self hosted llm workload should be treated as an all-in operating cost, not as the advertised GPU rate alone. Start with memory and power fit, then add compute, storage, data transfer, commitment, idle-time, and deletion consequences. Compare those items against your interruption tolerance, and verify the selected machine and billing state before deployment.

Prepare the workload before comparing quotes

The cost worksheet for a self hosted llm workload should start with the model revision, runtime, precision or quantization choice, request shape, required accelerator memory, and whether interruptions or stopped storage are acceptable. Before opening a provider quote, capture the offered GPU and count, compute billing unit, storage capacity and lifecycle, transfer terms, any commitment or refund term, and the provider’s reliability or verification labels.

The published product specifications provide a capacity screen rather than a rental quote or performance result:

GPU	Published memory	Published power
GeForce RTX 4090	24 GB GDDR6X	450 W total graphics power; 850 W required system power
GeForce RTX 5090	32 GB GDDR7	575 W total graphics power; 1000 W required system power
NVIDIA L4	24 GB GPU memory	Maximum TDP of 72 W
NVIDIA L40S	48 GB GDDR6 with ECC	Maximum power consumption of 350 W
NVIDIA A100	80 GB HBM2e; up to 7 MIG instances of 10 GB each	Maximum TDP of 300 W for PCIe or 400 W for SXM
NVIDIA H100	80 GB for SXM; 94 GB for NVL	Up to 700 W configurable for SXM; 350–400 W configurable for PCIe NVL
NVIDIA H200	141 GB	Up to 700 W configurable for SXM; up to 600 W configurable for NVL

Use memory as an initial filter, then consult the selected runtime’s current documentation for actual capacity requirements. Published memory and power do not establish model fit, availability, throughput, or the price you will pay.

Consumer and datacenter GPU selection also involves a licensing check. NVIDIA’s GeForce software license says GeForce or Titan driver software is licensed only for GeForce or Titan hardware you own and is not licensed for datacenter deployment. Clause 2.7 also prohibits providing commercial hosting services with the software unless expressly granted. Do not assume a marketplace listing grants that permission. This is general information, not legal advice.

Turn provider terms into an all-in quote

Read the provider’s own pricing page, offer, resource listing, and final statement. Check:

Pricing page: compute rate and unit, storage rate and billing trigger, and whether upload or download traffic is charged.
Offer terms: minimum balance, upfront commitment, refund terms, expiration, and whether a plan remains attached after a Pod stops.
Resource listing: GPU model and count, memory, reliability or verification status, and interruption behavior.
Final statement: every separately itemized charge; do not compare one provider’s compute subtotal with another provider’s all-in total.

Use placeholders rather than inserting unverified prices:

TOTAL_PROVIDER_CHARGE =
  YOUR_COMPUTE_CHARGE
+ YOUR_STORAGE_CHARGE
+ YOUR_DATA_TRANSFER_CHARGE
+ YOUR_OTHER_SEPARATELY_ITEMIZED_CHARGE

Record an upfront commitment separately as a cash-flow requirement. If it changes the quoted compute rate, do not add the same commitment to the provider-charge formula again.

Vast.ai

Vast.ai operates as a marketplace where hosts set their own prices. Its instance pricing guide says the total combines GPU compute, storage, and bandwidth, with actual usage billed by the second. Storage charges continue while an instance is stopped and cease only when the instance is deleted. Upload and download traffic are each charged per byte at host-specific rates.

The type choice changes more than the displayed price. Vast.ai’s instance type documentation labels on-demand instances as fixed-price and high-priority, reserved instances as prepaid and discounted, and interruptible instances as lowest-cost but pausable. An interruptible renter sets a bid and can be stopped by a higher bid, which kills running processes. Its Rental Types FAQ says paused data is preserved, although the instance is not functional, and the instance resumes automatically when priority returns.

For availability checks, Vast.ai’s Concepts page defines the reliability score as historical uptime and health. Its Find and rent guide says verified machines have passed Vast’s internal tests, while unverified machines are hidden by default.

RunPod Pods

RunPod’s Pod pricing guide says Pods are billed by the second for compute and storage, with no fees for data ingress or egress. Container and volume disks are billed per second, network volumes hourly, and charges do not accrue while the host is unavailable. On-demand Pods use pay-as-you-go pricing, their resources cannot be displaced by other users, and deployment requires a balance covering at least one hour of the selected configuration.

Savings plans require an upfront 3-month or 6-month commitment. After a Pod stops, the plan applies to the next deployment using the same GPU type. These plans are non-refundable and have fixed expiration dates, so include both the cash commitment and expiration in the approval checklist.

AWS stopped instances

The AWS EC2 instance lifecycle documentation says a stopped instance is not billed for instance usage, but EBS volumes and Elastic IP addresses can incur charges regardless of instance state. Each stop-to-running transition starts a new billing period with a one-minute minimum. An instance being hibernated is billed while it remains in the stopping state, so “stopped” does not mean every attached resource has stopped billing.

Deploy with the lifecycle cost defined

For RunPod, the Pod management guide gives this console flow:

Select the GPU type.
Name the Pod.
Optionally choose a template.
Set the GPU count.
Click Deploy On-Demand.

The documented CLI creation form is:

runpodctl pod create --name NAME --gpu-id "GPU" --image "IMAGE"

The documented REST request uses a POST to the Pod endpoint:

curl --request POST \
  --url https://rest.runpod.io/v1/pods \
  --header 'Authorization: Bearer RUNPOD_API_KEY'

The lifecycle guidance in the same Pod management documentation determines whether storage and data charges continue:

Stopping releases the GPU, preserves data in /workspace on the volume disk, and clears the container disk.
A stopped Pod remains subject to volume-disk storage charges; termination is the documented way to end that charge.
Terminating permanently deletes data not stored on a network volume, so export anything that must survive.
Editing a running Pod resets it and erases data outside /workspace or a network volume.

If deployment returns OCI runtime create failed, the documented remedy is to open Additional filters → CUDA Versions and select compatible machines.

Verify the GPU and the billing state

Use the commands in the NVIDIA System Management Interface documentation to inspect the machine:

nvidia-smi -q
nvidia-smi --query-gpu=pci.bus_id,persistence_mode
nvidia-smi --help-query-gpu

The -q option displays all GPU attributes, with unsupported values shown as N/A. The query option accepts a comma-separated property list, and --help-query-gpu lists the available properties. Compare the reported board serial with the number physically printed on the board, and check memory and assigned GPU details against the order.

The diagnostic output also exposes ECC, retired-page, row-remapping, temperature, and power information. Do not treat reported fan speed as a physical airflow measurement: NVIDIA says it is the intended speed, so a physically blocked fan may not appear in that value.

On Linux, search the kernel log for "NVRM: Xid". NVIDIA’s Xid introduction defines an Xid message as a driver error report that can indicate a hardware problem, an NVIDIA software problem, or a user application problem. Preserve the Xid value and surrounding logs rather than assigning a cause without context.

Afterward, compare the provider’s charge details with your worksheet and update the state used in the formula: running, stopped, paused, deleted, or terminated. For interruptible rentals, include interruption and restart time in the cost assessment.

Calculate cost from your own workload

Use a representative request shape and keep the model and runtime settings consistent when comparing candidates. Record elapsed time, successful and failed jobs, retries, idle or nonfunctional intervals, and the provider’s total charges. Do not infer throughput from GPU memory or power specifications.

Use these fill-in calculations after deployment:

COST_PER_SUCCESSFUL_JOB =
  TOTAL_PROVIDER_CHARGE / SUCCESSFUL_JOB_COUNT

EFFECTIVE_ALL_IN_RATE =
  TOTAL_PROVIDER_CHARGE / WALL_CLOCK_HOURS

NONPRODUCTIVE_COST_SHARE =
  NONPRODUCTIVE_CHARGE / TOTAL_PROVIDER_CHARGE

Keep failed jobs, retries, paused periods, and persistent storage charges in the total rather than reporting only the apparent cost of active GPU time. If the current provider documentation or quote does not settle a rate, storage trigger, commitment, refund term, or availability condition, resolve it before deployment.

No command or workload was run or measured for this article. The procedure is compiled from the official documentation linked in the text and should be checked against the current version before use.