Compare GPU Cloud Prices for Self-Hosted LLMs
Compare AWS and Vast.ai GPU quotes using per-second billing while accounting for commitments, interruptions, storage, and other continuing charges.
For llm hosting, compare quotes by holding the exact GPU model and count, memory, host resources, location, storage, and workload constant, then add every compute and non-compute line item. Amazon EC2 On-Demand bills by the second, while Vast.ai rental pricing combines GPU compute, storage, and bandwidth and bills actual usage by the second. Use each provider’s current quote and billing statement rather than comparing headline GPU rates alone.
Normalize every llm hosting quote
Create one comparison row for every candidate. A row is comparable only when the hardware, workload, location, and billing assumptions match.
| Field | Record from the quote |
|---|---|
| Accelerator | Exact GPU model, GPU count, and memory per card |
| Host resources | CPU, system memory, machine type, and local storage |
| LLM workload | Model or fine-tune, quantization, context setting, concurrency, and expected runtime |
| Location | Cloud, Region, and Availability Zone where documented |
| Storage | Capacity, type, retention period, and deletion plan |
| Network | Expected inbound and outbound transfer and whether traffic crosses locations |
| Billing | Purchase option, active and stopped periods, retry or fallback plan |
| Quote validity | Quote date, currency, rate unit, and any expiration shown |
Do not collapse different GPU models into a generic “GPU” category. GPU memory can help you check whether a candidate belongs in the comparison, but it is not itself a throughput or model-performance guarantee. Likewise, two prices are not equivalent if one includes more system memory, local storage, or committed capacity while the other does not.
Compare purchase models before rates
In Amazon EC2’s billing and purchasing options, On-Demand billing is charged by the second. A Savings Plan commits to consistent usage measured in USD per hour for a 1-year or 3-year term; a Reserved Instance commits to a specific instance type and Region for a 1-year or 3-year term; and Capacity Blocks can reserve a cluster of GPU instances.
The Vast.ai pricing guide describes a marketplace in which hosts set their own prices. Its combined rental cost includes GPU compute, storage, and bandwidth, with actual usage billed by the second. The same guide says storage charges continue while an instance is stopped and cease only after the instance is completely deleted.
For every candidate, record:
- The billed unit, such as per second, per hour, per storage unit, or per transferred unit
- Any minimum documented for that charge
- The commitment term and end date shown in the quote
- Included and excluded resources
- Cancellation terms stated in the current documentation or order
- The exact Region, Availability Zone, or host location
A commitment should remain a separate comparison column. Do not convert it into an unrestricted hourly purchase merely because both columns display a monetary rate.
Price interruption and retry behavior
Amazon EC2 Spot Instances use prices set by instance type and Availability Zone, adjusted gradually with long-term supply and demand. Spot Instances are not covered by Savings Plans. When On-Demand capacity is unavailable at launch, the request can fail with an insufficient capacity error, while a Spot request keeps retrying; EC2 also provides a rebalance recommendation when interruption risk is elevated.
Google Compute Engine Spot VMs can be preempted at any time. The shutdown period is best effort and up to 30 seconds, Spot VMs are excluded from the Compute Engine SLA, and preemption is signalled by setting the VM metadata value preempted to TRUE.
Do not assume that every interruption has the same cost or operational effect. If your workload requires a continuously available endpoint, enter fallback capacity as a separate candidate. If it can retry, include every attempt’s compute, storage, and network entries from the eventual bill rather than counting only the successful attempt. Never build a reliability claim around an assumed interruption rate that the provider’s current documentation does not state.
Add charges outside GPU runtime
According to Amazon EC2 instance state changes, a stopped instance is not billed for instance usage, but each stopped-to-running transition starts a new billing period with a one-minute minimum. EBS volumes and Elastic IP addresses can still incur charges regardless of the instance state, so stopping compute does not establish that the whole environment costs nothing.
The AWS data transfer documentation identifies same-Region cross-Availability-Zone transfer under DataTransfer-Regional-Bytes and transfer from AWS to the internet under DataTransfer-Out-Bytes. Transfer from the internet into AWS is not charged. For inter-Region transfer, the charge applies to data leaving the source Region rather than entering the destination Region.
Match the network direction and location before entering these costs. Separate compute, storage, address, cross-location transfer, internet egress, and provider-specific line items in the worksheet. This prevents a low compute figure from obscuring a different storage or transfer total.
Calculate without mixing units
The actual amount depends on the provider’s current quote and your bill; this guide does not insert a currency figure. Replace each YOUR_VALUE with the exact billed quantity and quoted rate from the same candidate:
PER_SECOND_COMPUTE = YOUR_VALUE × YOUR_VALUE
HOURLY_COMPUTE = YOUR_VALUE × YOUR_VALUE
STORAGE_COST = YOUR_VALUE × YOUR_VALUE
NETWORK_COST = YOUR_VALUE × YOUR_VALUE
TOTAL_MATCHED_COST = YOUR_VALUE + YOUR_VALUE + YOUR_VALUE + YOUR_VALUE + YOUR_VALUE
For per-second compute, the first value is billed compute seconds and the second is the quoted rate per second. For hourly compute, use billed hours and the quoted hourly rate. Apply the same rule to storage and network charges: keep the provider’s billing unit unchanged.
Do not multiply hours by a per-second rate, combine per-GPU and whole-instance quotes, or convert between currencies before recording the provider’s amount. For a commitment candidate, copy every charge written into the quote, including storage, addresses, transfer, and other stated items. For a variable candidate, use the billable quantities shown in the quote or statement.
You can then calculate:
EFFECTIVE_COST_PER_WORKLOAD_UNIT = TOTAL_MATCHED_COST ÷ YOUR_VALUE
Use the same workload denominator for every candidate. Choose it before reviewing the totals, and do not change it for one provider merely because its result appears less favourable.
Verify the result and catch common errors
Before choosing, verify that:
- The exact GPU model, GPU count, memory, CPU, and system memory match.
- The model, quantization, context, concurrency, and runtime assumptions match.
- Region, Availability Zone, storage location, and transfer direction match.
- Running, stopped, retry, and fallback assumptions are explicit.
- Every rate uses the unit printed in its quote.
- Compute, storage, address, network, and other line items are present.
- Commitment terms and quote validity are recorded beside the total.
The most common errors are comparing different machine bundles, mixing billing units, omitting storage or network charges from one candidate, and treating a commitment as flexible hourly purchasing. Correct those differences before interpreting the totals.
The procedure is compiled from the official documentation linked in the text, checked on October 1, 2026; nothing was run or measured, and the current version should be checked before use.