Review GPU Charges Before Hosting an LLM
Review data transfer, snapshot, and static IP charges, then compare post-shutdown billing across AWS, Azure, Google Cloud, and Alibaba Cloud for an LLM stack.
Review llm hosting costs as a full running bill, not as a GPU instance rate: include storage, snapshots, public IP addresses, NAT processing, and data transfer before selecting a purchase option. A stopped compute instance does not automatically end the bill, because disks, addresses, and attached resources may follow different billing rules. Build the comparison from each provider’s current quote, billing page, and usage report rather than from a generic hourly estimate.
Build the llm hosting bill from every chargeable resource
Treat the worksheet as a quote-mapping tool. If a provider bundles a unit, enter the bundled charge once instead of multiplying it again. Read the purchase model, Region, operating state, storage type and size, snapshot status, IP attachment state, and the direction of transferred data before filling the fields.
Where the quote exposes those units, the worksheet can look like this:
GPU compute = YOUR_QUOTED_GPU_COMPUTE_CHARGE
Volume storage = YOUR_BILLED_VOLUME_GB × YOUR_VOLUME_STORAGE_RATE
Snapshots = YOUR_BILLED_SNAPSHOT_GB × YOUR_SNAPSHOT_RATE
Public IP = YOUR_BILLABLE_PUBLIC_IP_COUNT × YOUR_PUBLIC_IP_BILLING_PERIOD × YOUR_PUBLIC_IP_RATE
NAT = YOUR_NAT_AVAILABLE_HOURS × YOUR_NAT_HOURLY_RATE
+ YOUR_NAT_PROCESSED_GB × YOUR_NAT_PER_GB_RATE
Regional transfer = YOUR_BILLABLE_REGIONAL_BYTES × YOUR_REGIONAL_TRANSFER_RATE
Internet outbound = YOUR_INTERNET_OUT_BYTES × YOUR_INTERNET_OUT_RATE
Internet inbound = YOUR_INTERNET_IN_BYTES × YOUR_INTERNET_IN_RATE
Inter-Region outbound = YOUR_BILLABLE_SOURCE_BYTES × YOUR_INTER_REGION_RATE
Total = YOUR_GPU_COMPUTE + YOUR_STORAGE + YOUR_SNAPSHOTS + YOUR_PUBLIC_IP
+ YOUR_NAT + YOUR_REGIONAL_TRANSFER + YOUR_INTERNET_TRANSFER
+ YOUR_INTERNET_INBOUND + YOUR_INTER_REGION_TRANSFER
+ YOUR_OTHER_PROVIDER_LINE_ITEMS
The total is a reconciliation figure, not a prediction. It shows whether compute dominates or whether retained storage, addresses, and network paths change the decision.
Compare what remains billable after shutdown
| Provider | Documented behavior | Include in the review |
|---|---|---|
| AWS | The Amazon EC2 instance state documentation says a stopped instance is not billed for instance usage, but EBS volumes and Elastic IP addresses can still incur charges. Each stop-to-running transition starts a new billing period with a one-minute minimum; a hibernating instance is billed while stopping. | Compute state, EBS volumes and snapshots, Elastic IPs, restart pattern |
| Azure | The Azure VM states and billing documentation says a VM in Stopped (allocated) remains billed, while disks and networking can continue to incur charges after deallocation. | Allocated versus deallocated state, disks, networking |
| Google Cloud | The Compute Engine stop documentation says attached resources remain chargeable after an instance is stopped; disk pricing bills the full provisioned capacity until the disk is relinquished; and network pricing says a static external IP attached to a stopped VM still counts as in use, while an unassigned reserved address has a higher rate. | Attached objects, provisioned disk size, static IP assignment |
| Alibaba Cloud | The pay-as-you-go documentation says ordinary stopping still charges when savings stop is not enabled. With savings stop, instance specifications, fixed public bandwidth, and image-license charges pause, but cloud disks and Elastic public IPs continue; the block-storage documentation says a disk starts charging after creation regardless of attachment or purpose. | Stop mode, instance specification, bandwidth mode, disk, Elastic public IP |
The key question is not only whether compute is stopped, but which objects remain attached, allocated, or billable after shutdown. Record that state before launch and repeat the review after every stop-policy change.
Price the network path, not only the GPU
On AWS, NAT gateway pricing includes charges for each hour the gateway is available and each gigabyte it processes. AWS also recommends placing resources in the same Availability Zone as the gateway to reduce data-transfer charges.
Use the AWS data-transfer guide to map same-Region, cross-Availability-Zone traffic to DataTransfer-Regional-Bytes and AWS-to-internet traffic to DataTransfer-Out-Bytes. It states that internet-to-AWS inbound transfer is not charged. For inter-Region transfer, the charge applies to data leaving the source Region, while data entering the destination Region is free.
Public addresses require separate checks. The AWS VPC IP-addressing documentation says AWS charges for every public IPv4 address, including addresses associated with running instances and Elastic IP addresses. Its Elastic IP documentation further states that allocated and idle Elastic IPs are both chargeable.
For Alibaba Cloud, public-bandwidth billing says a fixed public IP is charged for outbound traffic but not inbound traffic. It supports charging by fixed bandwidth or by traffic used; under the traffic-used model, the configured bandwidth peak is a limit rather than a business commitment.
Check storage and snapshot residue
AWS snapshots are incremental backups containing changed blocks, and their charges are based on stored data. Because of that incremental structure, deleting one snapshot might not reduce storage costs, according to Amazon EBS snapshots.
Deleting the source volume does not complete snapshot cleanup. AWS states that deleting an EBS volume does not automatically delete its snapshots, and that a load balancer must be terminated before its registered EC2 instances can be deleted in its unexpected-charges checklist.
Alibaba Cloud separately states that pay-as-you-go snapshots are measured hourly from completion of creation until deletion in its pay-as-you-go documentation. List retained snapshots before approving a monthly budget.
Match the purchase option and interruption risk
Read the Amazon EC2 billing and purchasing options before accepting a commitment. AWS describes On-Demand Instances as billed by the second. A Savings Plan commits to consistent usage measured in USD per hour for a one- or three-year term; a Reserved Instance commits to a specific instance type and Region for the same term lengths; and Capacity Blocks can reserve a cluster of GPU instances.
Check capacity eligibility as well as price structure. The EC2 On-Demand documentation says quotas are counted in vCPUs rather than instances. It lists an adjustable default quota of zero vCPUs for the G and VT instance families, so an increase is needed before launching the intended configuration.
Spot capacity follows different rules. AWS says Spot Instances are not covered by Savings Plans; when capacity is unavailable, an On-Demand launch receives an insufficient capacity error, while a Spot request continues retrying, as described in its Spot Instance documentation.
Plan for interruption separately from ordinary shutdown. AWS issues a Spot interruption notice two minutes before stopping or terminating an instance, but delivery is best-effort; its termination-notice documentation recommends checking every five seconds and identifies this IMDSv2 path:
http://169.254.169.254/latest/meta-data/spot/instance-action
An interrupted Spot Instance that reaches the stopped state is still charged for its preserved EBS volumes, according to AWS interrupted Spot billing.
Check the GPU hardware envelope
Published specifications help you check memory, power, and physical fit; they do not establish hosting prices or performance results.
| NVIDIA option | Memory | Published power figure | Other published constraint |
|---|---|---|---|
| GeForce RTX 4090 | 24 GB GDDR6X | 450 W total graphics power; 850 W required system power | 304 mm long, 137 mm wide, three slots |
| GeForce RTX 5090 | 32 GB GDDR7 | 575 W total graphics power; 1000 W required system power | 304 mm long, 137 mm wide, two slots |
| L4 | 24 GB | Maximum TDP of 72 W | One-slot, low-profile PCIe card |
| L40S | 48 GB GDDR6 with ECC | Maximum power consumption of 350 W | Dual-slot, passively cooled, 16-pin power connector |
| A100 | 80 GB HBM2e | Maximum TDP of 300 W for PCIe and 400 W for SXM | Up to seven MIG instances of 10 GB each |
| H100 | 80 GB for SXM; 94 GB for NVL | Up to 700 W for configurable SXM and 350–400 W for configurable PCIe NVL | SXM or dual-slot, air-cooled PCIe form factor |
Use memory as a capacity constraint, power as an infrastructure constraint, and dimensions as a chassis constraint. Then verify the selected model’s software and memory requirements separately before treating a self-built system as comparable with rented GPU capacity.
Set controls before the first invoice
Assign an owner to reconcile the first usage report against the quote. AWS Cost Explorer can show up to thirteen months of history, forecast the next eighteen months, and refreshes cost data at least once every twenty-four hours.
Set alerts through AWS Budgets, which supports actual and forecasted spend alerts. Its information is updated up to three times a day and typically appears eight to twelve hours after the previous update, so alerts are not real-time; AWS Budget best practices also notes that forecast generation requires roughly five weeks of usage data.
Finally, activate the relevant cost-allocation tags. The AWS cost-allocation tag documentation distinguishes AWS-generated from user-defined tags and says they can take up to twenty-four hours to appear in the Billing and Cost Management console.
This guide is compiled from the official documentation linked in the text; nothing was run or measured for it, so check the current provider and NVIDIA documentation before use.