Build a GPU Cost Model for Self-Hosted LLM Hosting

A practical GPU cost model covering compute, power, colocation, labor, and depreciation, grounded in linked AWS, NVIDIA, and GHG Protocol documentation.

A self hosted llm cost model should separate GPU compute, electricity and cooling, colocation, operations labor, and hardware depreciation, then compare that total with a rented-capacity quote for the same workload. Fill every rate from the provider’s current bill or quote instead of inserting an assumed price. Follow the calculation order below so commitments, extra charges, and power overhead stay visible.

Define the workload and cost boundary

Choose the workload unit and model period before entering prices. Requests, generated tokens, training runs, and GPU-hours can each work, but the selected unit must remain consistent across owned and rented scenarios. Record planned utilization separately from billable runtime so idle headroom is not mistaken for productive work.

Use a worksheet with fields similar to these:

MODEL_PERIOD = YOUR_VALUE
WORKLOAD_UNIT = YOUR_VALUE
WORKLOAD_VOLUME = YOUR_VALUE
GPU_TYPE_AND_COUNT = YOUR_VALUE
GPU_RUNTIME = YOUR_VALUE
IT_ENERGY_KWH = YOUR_VALUE
PUE = YOUR_VALUE
ELECTRICITY_RATE = YOUR_VALUE
COLOCATION_QUOTE = YOUR_VALUE
LABOR_COST = YOUR_VALUE
CAPEX = YOUR_VALUE
RESIDUAL_VALUE = YOUR_VALUE
USEFUL_LIFE_MONTHS = YOUR_VALUE

This is a calculation template, not a provider configuration file. Apply the same accounting boundary to every scenario:

TOTAL_COST = COMPUTE + FACILITY_POWER + COLOCATION + LABOR + DEPRECIATION + OTHER_CHARGES

FACILITY_POWER = IT_ENERGY_KWH * PUE * ELECTRICITY_RATE

DEPRECIATION = (CAPEX - RESIDUAL_VALUE) / USEFUL_LIFE_MONTHS

UNIT_COST = TOTAL_COST / WORKLOAD_VOLUME

Keep one-time purchases, recurring operating costs, and depreciation in separate columns. That prevents capital expenditure from being counted repeatedly while still allowing an owned deployment to be compared with rented capacity.

Model self hosted llm ownership against rented capacity

For owned capacity, include the GPU and supporting hardware, facility electricity, cooling, colocation, installation, and labor. For rented capacity, include compute, the commitment or purchase method, storage, addressing, networking, and any other billable resources.

OWNED_TOTAL = DEPRECIATION + OWNED_FACILITY + OWNED_LABOR

RENTED_TOTAL = RENTED_COMPUTE + RENTED_STORAGE + RENTED_NETWORK + RENTED_EXTRAS

FINANCIAL_DIFFERENCE = RENTED_TOTAL - OWNED_TOTAL

Treat depreciation as an allocation of the original purchase cost, not as the bill paid during the current period. Track the cash purchase separately. Review the result under different utilization, hardware-life, energy-price, and commitment assumptions rather than relying on a single total.

Price GPU compute and the commitment

Use the provider named in the quote. For an AWS scenario, EC2 On-Demand billing counts only seconds in the running state and applies a 60-second minimum. The applicable calculation is:

ON_DEMAND_BILLABLE_SECONDS = MAX(RUNNING_SECONDS, 60)
ON_DEMAND_COMPUTE = ON_DEMAND_BILLABLE_SECONDS * QUOTED_SECONDARY_RATE
COMPUTE_TOTAL = SUM_OF_APPLICABLE_BILLED_COMPUTE_LINES

AWS describes three commitment structures in its EC2 billing and purchasing options: a Savings Plan commits to consistent usage measured in USD per hour for a 1- or 3-year term; a Reserved Instance commits to a specific configuration, including instance type and Region, for a 1- or 3-year term; and Capacity Blocks can reserve a cluster of GPU instances.

Read the quote for the exact commitment amount, eligible configuration, Region, term, and additional-use treatment. Do not infer one payment schedule for every purchase option. Compare the commitment cost with the runtime your workload actually needs, and keep enough On-Demand or interruptible capacity in the scenario if demand can change.

Add electricity, cooling, and colocation

Google’s PUE explanation compares non-computing overhead energy, including cooling and power distribution, with energy used by IT equipment. Its example says a PUE of 2.0 means one additional watt of overhead for each watt of IT power, while a PUE nearer to 1.0 means nearly all energy is used for computing.

FACILITY_ENERGY_KWH = IT_ENERGY_KWH * PUE
FACILITY_POWER_COST = FACILITY_ENERGY_KWH * ELECTRICITY_RATE

Use metered IT consumption where available rather than treating a GPU’s rated power as constant actual consumption. Ask a colocation provider to separate recurring facility charges, power capacity or metered electricity, cooling, connectivity, and one-time installation charges. Do not add electricity again if the colocation quote already includes it, and do not apply a provider’s fleet-wide PUE to your own facility without supporting data.

Turn GPU specifications into planning constraints

Published specifications help expose power and physical-fit requirements, but they do not establish workload throughput:

GPU	Published memory	Published power figures	Published physical fit
GeForce RTX 4090	24 GB GDDR6X	450 W total graphics power; 850 W required system power	3-slot; 304 mm × 137 mm
GeForce RTX 5090	32 GB GDDR7	575 W total graphics power; 1000 W required system power	2-slot; 304 mm × 137 mm
NVIDIA L4	24 GB	72 W maximum TDP	1-slot low-profile PCIe
NVIDIA L40S	48 GB GDDR6 with ECC	—	—

Required system power is not a substitute for a power meter, and memory capacity alone does not prove that a particular model, context length, or concurrency setting will fit. Keep model-fit and capacity acceptance tests separate from the financial model.

Price labor and depreciation

Count deployment, monitoring, security updates, model upgrades, incident recovery, user support, and billing reconciliation. Enter hours from internal records and apply your organization’s approved labor rate:

LABOR_COST = SUM(RECORDED_HOURS * APPROVED_INTERNAL_RATE)

For a China enterprise tax discussion, Article 60 of the EIT Implementation Regulations lists electronic equipment at a minimum depreciation life of 3 years. Article 59 says straight-line depreciation for fixed assets is deductible, begins in the month after the asset is placed in use, and requires the estimated net residual value to be determined reasonably; once determined, it may not be changed.

The current text of Article 98 allows shortened depreciation for fixed assets that update quickly because of technological progress, but the shortened period cannot be less than 60% of the Article 60 period. It also permits double-declining balance or sum-of-the-years-digits methods for accelerated depreciation. Verify the current legal text before applying these rules, and keep the engineering useful-life assumption separate from the tax treatment.

Audit cloud charges line by line

Reconcile the compute total with every other applicable row in the provider’s bill:

EC2 lifecycle documentation states that EBS volumes and Elastic IP addresses can incur charges regardless of instance state, even though a stopped instance is not billed for instance usage. Each stop-to-running transition starts a new billing period with a one-minute minimum.
Every public IPv4 address is chargeable, including addresses associated with running instances and Elastic IPs, according to AWS VPC IP addressing guidance.
A NAT gateway is charged for each hour it is available and for each gigabyte it processes. AWS NAT gateway pricing guidance also advises keeping workloads in the same Availability Zone as the gateway to reduce data transfer charges.
AWS data transfer documentation identifies cross-Availability-Zone traffic inside a Region under DataTransfer-Regional-Bytes and internet egress under DataTransfer-Out-Bytes. Internet-to-AWS inbound transfer is not charged; for inter-Region transfer, the charge applies to data leaving the source Region, while entry into the destination Region is free.
The AWS unwanted-charge checklist notes that deleting an EBS volume does not delete its snapshots. It also says a load balancer must be terminated before its registered EC2 instances can be deleted.

Add distinct invoice lines without double-counting them:

CLOUD_TOTAL = COMPUTE + STORAGE + IP_ADDRESSING + NAT_GATEWAY + DATA_TRANSFER + OTHER_USAGE_TYPES

Keep emissions reporting separate from cash cost

The GHG Protocol Scope 2 Guidance places indirect emissions from the generation of purchased or acquired electricity, steam, heat, or cooling in Scope 2. Its location-based method uses the average emissions intensity of the grid where consumption occurs; its market-based method uses factors associated with the consumer’s chosen electricity supplier, product, or contractual instrument.

LOCATION_BASED_EMISSIONS = ELECTRICITY_USE * LOCATION_GRID_FACTOR
MARKET_BASED_EMISSIONS = ELECTRICITY_USE * CONTRACTUAL_FACTOR

The guidance says companies operating in markets with contractual instruments must report Scope 2 both ways and label each result by method. Retain these calculations beside the energy-cost worksheet without converting either emissions result into a financial rate. This reporting and tax discussion is general information, not legal advice.

Validate the model and handle failure modes

Recalculate the model from the invoice lines, confirm that runtime and workload volume cover the same period, and verify that power costs use the same facility boundary as PUE. Test the break-even point under several utilization and commitment scenarios.

Do not treat Spot capacity as a guaranteed fixed baseline. AWS states that Spot Instances are not covered by Savings Plans; when launch capacity is unavailable, a Spot request keeps retrying, while an On-Demand request can receive an insufficient capacity error.

An EC2 interruption notice is issued two minutes before EC2 stops or terminates the instance, although notices are delivered on a best-effort basis. AWS recommends checking every 5 seconds. The documented IMDSv2 path is:

http://169.254.169.254/latest/meta-data/spot/instance-action

Use that signal as one input to interruption handling, not as a guarantee of uninterrupted service. Nothing in this article was run or benchmarked. The steps are compiled from the official documentation linked in the text and should be checked against the current version before use.