AI DeploySelf-hosted LLMGet it deployed

Audit Serverless GPU Inference Billing

Audit serverless GPU inference billing by checking compute, idle windows, storage, addresses, and data transfer against usage records.

An llm hosting invoice should be audited by separating GPU compute, CPU and memory, startup, idle or keep-alive time, storage, public infrastructure, data transfer, credits, and every other documented charge. Serverless does not create a universal billing unit: first identify what the provider considers billable, then match its resource states and minimum billing windows to the invoice. The result should be a reproducible ledger that explains each line, not a generic cost-per-request estimate.

Start the llm hosting audit

Before calculating anything, collect the records needed to explain the invoice:

  • Billing basis: Save the provider’s current billing page, your quote, and the invoice. Record the plan, billing cycle, effective date, rates, measurement units, included usage, credits, subscription charges, and any other recurring line items.
  • Activity timeline: Record request arrival, model or container startup, processing start and finish, the last input, scale-up, scale-to-zero, and every relevant resource-state transition. Normalize timestamps to the same timezone.
  • Resource configuration: Record the requested GPU, CPU, memory, replica settings, keep-alive setting, storage, and attached network resources where applicable. Distinguish requested resources from resources actually used.
  • Network activity: Record transferred bytes, direction, source and destination, relevant Region or Availability Zone, and the provider’s usage type.
  • Attribution: Associate each resource with an application, model, environment, and responsible team. Keep management allocations separate from charges that the provider actually issued.

If a quote and an invoice differ, preserve both and ask the provider to explain the discrepancy. Do not silently replace the invoice with an estimate.

Build a serverless GPU inference ledger

Use the provider’s native billing units. A GPU time, CPU time, memory quantity, storage quantity, and transferred byte are different units and must not be combined without the corresponding rates.

gpu_compute = billable_gpu_units * YOUR_GPU_RATE
cpu_memory = billable_cpu_memory_units * YOUR_CPU_MEMORY_RATE
storage = billable_storage_units * YOUR_STORAGE_RATE
transfer = billable_transfer_units * YOUR_TRANSFER_RATE

estimated_total =
    sum(all_documented_resource_charges)
    + other_documented_charges
    - documented_credits

variance = invoice_total - estimated_total

Enter only units that the provider documents as billable. If startup, an idle window, or a minimum billing period applies, include it in the relevant field. If the treatment is unclear, mark the amount as unresolved rather than entering zero.

For shared infrastructure, a request-level allocation can help with internal reporting, but label it as a management estimate. The provider invoice remains the reconciliation target.

Read each provider’s billing definition

Modal

Modal’s pricing documentation says included compute for Functions, Sandboxes, and Notebooks is metered by the second for CPU, memory, and GPU time. CPU and memory are charged at the higher of the requested or actual amount, and each container has a minimum request of 128 MiB of memory and 0.125 CPU cores. Modal does not impose a minimum usage time per instance.

Billable time includes application loading, input processing, and a default 60 seconds of container keep-alive after the last input. Once the application has scaled to zero containers, charging stops. In the ledger, keep loading, processing, and keep-alive time as separate components so that the charge can be traced to a documented event.

RunPod

RunPod’s billing overview describes a prepaid credit system: funds are added first, and resource usage is deducted from the balance. Compute and storage are billed per second, data transfer has no fee, and the billing job runs every 5 minutes. Storage on a network volume continues accruing charges while Pods are stopped, while deploying a new Pod requires at least one hour’s worth of credits for the selected configuration.

The same billing overview, as checked on 2026-10-01, says that a $0 balance automatically stops all running Pods; Pods without a network volume are terminated and their data cannot be recovered. This provider-published condition may change. Record the balance threshold separately from compute and storage consumption so that depletion risk is visible before the balance event.

Replicate

For public models, Replicate’s billing guide limits charges to active request-processing time and says setup and model idle time are free. By default, public models use a hardware pool and queue shared with other customers, so a request can enter a shared queue.

When a model has no demand, Replicate scales it to its configured minimum number of instances, which is 0 by default and can be customized for deployments. After a gap in requests, an instance can remain idle for a few minutes instead of shutting down immediately. Apply the public-model billing rule only where it applies; use the current documentation and quote for a custom deployment instead of extrapolating from the public-model terms.

Check infrastructure that outlives the request

EC2 state and retained resources

A stopped Amazon EC2 instance is not billed for instance usage, but AWS’s instance-state documentation says EBS volumes and Elastic IP addresses can incur charges regardless of the instance state. Each stop-to-running transition also starts a new billing period with a one-minute minimum.

Reconcile the state timeline with the invoice. A stopped compute resource may remove one charge while leaving storage or address charges, and a restart can create a new minimum billable interval.

NAT gateway charges

The AWS NAT gateway pricing documentation describes two billing components: each hour that the gateway is available and each gigabyte it processes. Record gateway availability time and processed data separately.

AWS also recommends placing resources in the same Availability Zone as the NAT gateway to reduce data-transfer charges. Availability Zone placement therefore belongs in the audit worksheet, not only in the compute configuration.

Data-transfer direction and usage types

AWS’s data-transfer documentation maps same-Region, cross-Availability-Zone transfer to the DataTransfer-Regional-Bytes usage type and transfer from AWS to the internet to DataTransfer-Out-Bytes. Inbound transfer from the internet to AWS is not charged.

For inter-Region transfer, the charge applies to data leaving the source Region, while data entering the destination Region is free. Match the invoice usage type, direction, and Region before applying the applicable rate; “egress” alone is not a sufficiently precise audit label.

Reconcile and close the audit

Work through the ledger in this order:

  • Match every invoice line to a documented rate, usage record, credit, or plan charge.
  • Reconcile GPU and CPU/memory runtime with startup, scale, and request events.
  • Reconcile storage and other persistent resources independently of compute runtime.
  • Verify transfer direction, geography, and usage type.
  • Explain each difference caused by rounding, minimum billing windows, credits, or a shared-resource allocation.
  • Ask the provider to resolve any remaining unexplained variance using the deployment identifier, timestamps, resource state, usage type, and quoted rate.

The audit is complete when every invoice line has a source, every billable interval has a recorded cause, all persistent resources have been checked, and variance is either zero or supported by a documented explanation.

Where serverless GPU inference audits commonly fail

  • Assuming every idle interval is free: Modal documents a billable default keep-alive window, while Replicate says public-model idle time is free. Use the provider-specific meter rather than a universal grace period.
  • Treating “stopped” as “removed”: AWS’s state documentation warns that EBS volumes and Elastic IP addresses can remain billable when the instance is stopped.
  • Checking only GPU time: RunPod distinguishes compute from continuing network-volume storage, while AWS data-transfer records distinguish transfer by direction and usage type. Include storage and networking in the same audit.
  • Applying public-model terms to a custom deployment: Replicate’s billing rules expressly allow deployments to customize minimum instances. Confirm the deployment type before choosing a billing treatment.

No deployment was run or measured for this article; the steps are compiled from the official documentation linked in the text and should be checked against the current version before use.

Sources