AI DeploySelf-hosted LLMGet it deployed

Calculate Serverless GPU Egress for LLM Inference

Learn how to calculate serverless GPU inference egress, verify RunPod's free data transfer, and include AWS cross-region transfer charges.

For llm hosting on serverless GPUs, calculate egress by assigning each traffic path to the provider’s documented direction and billing unit, multiplying the billed data by that path’s rate, and then adding compute, storage, and other charges. RunPod’s billing overview states that data transfer is free, while AWS’s transfer billing guide separates internet-bound, cross-Availability-Zone, and inter-Region charges; one generic bandwidth multiplier can therefore misstate the bill.

Prepare a serverless GPU cost worksheet

Build a separate worksheet for each provider account, deployment Region, inference endpoint, and billing cycle. From the provider’s own current billing page or quote, record:

  • The directions that are billed, such as outbound, inbound, or traffic leaving a Region.
  • The billing unit for every rate.
  • Compute, storage, egress, cross-zone, and inter-Region line items.
  • Any included allowance, usage threshold, billing minimum, or tier boundary.
  • Subscription charges, incremental usage charges, and available credits.
  • The exact usage types or billing keys shown in reports.

Keep each calculation in the unit used by the provider. Do not convert bytes, gibibytes, or another unit unless the provider defines the conversion.

YOUR_EGRESS_TOTAL =
  (YOUR_BILLED_OUTBOUND_DATA * YOUR_OUTBOUND_RATE)
  + (YOUR_BILLED_CROSS_AZ_DATA * YOUR_CROSS_AZ_RATE)
  + (YOUR_BILLED_INTER_REGION_DATA * YOUR_INTER_REGION_RATE)

If the provider uses tiered rates, calculate each tier separately rather than applying the highest rate to every billed unit. Then construct the hosting estimate:

YOUR_HOSTING_TOTAL =
  YOUR_COMPUTE_CHARGES
  + YOUR_STORAGE_CHARGES
  + YOUR_EGRESS_TOTAL
  + YOUR_OTHER_DOCUMENTED_CHARGES
  - YOUR_CREDITS

Leave unsupported rows unresolved. An empty egress rate is not evidence that traffic is free, just as a free transfer statement does not remove compute or storage from the estimate.

Map llm hosting traffic before applying rates

Start with traffic paths rather than token counts. For each inference path, identify the source, destination, direction, Region or data center, and billed unit.

Your ledger should distinguish:

  • Response bytes sent from the inference endpoint to callers.
  • Model, cache, or artifact downloads initiated by the runtime.
  • Traffic crossing Availability Zones or Regions.
  • Traffic processed by a billed network component.
  • Client requests entering the service, which must remain separate from outbound traffic.

Use byte counters or provider usage records that you can reconcile with the invoice. Do not treat prompt and completion token counts as byte measurements unless your own method establishes the conversion.

Calculate each mutually exclusive billed route first:

YOUR_ROUTE_COST =
  YOUR_BILLED_ROUTE_DATA * YOUR_RATE_FOR_THAT_ROUTE

YOUR_EGRESS_TOTAL =
  SUM_OF_YOUR_ROUTE_COSTS

A request can cross several infrastructure layers, but those layers should not be combined blindly. Add separate costs only when the provider’s current documentation or invoice identifies them as separate charge types.

Calculate the RunPod serverless case

The cited RunPod billing overview says compute and storage are billed per second, data transfer is free, and the billing process runs every 5 minutes. For an estimate based on that documentation, leave out a separate data-transfer charge and obtain the current compute and storage rates from your configuration or quote:

YOUR_RUNPOD_TOTAL =
  YOUR_COMPUTE_CHARGES
  + YOUR_STORAGE_CHARGES
  + YOUR_OTHER_DOCUMENTED_CHARGES

The per-second billing rule does not make a forecast identical to the invoice. Your worksheet should still preserve the configuration, runtime interval, storage selection, and any other resources associated with each workload.

Storage also needs its own stopped-workload scenario. RunPod’s network-volume documentation says attaching a network volume to a Serverless endpoint constrains worker deployments to that volume’s data center. It also says storage charges continue while Pods are stopped and warns that a volume may eventually be terminated if charges cannot be covered, after which its data cannot be recovered.

YOUR_STOPPED_POD_SCENARIO =
  YOUR_COMPUTE_CHARGES
  + YOUR_RETAINED_STORAGE_CHARGES
  + YOUR_OTHER_DOCUMENTED_CHARGES

Keep that data-center placement choice separate from the egress arithmetic. It may change the deployment design, but it does not replace the provider’s documented transfer treatment.

Keep AWS transfer paths separate

For an AWS deployment, apply a rate only to the route identified in the usage record. The cited AWS data-transfer guide records same-Region traffic between Availability Zones under DataTransfer-Regional-Bytes and data sent from AWS to the internet under DataTransfer-Out-Bytes. It states that internet-to-AWS transfer is not charged. For inter-Region transfers, the charge applies to data leaving the source Region, while transfer into the destination Region is free.

YOUR_AWS_TRANSFER_TOTAL =
  YOUR_INTERNET_OUTBOUND_COST
  + YOUR_CROSS_AZ_COST
  + YOUR_INTER_REGION_COST

YOUR_INTER_REGION_COST =
  YOUR_BYTES_LEAVING_SOURCE_REGION * YOUR_ROUTE_RATE

Do not add inbound internet traffic as a positive egress charge, and do not apply the internet-out rate to inter-Region movement merely because both involve bytes leaving one boundary.

A NAT gateway introduces another billing layer. AWS NAT gateway pricing states that the gateway is charged for each hour it is available and for each gigabyte it processes. AWS also recommends placing resources in the same Availability Zone as the gateway when they exchange significant traffic, to reduce data-transfer charges.

YOUR_NAT_TOTAL =
  (YOUR_NAT_AVAILABLE_HOURS * YOUR_NAT_HOURLY_RATE)
  + (YOUR_NAT_PROCESSED_GB * YOUR_NAT_PER_GB_RATE)

Keep NAT processing and the underlying route-transfer charge as separate lines until the usage records show whether both apply to the same architecture. Blending them into one bandwidth assumption makes the result impossible to audit.

Handle Modal without inventing an egress rate

Modal’s billing documentation describes charges for compute used or requested, without reservations or minimum usage-time increments. It says Workspaces are billed monthly, incremental usage charges can occur the first time specified thresholds are exceeded, and subscription fees apply to non-Starter plans. Credits reduce the amount charged.

That billing page does not state an egress rate. Until the current documentation, quote, or contract identifies a transfer charge and its unit, leave that row unresolved rather than treating the missing line item as free.

YOUR_MODAL_ESTIMATE =
  YOUR_COMPUTE_CHARGES
  + YOUR_INCREMENTAL_USAGE_CHARGES
  + YOUR_NON_STARTER_SUBSCRIPTION_FEES
  + YOUR_EGRESS_CHARGE_IF_DOCUMENTED
  - YOUR_CREDITS

Do not present the estimate as complete until the egress row is documented. If the billing materials still do not settle it, ask the provider to identify the applicable transfer label, direction, unit, and rate before approving the deployment budget.

Verify the estimate against billing data

Reconcile the worksheet after the first invoice and whenever the model, response format, Region, storage class, or network architecture changes.

  • Match every invoice line to the provider’s exact usage type.
  • Confirm the Region, direction, billing unit, and charged quantity.
  • Recalculate each route independently before summing it.
  • Keep provider credits and subscription charges in their own rows.
  • Compare the calculated total with the amount actually invoiced.
  • Investigate differences before applying the result to future traffic forecasts.

AWS Cost Explorer provides historical data for up to the last 13 months, forecasts spending for the next 18 months, and refreshes cost data at least once every 24 hours. Use that view for scenario analysis, but do not substitute a forecast or delayed cost view for the final invoice.

Common calculation mistakes

The most frequent errors are applying one rate to unrelated paths, converting units without provider-defined rules, treating tokens as bytes, and assuming that an undocumented charge is free. Another error is removing storage from the model merely because compute has stopped. Reconcile each layer against its own usage record, and preserve unresolved items as explicit placeholders rather than estimates presented as facts.

This guide is compiled from the official documentation linked in the text; no deployment or traffic pattern was run or measured for it, and the steps should be checked against the current versions before use.

Sources