AI DeploySelf-hosted LLMGet it deployed

Estimate Replicate Costs for LLM Inference

Replicate bills private models and deployments for online time, unlike active-only public runtime; estimate tokens, warm capacity, and transfer costs.

For llm hosting on Replicate, Replicate’s billing documentation says public models share a hardware pool by default, where cold boots or scaling limits can occur, but only active request-processing time is charged; setup and idle time are free. The same billing guidance bills private models and deployments for all online time—setup, idle, and active processing—while fast-booting fine-tunes are charged only for active time; it also places hardware under the exact Run time and cost heading and adds downstream compute when one model calls another. Replicate’s official-model documentation says official models are priced by output metrics such as input and output tokens and are always warm, so use a token-based worksheet instead of an instance-time formula for that category.

Map llm hosting demand to the billing model

Build the worksheet before choosing a formula. Give each root model, downstream model, and external data path its own row so that different billing units do not disappear into one estimate.

Record the following:

  • BILLING_CATEGORY: public model, official model, private model, deployment, or fast-booting fine-tune.
  • USAGE_QUANTITY: input tokens, output tokens, active processing time, or online time.
  • ONLINE_INSTANCES: the number of deployment instances available during each relevant interval.
  • DEPENDENCY: every downstream model invoked by the root model.
  • RATE_AND_UNIT: the current rate and unit shown by Replicate for the selected model or deployment.
  • TIME_WINDOW: the billing or planning period covered by the worksheet.

Use the matching calculation template:

PUBLIC_OR_FAST_BOOTING_COST
= YOUR_ACTIVE_PROCESSING_TIME × YOUR_DOCUMENTED_RATE_PER_TIME_UNIT


OFFICIAL_LLM_COST
= YOUR_INPUT_TOKENS × YOUR_DOCUMENTED_INPUT_TOKEN_RATE
+ YOUR_OUTPUT_TOKENS × YOUR_DOCUMENTED_OUTPUT_TOKEN_RATE


PRIVATE_OR_DEPLOYMENT_COST
= Σ(
    YOUR_ONLINE_INSTANCES_IN_INTERVAL
    × YOUR_ONLINE_TIME_IN_INTERVAL
    × YOUR_DOCUMENTED_INSTANCE_RATE
  )


MODEL_CHAIN_COST
= YOUR_ROOT_MODEL_COST
+ Σ(YOUR_DOWNSTREAM_MODEL_COSTS)

Run the private-model or deployment sum separately for every interval and documented rate. Do not mix token quantities with hours in the same multiplication.

For an official-model integration, the documented endpoint does not require a version:

POST /models/<owner>/<name>/predictions

Feed your own input and output token quantities into the official-model formula. Do not substitute a remembered GPU rate unless the current model page or quote identifies it as the applicable billing rate.

Configure a deployment without losing the cost boundary

Replicate’s Deployments documentation says deployments autoscale with demand. A maximum instance count can be used to limit maximum spend, while a minimum keeps instances warm and ready. Because deployment instances are charged while online, a warm minimum creates a recurring baseline; the maximum controls instance count, but it does not replace the rest of the cost worksheet.

The documented deployment-creation request targets the following endpoint and accepts these fields:

POST /v1/deployments

name
model
version
hardware
min_instances
max_instances

Those keys come from the deployment creation documentation. Before submitting a request, copy the hardware and instance settings into your cost worksheet. Treat any example in the documentation as syntax rather than as a recommended budget, then calculate the online intervals that those settings can produce.

Estimate a workload you can repeat

NVIDIA’s GenAI-Perf documentation describes a command-line tool for measuring throughput and latency on generative AI models served through an inference server. It also states that GenAI-Perf is no longer receiving new features and directs new benchmarking work to AIPerf, so use AIPerf’s current documentation for the executable command and options.

Build the workload from your own traffic rather than a generic request set. Include representative input lengths, requested output lengths, concurrency, request gaps, and any cold or warm states that matter to your application. Record throughput, time to first token, and inter-token latency, then keep the test inputs and configuration with the estimate.

If throughput and request units match, you can derive an active-time estimate:

YOUR_ESTIMATED_ACTIVE_TIME
= YOUR_BILLABLE_REQUESTS / YOUR_OBSERVED_SUSTAINED_THROUGHPUT

Use sustained results that reflect the arrival pattern you expect. A short peak measurement can disappear during idle periods and will not represent the full workload. For a public runtime or fast-booting fine-tune, pass the resulting active time into the active-time formula. For a private model or deployment, also add every setup and idle interval, multiplied by the number of online instances during that interval.

Add data transfer without mixing providers

If your architecture also contains AWS resources that you procure, keep their estimated charges in a separate worksheet rather than folding them into the Replicate inference subtotal.

AWS data-transfer documentation identifies several usage types relevant to this check. Cross-Availability-Zone transfer inside one Region appears under DataTransfer-Regional-Bytes. For inter-Region transfer, the charge applies to data leaving the source Region, while data entering the destination Region is not charged. Transfer from AWS to the internet appears under DataTransfer-Out-Bytes, while transfer from the internet into AWS is not charged.

Create a row for each direction your workload uses:

DATA_TRANSFER_COST
= Σ(
    YOUR_USAGE_TYPE_QUANTITY
    × YOUR_DOCUMENTED_DATA_TRANSFER_RATE
  )

Take the quantity from your usage report or bill and the applicable rate from your current provider quote or bill. If a route does not have a matching usage record, investigate it rather than assuming that every direction is free.

Check the estimate against actual charges

Recalculate the worksheet from the records used to create it. Then compare the estimate with the corresponding provider charge line by line:

  • Confirm that each root and downstream model appears once.
  • Match token counts or time units to the selected formula.
  • Check setup, idle, and active intervals separately where the worksheet uses online time.
  • Reconcile instance counts with the intervals entered for a deployment.
  • Add separately tracked data-transfer rows.
  • Replace copied rates with the provider’s current entries before relying on the result.

Calculate any remaining difference directly:

RECONCILIATION_VARIANCE
= YOUR_OBSERVED_CHARGE - YOUR_RECALCULATED_ESTIMATE

A zero variance confirms only that the chosen quantities and rates reproduce the charge being checked. It does not make the same workload or rate a safe assumption for the next planning period.

Where estimates usually go wrong

The main risk is applying one billing assumption to every request path. Check the worksheet for these calculation errors:

  • Using the public active-time formula for a private model or deployment.
  • Applying an instance-time formula to an official LLM model.
  • Leaving setup or idle intervals out of an online-time total.
  • Counting active processing once as active time and again inside the online total.
  • Treating a per-unit rate as a complete workload cost.
  • Recording the root model while omitting downstream model calls.
  • Treating a deployment’s maximum instance setting as the entire budget.
  • Combining Replicate inference charges with separately procured cloud or data-transfer charges.
  • Reusing an old rate instead of copying the current model-specific entry.

These steps are compiled from the official documentation linked in the text and should be checked against the current version before use.

Sources