AI DeploySelf-hosted LLMGet it deployed

Compare API and Self-Hosted LLM Deployment Paths

Compare Replicate API billing with self-hosted LLM operations, then choose a path through endpoint, hardware, migration, and cost-control checklists.

An API path delegates model hosting to the provider; a self hosted llm path makes your team responsible for the host, runtime, updates, and endpoint. In the documented examples, Replicate separates billing by model type, while vLLM serves one model per process from http://localhost:8000. Compare cost exposure, hardware control, endpoint compatibility, and maintenance work before moving traffic; neither label is cheaper by definition.

Compare the Two Paths Before Choosing

This comparison uses Replicate as the managed API path and vLLM on a machine or rented GPU as the self-host path. It does not turn one provider’s terms into a universal API rule: another service may meter usage, expose endpoints, and support model features differently, so its current documentation must control the decision.

Replicate says official models stay warm and are billed by predictable output metrics, including input and output tokens. The Replicate official models guide documents POST /models/YOUR_VALUE/YOUR_VALUE/predictions as the call path and says you do not specify a version. This gives the API side a concrete endpoint and billing model without implying that every managed service behaves the same way.

When self hosted llm fits the operating model

Use the path that matches the work you are prepared to own:

  • Choose the API path when avoiding responsibility for the host operating system, GPU lifecycle, capacity, and serving processes is more important than controlling those layers directly.
  • Choose the self-host path when your team can maintain the serving stack, reproduce its environment, manage hardware changes, and investigate endpoint or compatibility failures.
  • Choose neither automatically: write down the required model capabilities, request patterns, recovery expectations, budget owner, and acceptable billing variability first.

These are operating-fit questions, not a ranking of Replicate or vLLM. A provider can supply the HTTP service while your application still owns credentials, client compatibility, monitoring, and incident response.

Identify the API Billing Mode First

The Replicate billing documentation separates charges by model type and instance state. For public models, hardware and the request queue are shared with other customers by default. Without demand, a model scales down to its configured minimum instance count, which is 0 by default; after a gap in requests, an instance can remain idle for a few minutes. Setup and idle time are free, while active request-processing time is billed. Most private models use dedicated hardware without a shared queue, but their setup, idle, and active time are all charged. Fast-booting fine-tunes are billed only for active time whether public or private, while deployments are billed for all time their instances remain online. Failed runs are generally not charged, but cancellation rules depend on the category: a cancelled official-model run may still be charged, a cancelled time-billed run is charged through cancellation, and failed or cancelled private-model and deployment runs are billed for instance active time. A model that calls other models adds the root model’s compute charges to the downstream model charges. Accounts billed in arrears are charged at the start of each month for the previous month’s usage, while prepaid-credit usage is deducted from the balance. Replicate may also charge early when usage first crosses certain limits as a fraud check. The hardware assigned to a model appears under Run time and cost on its model page.

Build a Cost Worksheet Before Implementation

Use placeholders rather than assumed prices. For an official model, build the estimate from the applicable input and output token quantities and published rates in the official-model documentation:

OFFICIAL_TOKEN_MODEL_ESTIMATE =
  (YOUR_VALUE × YOUR_VALUE) + (YOUR_VALUE × YOUR_VALUE)

PUBLIC_MODEL_ESTIMATE =
  YOUR_VALUE × YOUR_VALUE

PRIVATE_OR_DEPLOYMENT_ESTIMATE =
  (YOUR_VALUE + YOUR_VALUE + YOUR_VALUE) × YOUR_VALUE

MODEL_CHAIN_ESTIMATE =
  YOUR_VALUE + YOUR_VALUE

SELF_HOST_CLOUD_ESTIMATE =
  YOUR_VALUE + YOUR_VALUE

The first line represents input-token and output-token units multiplied by their applicable rates. The second uses active processing time and its rate. The third adds setup, idle, and active time before applying the online-time rate. The fourth separates root-model compute from downstream-model charges. The fifth selects the applicable instance or commitment charge and adds other quoted line items.

Treat each result as an exposure estimate, not a provider quote. Read the current model page or account terms, match every request to the correct billing category, and include cancellation, downstream-model, and account-level charges where applicable.

Install, Start, and Verify vLLM

The vLLM quickstart requires Linux and Python 3.10 through 3.13 and documents an installation environment using Python 3.12 and uv. The project’s GPU installation guide says Windows is not supported natively, directs Windows users to WSL with a compatible Linux distribution, and requires NVIDIA GPUs with compute capability 7.5 or higher.

With uv available, create and activate the documented environment:

uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm --torch-backend=auto

The --torch-backend=auto option lets uv inspect the installed CUDA driver version and select the PyTorch index. For the documented prebuilt CUDA path, the package contains precompiled C++ and CUDA 12.9 binaries. The GPU guide also states that B200 and GB200 require at least CUDA 12.8.

The corresponding pip installation command is:

pip install vllm --extra-index-url https://download.pytorch.org/whl/cu129

Start one server with the model name:

vllm serve Qwen/Qwen2.5-1.5B-Instruct

The server listens at http://localhost:8000 by default; --host and --port set its address. One server process hosts one model at a time. Check reachability and model registration with:

curl http://localhost:8000/v1/models

Treat this as a server and routing check, not as validation of application output quality. Keep the service local while you design network exposure, authentication, monitoring, and process recovery.

Check Compatibility Before Moving Traffic

The vLLM OpenAI-compatible server guide says the server implements OpenAI’s Completions and Chat APIs. Use its endpoint list to build the migration inventory:

  • /v1/completions applies to text-generation models and does not support suffix.
  • /v1/chat/completions requires a chat template and ignores the user parameter.
  • /v1/responses supports retrieval and cancellation through its response sub-paths.
  • /v1/embeddings applies only to embedding models.
  • /v1/audio/transcriptions applies only to Automatic Speech Recognition models.
  • /v1/chat/completions/batch provides the documented batch chat-completions path.

The same compatibility guide records request-level differences that can break a client migration. Setting parallel_tool_calls=false limits a response to zero or one tool call, while the default value true permits more than one. Parameters not present in the OpenAI client, such as top_k, can be supplied through extra_body; with direct HTTP calls, they are merged into the JSON payload. The Chat API does not support image_url.detail.

Inventory these fields before changing the destination URL. A successful connection is not enough if the application relies on an unsupported parameter, the wrong model capability, or an endpoint the current server does not expose.

Account for Rented GPU Commitments

The Amazon EC2 purchasing options says On-Demand Instances are billed by the second. It describes Savings Plans as commitments to consistent usage measured in USD per hour for a one- or three-year term. Reserved Instances commit to a specific instance configuration, including instance type and Region, for the same available term lengths. Capacity Blocks are documented as a way to reserve a cluster of GPU instances.

For a rented-GPU self-host path, request the current quote for the exact configuration and identify the applicable instance or commitment charge, billing unit, term, Region constraints, and every other quoted line item. The purchasing documentation explains the structures, but it does not establish your effective rate. Enter those current values into the self-host worksheet rather than treating a provider’s general pricing page as a quote.

Make the Final Decision in This Order

  1. Classify each workload by model type, billing mode, endpoint, and expected usage pattern.
  2. Add the applicable formula inputs, including cancellation exposure, downstream calls, and cloud line items.
  3. Verify the host operating system, Python version, GPU requirements, and installation environment.
  4. Start the server, confirm /v1/models, then test representative requests against every endpoint and payload field the application uses.
  5. Move production traffic only after compatibility, operational ownership, billing controls, and rollback configuration are recorded.

These steps are compiled from the official documentation linked in the text and should be checked against the current version before use.

Sources