Choose a GPU Cloud for Self-Hosted LLM Inference

Compare RunPod, Lambda, Vast.ai, and SaladCloud: RunPod meters Pod compute and storage per second, while stopped Pods still incur storage.

Choose llm hosting by matching the execution model to your needs around persistence, interruption, scaling, and billing; a GPU model alone cannot answer those questions. This guide compares RunPod, Lambda, Vast.ai, and SaladCloud, gives RunPod’s documented creation paths, and provides a like-for-like worksheet for comparing quotes and invoices.

What must llm hosting preserve?

Start with the data lifecycle rather than the GPU:

Model weights and cache: Decide whether they must remain local, can be downloaded again, or need independent retention.
Checkpoints and generated files: Record whether a stop should preserve them and whether termination must remove them.
User and session state: Separate durable application data from state that can be reconstructed.
Temporary files: Treat these as disposable unless your recovery design says otherwise.

Next, write down whether the inference process must stay on the same machine, can restart elsewhere, or can retry requests. A replicated, stateless design may tolerate reallocation differently from a single long-running process. Do not treat model persistence as backup: test your export and restoration procedure separately.

Region selection, legal data-location requirements, failover boundaries, and recovery expectations belong in the same worksheet. Do not infer a latency winner from pricing pages; current product documentation and the path from your users to the endpoint are separate inputs.

Build a like-for-like cost worksheet

Open each provider’s current pricing page or quote and copy the actual line items. Use your invoice or quote to fill these formulas rather than inserting unsupported estimates:

RUNPOD_TOTAL = POD_COMPUTE + CONTAINER_DISK + VOLUME_DISK + NETWORK_VOLUME

RunPod’s Pod pricing distinguishes per-second container and volume disk charges from hourly network-volume charges and states that data ingress and egress are not charged. Include a storage line only when your configuration or quote contains it.

VAST_TOTAL = GPU_COMPUTE + STORAGE + BANDWIDTH

Vast.ai’s instance pricing guide says a rental combines GPU compute, storage, and bandwidth. It describes actual usage as billed by the second, while transfer is charged per byte for both upload and download at a host-specific rate.

SALAD_TOTAL = RUNNING_TIME_CHARGES

SaladCloud’s billing overview says running time is billed per second and that no charge applies while a container group is allocating or downloading. Check the current page for any other applicable service or network line before approving the quote.

LAMBDA_TOTAL = INSTANCE_USAGE + FILESYSTEM_USAGE

Lambda’s billing overview says On-Demand Cloud instances use hourly rates with one-minute billing increments. Filesystems are charged for GiB used per month in one-hour increments, and the page states that ingress and egress are not charged.

Before committing, record the billing increment, commitment, refund form, support terms, SLA, Region, storage lifetime, and deletion rules from the provider’s own page and quote. These documents do not establish one provider as universally cheaper, faster, or more reliable.

Match the runtime model to the failure model

RunPod: dedicated Pod resources and explicit billing states

RunPod’s Pod pricing and plan rules describe on-demand Pods as pay-as-you-go with dedicated resources that cannot be displaced by other users. Deployment requires credits covering at least one hour of the selected configuration. Savings plans use a 3- or 6-month upfront commitment, are non-refundable, and have fixed expiration dates; storage charges continue while a Pod is stopped.

Use this model when dedicated resources and a clear Pod lifecycle are central to your decision. Treat any longer commitment as a separate approval because stopping compute does not necessarily end every charge.

Vast.ai: marketplace pricing and different interruption levels

Vast.ai’s instance pricing model uses a marketplace in which hosts set their own prices. Its documented rental choices are on-demand, reserved with prepayment and a discount, and interruptible at low priority; an interruptible instance may be paused.

If an inference process cannot tolerate interruption, do not treat the lowest-priced rental class as interchangeable with on-demand execution. If you can checkpoint, retry, and resume, document that workflow before accepting interruption.

SaladCloud: distributed containers rather than attached-volume VMs

SaladCloud’s architecture overview describes a two-sided marketplace connecting PC owners with compute-intensive workloads. A container group contains replicas of the same image, with each replica placed on a separate node.

These are not VMs with attached volumes. The documentation says both image and runtime data are removed when the application stops, and Docker-in-Docker is not permitted. This model can fit inference that rebuilds local state and tolerates replica replacement, but durable checkpoints and other retained artifacts need a separate persistence design.

Lambda: billing continues across the instance lifecycle

Lambda’s billing overview says an On-Demand Cloud instance is billed from launch after health checks until termination, whether or not it is actively serving requests, and that invoices arrive weekly. A filesystem continues accruing charges while it exists, even when unmounted. Refunds are issued as service credits rather than cash and expire after 12 months.

Include idle runtime, filesystem lifetime, and credit treatment in your approval process rather than comparing only the displayed instance rate.

Deploy a RunPod Pod

RunPod’s Pod management documentation documents the following console sequence: Select GPU type → Name your Pod → (Optional) Choose a template → Set GPU count → Click Deploy On-Demand.

It also provides a CLI creation form using runpodctl pod create, --name, --gpu-id, and --image. Replace the placeholders with values for your environment:

runpodctl pod create --name YOUR_NAME --gpu-id "YOUR_GPU_ID" --image "YOUR_IMAGE"

The documented REST method, endpoint, and authorization header are:

POST https://rest.runpod.io/v1/pods
Authorization: Bearer YOUR_API_KEY

That block is not a complete request body. Obtain the currently required fields from the current API reference before sending it. Likewise, Pod creation does not provide a universal model-server startup command: after the environment is available, configure and launch your chosen inference server according to that project’s official documentation.

Check the inference path, not only Pod creation

Verify the deployment from the outside in:

Environment state: Use the current console or API status to confirm that the Pod has reached the state required by your workload.
Runtime health: Inspect logs for model-loading failures, device errors, missing dependencies, and the service’s listening state.
Inference response: Send a minimal request to the endpoint defined by your application or inference server. Do not invent a provider health URL that does not appear in current documentation.
Persistence behavior: Place a disposable test file in your selected storage location and perform the stop or restart action you expect to support. Confirm that its lifetime matches the design.
Metering: Review the provider’s usage view after the controlled run and confirm that the active or stopped resources match your billing assumptions.

A successful creation response establishes only that the provisioning request was accepted; it does not prove that the model loaded or that the inference endpoint returned a valid response.

Where deployments commonly go wrong

Selecting a GPU before defining the runtime: Model loading, concurrency, quantization, and context requirements should shape the hardware choice. No benchmark from this article should be treated as your expected result.
Keeping the only copy on runtime storage: Export important artifacts and verify restoration before relying on any stop, detach, or deletion behavior.
Treating creation as readiness: Start the process, load the model, exercise the application endpoint, and inspect logs as separate checks.
Ignoring runtime compatibility: For OCI runtime create failed, RunPod’s Pod troubleshooting guidance says to use Additional filters → CUDA Versions to select compatible machines.
Comparing prices without a lifecycle model: Include idle periods, storage continuation, transfer, commitment, refund form, Region, and recovery requirements in the final worksheet.

These steps are compiled from the official documentation linked in the text; nothing was run or measured for this article, so check them against the current version before use.