AI DeploySelf-hosted LLMGet it deployed

Choose a GPU Host for Self-Hosted LLM Deployment

Compare documented regional GPU availability, capacity controls, purchase terms, extra charges, and SLA remedies before selecting a self-hosted LLM host.

Choose an llm hosting provider by checking whether the exact GPU is offered in the selected Region or zone, whether quota and capacity controls permit launch, whether purchase and continuing-charge rules fit the workload, and whether the SLA remedy is acceptable. Proximity to users matters, but it does not by itself settle GPU availability, resilience, or bill composition. Use the official checks below to narrow the candidate before creating a billable GPU instance.

Check model fit before choosing llm hosting

Start with a requirement sheet, not a provider brand. Record the model and revision, serving runtime, quantization method, context and concurrency targets, required GPU type and GPU memory, acceptable Regions, and any legal or data-location constraint. The provider documentation cited here does not determine your model’s memory requirement or prove that it fits a particular instance; obtain those values from current documentation for the exact model and runtime.

Google Cloud states that the availability of a specific GPU model varies by Region and zone, so use its GPU locations documentation to filter candidates before comparing purchase options. AWS Regions are isolated, and resources are not replicated across Regions automatically; the AWS Regions and Zones guide advises selecting a Region close to customers or one that meets legal requirements. It also says that launching EC2 instances across multiple Availability Zones can protect an application from failure of a single location, while Local Zones extend an AWS Region geographically closer to users for low-latency access.

Google recommends choosing a Region or zone close to the point of service and deploying fault-tolerant, high-availability applications across multiple zones and Regions in its Compute Engine regions and zones guidance. Treat those location and resilience goals as separate checks: a nearby location may reduce network delay, but neither location nor multi-zone deployment by itself proves that your exact GPU model is available.

Separate GPU availability, quota, and capacity

Check availability, quota, and capacity as separate gates:

  • GPU location: Google defines ZONE_RESOURCE_POOL_EXHAUSTED as a zone lacking enough resources for the request. Its resource availability troubleshooting recommends trying again later or using another zone, and recommends reserving resources ahead of time to reduce the risk of recurrence.
  • Account quota: AWS sets EC2 quotas per Region and recommends requesting increases before they are needed, as explained in the Amazon EC2 service quotas documentation.
  • Live capacity: AWS distinguishes InsufficientInstanceCapacity, meaning insufficient current On-Demand capacity, from InstanceLimitExceeded, meaning a Regional launch quota has been reached. Its launch troubleshooting guidance suggests waiting and retrying, removing the Availability Zone from the request, or reducing the requested instance count for the capacity error.
  • Capacity assurance: If uninterrupted access in a specific Availability Zone is strictly required, AWS recommends a Capacity Reservation in its EC2 Capacity Reservations guide.
  • Local inventory: Alibaba Cloud’s regions and zones guide documents this query for the zones and saleable resources in a Region:
aliyun ecs DescribeZones --region {RegionId}

The same guide says VPC access to ECS within one Region does not incur a traffic fee, public access from ECS in China to ECS in another country or Region can have high latency and may involve packet loss, and a resource’s Availability Zone cannot be changed directly after creation. Use those constraints to confirm the location before committing to a deployment design.

Compare purchase options and the full cost

Before accepting a quote, ask the provider’s pricing and contract pages to identify:

  • The exact GPU, Region, zone, and purchase option supporting the proposed configuration.
  • The billing unit, applicable state charges, minimum duration, commitment term, and renewal conditions.
  • Separate compute, storage, allocated addresses, data transfer, request, support, tax, and other applicable charges.
  • Reservation conditions, term expiration, early-termination treatment, service-credit rules, and refund terms.

Use the provider’s current quote or bill to populate this calculation:

estimated_total = compute + storage + network + requests + commitment + other

compute = GPU_RATE × BILLED_GPU_HOURS
storage = STORAGE_RATE × PROVISIONED_STORAGE
network = NETWORK_RATE × EGRESS_VOLUME
requests = REQUEST_RATE × REQUEST_COUNT

Normalize every rate and usage value to the provider’s billing unit. Do not reuse a rate quoted for another GPU, Region, or purchase mode.

AWS documents several distinct choices. EC2 On-Demand billing counts only time in the running state, with a 60-second minimum. The AWS purchasing options guide defines Savings Plans as commitments to consistent usage measured in USD per hour for a one- or three-year term, Reserved Instances as commitments to a specific instance type and Region for the same term lengths, and Capacity Blocks as a way to reserve a cluster of GPU instances. Actual rates are not supplied here; read them from the provider’s current offer.

Include resources that remain billable after compute stops. The EC2 instance lifecycle documentation says a stopped instance is not billed for instance usage, but EBS volumes and Elastic IP addresses can incur charges regardless of instance state. Each stop-to-running transition starts a new billing period with a one-minute minimum, and an instance being hibernated is billed while it enters the stopping state.

Spot capacity requires a separate operating plan. The EC2 Spot Instances documentation says Spot Instances are not covered by Savings Plans, an On-Demand request receives an insufficient capacity error when capacity is unavailable, and a Spot request continues retrying. It also documents a rebalance recommendation when interruption risk is elevated. Under the Spot interruption notices documentation, the warning is issued two minutes before EC2 stops or terminates the instance, delivery is best-effort, and AWS recommends checking every five seconds. The documented IMDSv2 interruption-action path is:

http://169.254.169.254/latest/meta-data/spot/instance-action

Treat this as interruption metadata, not as a model-serving health endpoint. If the service cannot tolerate interruption or needs time to drain active requests, include recovery capacity and an operational response in the host decision.

Read the SLA as a contract

Under the Amazon Compute SLA, the Region-Level commitment is 99.99% monthly uptime when all running EC2 instances are concurrently deployed across two or more Availability Zones in the same Region; the stated alternative applies when a Region has only one Availability Zone. The Instance-Level commitment for a single EC2 instance is 99.5% monthly uptime, with “unavailable” defined as the instance having no external connectivity.

The same agreement states that service credits are the sole and exclusive remedy rather than a refund. A customer must submit a claim through the AWS Support Center, and the request must arrive by the end of the second billing cycle after the incident.

Google’s Compute Engine SLA defines VM downtime as loss of external connectivity or persistent disk access. It excludes partial or intermittent downtime lasting less than one minute and requires notification to technical support within 60 days after the customer becomes eligible for a financial credit. Read the uptime commitment, unavailability definition, exclusions, and claim process together rather than comparing percentages alone.

This is general information about documented service terms, not legal advice.

Verify the GPU host before model load

Approve the host only when the decision record contains:

  • The exact model, runtime, quantization, GPU, and GPU-memory requirements from current project documentation.
  • Confirmation that the exact accelerator is offered in the intended Region and zone.
  • The current quota status and the selected response to capacity shortages.
  • A complete cost estimate based on the provider’s quote or bill.
  • A data-protection and failure-recovery design appropriate to the deployment.
  • The applicable SLA, claim process, and internal owner responsible for a claim.
  • For Spot capacity, an interruption-monitoring and recovery procedure.
Model and revision:
Serving runtime:
Quantization:
GPU and memory requirement:
Provider, Region, and zone:
Quota status:
Purchase mode:
Capacity safeguard:
Data and recovery design:
SLA and claim owner:
Estimated monthly total:

The cited provider documentation does not define a universal model deployment command or model health endpoint; obtain those from the selected serving runtime’s current documentation. These steps are compiled from the official documentation linked in the text and should be checked against the current version before use.

Sources