AI DeploySelf-hosted LLMGet it deployed

LLM GPU Hosting: On-Demand vs Reserved vs Spot

Compare AWS purchase options for GPU hosting, including Spot’s best-effort two-minute interruption warning, commitment scope, billing, and credits.

For llm hosting, choose On-Demand for runtime acquired without a term, choose a Reserved option only when its documented scope matches a steady baseline, and choose Spot only when recovery from interruption is acceptable. AWS distinguishes these purchase models in its EC2 billing and purchasing options, while the EC2 Spot Instances guide documents Spot capacity behavior; “reserved” should not be treated as one undifferentiated promise. Decide by commitment scope, interruption exposure, total billed items, and the applicable SLA remedy.

Choosing llm hosting when interruption matters

Start by classifying the workload rather than the purchase option:

  • A stateful online endpoint needs a clear failover or replacement path because an interruption can affect active requests.
  • A restartable batch workload may tolerate interruption if its application can resume from a checkpoint.
  • A predictable baseline may justify a commitment, but only if the committed scope matches how that workload actually runs.

GPU fit is a separate check. Confirm the required GPU type, count, and memory against the provider’s current instance listing, then check the model project’s current documentation for supported runtimes and deployment requirements. A purchasing option cannot establish compatibility by itself.

Before comparing offers, record the expected runtime, acceptable interruption delay, restart procedure, deployment location, and state that must survive replacement of a worker.

On-Demand: acquire runtime when you need it

AWS charges only for the seconds when an On-Demand Instance is running, subject to a 60-second minimum, according to Using On-Demand Instances for Amazon EC2. That makes a runtime forecast the starting point for this option: longer continuous execution increases the amount that must be evaluated, while periods when the instance is not running are excluded from instance runtime billing subject to the stated minimum.

Check account capacity before launch. AWS manages On-Demand quotas by the vCPUs used by running instances rather than by instance count. Its listed default for G and VT GPU instance families is 0 vCPUs and is adjustable. Unused Capacity Reservations still count toward this quota, while instances in pending, stopping, stopped, and hibernated states do not.

Prepare the On-Demand option in this order:

  • Select the exact GPU instance type and deployment location from the current provider listing.
  • Enter the planned running time into the cost worksheet.
  • Check the current vCPU quota for that instance family.
  • Confirm that the selected GPU configuration supports the selected open-source model and runtime.
  • Record any storage, addressing, data-transfer, or other items that the provider includes in the quote.

Use this path when you do not want a term commitment or want to evaluate runtime directly. The important limitation is that the purchase decision still depends on current capacity, quota, and the total quoted cost—not on runtime billing alone.

Reserved: distinguish the commitment from reserved capacity

AWS documents three different scopes under this broad label: a Savings Plan commits to consistent usage measured in USD per hour for a 1- or 3-year term; an EC2 Reserved Instance commits to a specific instance configuration, including instance type and Region, for a 1- or 3-year term; and a Capacity Block reserves a cluster of GPU instances. These are separate purchasing choices, as set out in the EC2 purchasing-options documentation.

Your decision should follow the scope:

  • For a usage commitment, compare the hourly usage amount you can genuinely commit with the forecast.
  • For a configuration commitment, verify that the required instance type and Region remain appropriate for the term.
  • For a cluster reservation, evaluate the whole GPU cluster rather than comparing it with the price of one instance.

On-Demand Capacity Reservations introduce another billing obligation. According to Capacity Reservation pricing and billing, they are charged at the equivalent On-Demand rate whether instances run in the reserved capacity or not. They are billed per second with a 60-second minimum, and applicable Savings Plans or Regional Reserved Instance discounts can apply.

Before accepting any Reserved offer, read the current offer page and written quote for the commitment amount, covered configuration, location, payment timing, expiry rules, cancellation terms, and charges when capacity is unused. If one of those points is not stated, obtain a written answer rather than assuming that “reserved” means no cost when the resource is idle.

Spot: build recovery before requesting capacity

Spot is a different purchasing relationship. AWS states that Spot Instances are not covered by Savings Plans. When capacity is unavailable at launch, an On-Demand request can fail with an insufficient capacity error, while a Spot request keeps retrying. Spot pricing is set for each instance type in each Availability Zone and adjusts gradually with long-term supply and demand; the EC2 Spot Instances documentation contains those rules.

Interruption handling must be treated as part of the workload, not an exception added after launch. According to Spot Instance interruption notices, EC2 issues a notice two minutes before stopping or terminating an instance, but notices are delivered on a best-effort basis. AWS recommends checking for them every 5 seconds.

The documented IMDSv2 path is:

http://169.254.169.254/latest/meta-data/spot/instance-action

Use it as part of a documented interruption procedure:

  • Make the workload restartable and define where its recoverable state is stored.
  • Check the metadata path at the documented interval.
  • Define the action to take when replacement capacity is unavailable.
  • Test the fallback path separately from normal worker startup.
  • Keep retry and recovery work visible in the operational cost calculation.

A notice can trigger a handoff, but it cannot by itself complete recovery. The application must already know how to resume or fail over.

Build a total-cost worksheet from the quote

Do not compare only the displayed compute rate. On the provider’s current page and your quote, check:

  • The billing basis: runtime, usage commitment, configuration commitment, or capacity reservation.
  • Any minimum billing duration.
  • The covered Region, Availability Zone, instance configuration, or cluster size.
  • Payment, expiry, and cancellation terms.
  • Charges that may apply while capacity is unused or an instance is stopped.
  • Interruption notice and recovery terms.
  • The SLA architecture, claim deadline, and remedy.

Add every applicable billed item, not only GPU compute:

TOTAL_PROVIDER_COST =
  YOUR_COMPUTE_OR_COMMITMENT_COST
  + YOUR_STORAGE_COST
  + YOUR_ADDRESSING_COST
  + YOUR_DATA_TRANSFER_COST
  + YOUR_OTHER_BILLED_ITEMS

Then calculate the effective rate using the provider’s actual billing record:

EFFECTIVE_HOURLY_COST =
  TOTAL_PROVIDER_COST / YOUR_ACTUAL_BILLABLE_HOURS

For an internal decision, keep retry and service-impact costs outside the provider bill but visible in the comparison:

DECISION_COST =
  YOUR_PROVIDER_BILL
  + YOUR_RETRY_OR_FAILOVER_COST
  + YOUR_OPERATIONAL_DELAY_COST

Fill every field from your quote, bill, or internal cost record. Compare all options using the same workload assumptions and location. A commitment should not appear cheaper merely because unused capacity is omitted, and a Spot option should not appear cheaper if recovery work is excluded.

Check the SLA separately from the purchase method

The Amazon Compute Service Level Agreement defines two EC2 commitments. The Region-Level commitment is 99.99% monthly uptime when all running instances are deployed across two or more Availability Zones in the same Region. The Instance-Level commitment is 99.5% monthly uptime for a single instance.

The same SLA requires a claim to be submitted through AWS Support by the end of the second billing cycle after the incident. It calculates service credits as a percentage of the monthly bill, excludes one-time upfront payments such as Reserved Instance upfront fees, and applies credits against future EC2 payments rather than refunds. The agreement also identifies service credits as its sole and exclusive remedy for the failures it covers.

Read the SLA against the architecture you actually intend to deploy. A percentage alone does not show whether your placement, claim timing, or billing arrangement meets its conditions. This is general information, not legal advice.

Verify the choice before the first launch

The most common errors appear at the handoff between workload design and purchasing:

  • GPU fit: Confirm the model, runtime, GPU count, and memory requirement separately from the purchase option.
  • On-Demand quota: Check the current vCPU quota before assuming the GPU family can launch.
  • Reserved scope: Record whether the offer commits usage, a configuration, or a GPU cluster.
  • Capacity Reservation: Include the reservation charge in the total even when the forecast leaves capacity unused.
  • Spot recovery: Verify the metadata path, retry behavior, and fallback procedure before moving a required workload onto interruptible capacity.
  • Total cost: Replace every worksheet field with a current quote or billing value.
  • SLA: Save the applicable commitment, claim route, and deadline with the deployment record.

These steps are compiled from the official documentation linked in the text and should be checked against the current version before use.

Sources