Karpenter Spot On Demand Fallback Your Workload Can Survive

Karpenter Spot On Demand fallback needs three independent gates. These are proposed engineering acceptance criteria, not Karpenter features. A mixed-capacity NodePool does not establish that your workload can survive losing a worker.

1. Eligibility: does worker loss preserve acknowledged data, keep customer impact within the agreed SLO and restore useful throughput before an agreed deadline?
2. Baseline: what capacity must already exist to maintain safe service during that loss?
3. Fallback: how much additional On-Demand capacity may be provisioned, for how long and at what financial exposure?

Passing one gate does not satisfy the others. A restartable workload can still need a protected baseline; permission to use Spot is not permission for unlimited On-Demand spending.

Reject single-copy local state and uncheckpointed jobs whose replay exceeds their deadline. Also reject services whose minimum safe throughput requires replacement capacity to arrive immediately.

Treat node-local scratch storage as disposable. Correctly configured persistent storage can preserve data, but persistence alone proves neither availability nor recovery speed. Volume attachment, scheduling and application recovery still take time.

Hypothetical incident: a consumer loses its worker with a message in flight. Its replacement starts after 45 seconds, but useful throughput for that message remains stalled until the old five-minute lease expires. A Ready pod is not proof of recovery.

Karpenter Spot interruption handling cannot make warning delivery guaranteed. AWS documents interruption notices as best effort; hibernation starts immediately, without the usual two-minute preparation window. Eligibility must therefore cover abrupt worker loss, not just a graceful shutdown rehearsal.

Keep the Karpenter controller on independent On-Demand managed nodes, or suitable Fargate capacity. It must survive losing every Spot worker.

Prerequisites: working controller IAM, node access, subnet discovery and a Ready EC2NodeClass named production. Its amiSelectorTerms must use an explicit AMI ID or versioned alias, never @latest. These are prerequisites, not installation steps hidden inside a NodePool example.

Pin the controller and CRDs together

Use Karpenter 1.15.1 for both charts. For existing Helm-managed releases named karpenter and karpenter-crd, retain their configured values:

export KARPENTER_VERSION=1.15.1

helm upgrade karpenter-crd \
  oci://public.ecr.aws/karpenter/karpenter-crd \
  --namespace kube-system \
  --version "${KARPENTER_VERSION}" --reuse-values --wait

helm upgrade karpenter \
  oci://public.ecr.aws/karpenter/karpenter \
  --namespace kube-system \
  --version "${KARPENTER_VERSION}" --reuse-values --wait

helm list -n kube-system

kubectl -n kube-system get deployment karpenter \
  -o jsonpath='{range .spec.template.spec.containers[*]}{.name}{"\t"}{.image}{"\n"}{end}'

kubectl wait --for=condition=Ready \
  ec2nodeclass/production --timeout=120s

Check both chart versions and the deployed controller image; the intended Helm version is not proof of the running version.

Separate baseline from burst

This Karpenter On-Demand baseline maintains one node per London AZ. Static pools maintain node counts and are not consolidated. The two-node limit per pool deliberately leaves replacement headroom, rather than requesting two permanent nodes.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: baseline-${AZ_SUFFIX}
spec:
  replicas: 1
  limits:
    nodes: 2
  template:
    metadata:
      labels:
        capacity.cloudlineconsulting.com/lane: baseline
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: production
      taints:
        - key: capacity.cloudlineconsulting.com/lane
          value: baseline
          effect: NoSchedule
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand"]
        - key: node.kubernetes.io/instance-type
          operator: In
          values: ["m6i.xlarge"]
        - key: topology.kubernetes.io/zone
          operator: In
          values: ["${AZ}"]

Save as baseline.yaml; render once per AZ:

for suffix in a b c; do
  export AZ_SUFFIX="${suffix}" AZ="eu-west-2${suffix}"
  envsubst '${AZ_SUFFIX} ${AZ}' < baseline.yaml | kubectl apply -f -
done

The separate Karpenter mixed capacity NodePool prefers Spot but permits On-Demand. It guarantees neither available supply nor a fixed Spot percentage.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: burst
spec:
  limits:
    cpu: "40"
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 5m
    budgets:
      - nodes: "1"
  template:
    metadata:
      labels:
        capacity.cloudlineconsulting.com/lane: burst
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: production
      taints:
        - key: capacity.cloudlineconsulting.com/lane
          value: burst
          effect: NoSchedule
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: kubernetes.io/os
          operator: In
          values: ["linux"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["4"]
        - key: karpenter.k8s.aws/instance-cpu
          operator: In
          values: ["4", "8"]
        - key: topology.kubernetes.io/zone
          operator: In
          values: ["eu-west-2a", "eu-west-2b", "eu-west-2c"]

The 40-vCPU limit covers both capacity types. It is an eventually consistent resource guardrail, not a hard currency cap or separate fallback allowance. Concurrent provisioning can overshoot it; see Karpenter’s NodePool limit semantics.

Make workload placement explicit

Critical minimum replicas and opportunistic replicas need separate workload groups, usually separate Deployments. Use these Pod-spec fragments respectively:

spec:
  nodeSelector:
    capacity.cloudlineconsulting.com/lane: baseline
  tolerations:
    - key: capacity.cloudlineconsulting.com/lane
      operator: Equal
      value: baseline
      effect: NoSchedule
---
spec:
  nodeSelector:
    capacity.cloudlineconsulting.com/lane: burst
  tolerations:
    - key: capacity.cloudlineconsulting.com/lane
      operator: Equal
      value: burst
      effect: NoSchedule

Tolerating baseline nodes alone does not reserve them. Never add a hard Spot selector to fallback-eligible Pods: that blocks Karpenter Spot On-Demand fallback.

Spread critical replicas across zones using topology spread constraints. Hard spread rules can leave Pods Pending during AZ shortages. Size against remaining allocatable capacity after DaemonSets and existing requests, not the baseline’s nominal 12 vCPUs. Leave rollout headroom too; zero-downtime deployment capacity planning covers that trade-off within broader EKS architecture and platform engineering.

The warning path is: EC2 interruption event → Amazon EventBridge rule → cluster-specific Amazon SQS queue → Karpenter controller → concurrent node draining and replacement provisioning → application readiness.

That path does not extend EC2’s reclamation deadline. For terminate/stop interruptions, the warning normally gives two minutes; queue delays consume that window. A rebalance recommendation is a separate signal, not an interruption warning.

Installing Karpenter proves nothing about event delivery. Karpenter Spot interruption handling requires queue infrastructure and the controller’s interruption-queue setting.

Inspect the deployed wiring

Set these names to match your deployed resources. Run against the intended cluster context.

export AWS_DEFAULT_REGION=eu-west-2
export CLUSTER_NAME=production-eks
export QUEUE_NAME="$CLUSTER_NAME"
export RULE="${CLUSTER_NAME}-spot-interruption"

helm get values karpenter -n kube-system -a
kubectl get deployment karpenter -n kube-system \
  -o jsonpath='{.spec.template.spec.containers[?(@.name=="controller")].env}'

QUEUE_URL=$(aws sqs get-queue-url \
  --queue-name "$QUEUE_NAME" --query QueueUrl --output text)

aws sqs get-queue-attributes --queue-url "$QUEUE_URL" \
  --attribute-names QueueArn Policy ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisible

aws events describe-rule --name "$RULE"
aws events list-targets-by-rule --rule "$RULE"

• Controller: settings.interruptionQueue or INTERRUPTION_QUEUE must identify this queue.
• Rule: expect "source":["aws.ec2"] and "detail-type":["EC2 Spot Instance Interruption Warning"]. Confirm it is enabled and targets the queue ARN.
• Permissions: check the actual controller identity, including its IRSA or EKS Pod Identity role. It needs sqs:ReceiveMessage, sqs:DeleteMessage and sqs:GetQueueUrl on the intended queue. The queue policy must permit EventBridge delivery.

The version-tagged upstream infrastructure template contains these permissions and event patterns; compare against your installed release. If adopting existing infrastructure, import the queue into Terraform safely rather than accidentally replacing it.

Prove delivery during the experiment

Correlate EventBridge delivery metrics, SQS NumberOfMessagesSent/NumberOfMessagesDeleted, Kubernetes node events and controller logs. Align timestamps; use instance IDs in logs and events, since queue metrics are aggregate.

An empty queue proves nothing: the controller may have consumed everything, or nothing arrived. Do not manually receive production messages: you compete with the controller and can delay the warning.

Start in an isolated, production-like environment under representative load. Name an owner, open the recovery runbook and agree a one-worker blast radius. Keep continuous public-endpoint probes running throughout; a healthy dashboard is not enough.

Pin the target and the safety envelope

Resolve the selected node’s providerID, then verify its EC2 lifecycle and cluster ownership. Only then add a unique experiment tag. COUNT(1) limits selection; it does not excuse tagging the wrong machine.

# Run in Bash with AWS CLI v2, kubectl and jq.
set -euo pipefail
export AWS_REGION="$(aws configure get region)"
ACCOUNT="$(aws sts get-caller-identity --query Account --output text)"
PARTITION="$(aws sts get-caller-identity --query Arn --output text | cut -d: -f2)"
read -rp "EKS cluster name: " CLUSTER
read -rp "Selected Spot node: " NODE

# Confirm kubectl points at this EKS cluster.
test "$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')" = \
  "$(aws eks describe-cluster --name "$CLUSTER" \
      --query cluster.endpoint --output text)"

PROVIDER_ID="$(kubectl get node "$NODE" -o jsonpath='{.spec.providerID}')"
INSTANCE_ID="${PROVIDER_ID##*/}"
[[ "$INSTANCE_ID" =~ ^i-[0-9a-f]+$ ]]

INSTANCE="$(aws ec2 describe-instances --instance-ids "$INSTANCE_ID" \
  --query 'Reservations[0].Instances[0]' --output json)"

jq -e --arg cluster "$CLUSTER" '
  .InstanceLifecycle == "spot" and
  .State.Name == "running" and
  any(.Tags[]?;
    (.Key == ("kubernetes.io/cluster/" + $cluster) and .Value == "owned")
    or (.Key == "eks:cluster-name" and .Value == $cluster))
' <<<"$INSTANCE"

RUN="spot-fis-$(date -u +%Y%m%dT%H%M%SZ)-$(openssl rand -hex 4)"
aws ec2 create-tags --resources "$INSTANCE_ID" \
  --tags "Key=cloudline:fis-run,Value=$RUN"

Proposed policy: stop when public-request errors exceed 1% over one 60-second period, or a critical queue’s oldest message exceeds its agreed limit. These commands assume the probe publishes ErrorPercent every minute, including zeroes, for this isolated lane. Require both alarms to be OK before starting.

read -rp "Critical SQS queue name: " QUEUE
read -rp "Agreed oldest-message limit, seconds: " QUEUE_LIMIT
read -rp "Probe metric Lane dimension: " LANE
[[ "$QUEUE_LIMIT" =~ ^[0-9]+$ ]]

ERROR_ALARM="$RUN-public-errors"
QUEUE_ALARM="$RUN-queue-age"

aws cloudwatch put-metric-alarm \
  --alarm-name "$ERROR_ALARM" \
  --namespace Cloudline/RecoveryTest --metric-name ErrorPercent \
  --dimensions "Name=Lane,Value=$LANE" \
  --statistic Average --period 60 --evaluation-periods 1 \
  --datapoints-to-alarm 1 --threshold 1 \
  --comparison-operator GreaterThanThreshold \
  --treat-missing-data breaching

aws cloudwatch put-metric-alarm \
  --alarm-name "$QUEUE_ALARM" \
  --namespace AWS/SQS --metric-name ApproximateAgeOfOldestMessage \
  --dimensions "Name=QueueName,Value=$QUEUE" \
  --statistic Maximum --period 60 --evaluation-periods 1 \
  --datapoints-to-alarm 1 --threshold "$QUEUE_LIMIT" \
  --comparison-operator GreaterThanThreshold \
  --treat-missing-data breaching

aws cloudwatch describe-alarms \
  --alarm-names "$ERROR_ALARM" "$QUEUE_ALARM" \
  --query 'MetricAlarms[].{Alarm:AlarmName,State:StateValue}' \
  --output table

Create the bounded experiment

Use a dedicated FIS role, not the node role. The following creates spot-worker-fis.json: the complete, reusable experiment template, with actual alarm ARNs resolved from AWS. Save that file with the test record.

ROLE_NAME="$RUN-role"
ROLE_ARN="arn:$PARTITION:iam::$ACCOUNT:role/$ROLE_NAME"

cat > fis-trust.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "fis.amazonaws.com"},
    "Action": "sts:AssumeRole",
    "Condition": {
      "StringEquals": {"aws:SourceAccount": "$ACCOUNT"},
      "ArnLike": {
        "aws:SourceArn": "arn:$PARTITION:fis:$AWS_REGION:$ACCOUNT:experiment/*"
      }
    }
  }]
}
EOF

aws iam create-role --role-name "$ROLE_NAME" \
  --assume-role-policy-document file://fis-trust.json

jq -n --arg run "$RUN" \
  --arg resource "arn:$PARTITION:ec2:$AWS_REGION:$ACCOUNT:instance/*" '{
  Version: "2012-10-17",
  Statement: [
    {
      Effect: "Allow",
      Action: "ec2:SendSpotInstanceInterruptions",
      Resource: $resource,
      Condition: {"StringEquals": {"ec2:ResourceTag/cloudline:fis-run": $run}}
    },
    {
      Effect: "Allow",
      Action: [
        "ec2:DescribeInstances",
        "ec2:DescribeSpotInstanceRequests",
        "cloudwatch:DescribeAlarms"
      ],
      Resource: "*"
    }
  ]
}' > fis-permissions.json

aws iam put-role-policy --role-name "$ROLE_NAME" \
  --policy-name BoundedSpotInterruption \
  --policy-document file://fis-permissions.json

ERROR_ARN="$(aws cloudwatch describe-alarms \
  --alarm-names "$ERROR_ALARM" \
  --query 'MetricAlarms[0].AlarmArn' --output text)"
QUEUE_ARN="$(aws cloudwatch describe-alarms \
  --alarm-names "$QUEUE_ALARM" \
  --query 'MetricAlarms[0].AlarmArn' --output text)"

jq -n --arg run "$RUN" --arg role "$ROLE_ARN" \
  --arg errors "$ERROR_ARN" --arg queue "$QUEUE_ARN" '{
  description: "Interrupt one explicitly tagged Spot worker",
  roleArn: $role,
  stopConditions: [
    {source: "aws:cloudwatch:alarm", value: $errors},
    {source: "aws:cloudwatch:alarm", value: $queue}
  ],
  targets: {
    OneSpotWorker: {
      resourceType: "aws:ec2:spot-instance",
      resourceTags: {"cloudline:fis-run": $run},
      filters: [{path: "State.Name", values: ["running"]}],
      selectionMode: "COUNT(1)"
    }
  },
  actions: {
    InterruptWorker: {
      actionId: "aws:ec2:send-spot-instance-interruptions",
      parameters: {durationBeforeInterruption: "PT2M"},
      targets: {SpotInstances: "OneSpotWorker"}
    }
  },
  tags: {"cloudline:fis-run": $run}
}' > spot-worker-fis.json

TEMPLATE_ID="$(aws fis create-experiment-template \
  --cli-input-json file://spot-worker-fis.json \
  --query experimentTemplate.id --output text)"

# Allow IAM propagation before starting. Abort unless both alarms are healthy.
aws cloudwatch describe-alarms \
  --alarm-names "$ERROR_ALARM" "$QUEUE_ALARM" --output json |
  jq -e '.MetricAlarms | length == 2 and all(.[]; .StateValue == "OK")'

EXPERIMENT_ID="$(aws fis start-experiment \
  --experiment-template-id "$TEMPLATE_ID" \
  --query experiment.id --output text)"

aws fis get-experiment --id "$EXPERIMENT_ID"

# Emergency stop: run from a second terminal with this experiment ID.
aws fis stop-experiment --id "$EXPERIMENT_ID"

The operator needs permission to pass this dedicated role. Use the documented Spot interruption action, not kubectl delete node: deletion does not test AWS warning delivery. Stop conditions halt the experiment; they cannot undo termination already performed.

Keep the two results separate

• Run A — interruption handling: leave the Karpenter mixed capacity NodePool unchanged. A replacement Spot worker is a legitimate result.
• Run B — forced On-Demand recovery: first create an isolated burst-pool copy with a unique lane label and matching NoSchedule test taint. Only test workloads select and tolerate that lane. Provision its Spot worker before changing requirements.

For Run B, save the pool, freeze voluntary disruption, then make capacity On-Demand-only. Check that no expiry or other forceful disruption is due. Reconfirm the original Spot worker still exists before selecting and tagging it.

read -rp "Isolated Run B NodePool name: " TEST_POOL

kubectl get nodepool "$TEST_POOL" -o json |
  jq 'del(.status, .metadata.managedFields, .metadata.resourceVersion,
          .metadata.uid, .metadata.creationTimestamp, .metadata.generation)' \
  > run-b-pool.saved.json

kubectl patch nodepool "$TEST_POOL" --type merge \
  -p '{"spec":{"disruption":{"budgets":[{"nodes":"0"}]}}}'

kubectl get nodepool "$TEST_POOL" -o json |
  jq '{
    spec: {
      template: {
        spec: {
          requirements: (
            [.spec.template.spec.requirements[]
             | select(.key != "karpenter.sh/capacity-type")]
            + [{
                key: "karpenter.sh/capacity-type",
                operator: "In",
                values: ["on-demand"]
              }]
          )
        }
      }
    }
  }' > run-b-requirements.json

kubectl patch nodepool "$TEST_POOL" --type merge \
  --patch-file run-b-requirements.json

kubectl get node "$NODE"
aws ec2 describe-instances --instance-ids "$INSTANCE_ID" \
  --query 'Reservations[].Instances[].{ID:InstanceId,Lifecycle:InstanceLifecycle,State:State.Name}'

# Repeat target verification and the experiment with fresh tags and alarms.
# Afterwards, including after an aborted test:
kubectl apply -f run-b-pool.saved.json

Label Run B forced On-Demand recovery, not proof that Spot scarcity triggered native fallback. FIS insufficient-capacity API injection is not a universal Spot-only switch: documented request and multi-AZ limitations prevent treating it as selective Spot exhaustion.

A Ready node is not recovered service. For Karpenter Spot on demand fallback, measure useful throughput and correct results. Separate Pod readiness from externally healthy traffic; an ALB can still be waiting for healthy targets.

Record separate UTC fields for FIS experiment start, EC2 warning emission and controller warning receipt. The FIS action also emits a rebalance recommendation: experiment start is not the interruption-warning timestamp.

Recovery worksheet

Fill timestamps from evidence, not estimates. Record overlapping intervals; never add parallel provisioning and draining stages. Unspecified limits below are diagnostic checkpoints, not automatic passes.

• Service recovery: first customer-impacting error to sustained SLO compliance over an agreed observation window.
• Replacement lead time: warning receipt to useful replacement capacity.
• Replay recovery: interruption until the queue returns to its agreed steady state.

Record separate inputs for checkpoint age, in-flight work, lease timeout and duplicate side effects. Proposed pass criteria also require minimum baseline throughput throughout. Stricter workload SLOs require stricter limits.

Find the stalled stage

kubectl describe pod -n payments POD_NAME
kubectl get nodeclaims -o yaml
kubectl describe node NODE_NAME
kubectl logs -n kube-system deployment/karpenter --since=15m

Check scheduling mismatches first, then NodeClaims and controller logs for EC2 capacity or quota failures. Pod and node events expose image-pull and storage-attachment delays. Investigate readiness next, then message replay if capacity is healthy but backlog persists.

PodDisruptionBudgets and NodePool disruption budgets do not prevent EC2 reclamation. Long application grace periods cannot extend the underlying instance’s life.

Repeat across eligible AZs and instance shapes. Report every observation and the worst result; a tiny sample does not justify a percentile.

Set the budget before testing Karpenter Spot on demand fallback. A successful recovery can still leave an expensive fleet running. Treat fallback hours as an operational metric, alongside recovery time.

Capture London rates, not a headline discount

Run this worksheet with AWS CLI and jq. It records its actual retrieval timestamp, preserves source responses and extracts hourly On-Demand USD dimensions. Spot prices remain separate by Availability Zone; there is no single “London Spot price”.

SNAPSHOT_UTC=$(date -u +%FT%TZ)
mkdir -p "rates-$SNAPSHOT_UTC"
cd "rates-$SNAPSHOT_UTC"
printf '%s\n' "$SNAPSHOT_UTC" > retrieved-utc.txt

aws pricing get-products --region us-east-1 --service-code AmazonEC2 \
  --filters \
    Type=TERM_MATCH,Field=regionCode,Value=eu-west-2 \
    Type=TERM_MATCH,Field=instanceType,Value=m6i.xlarge \
    Type=TERM_MATCH,Field=operatingSystem,Value=Linux \
    Type=TERM_MATCH,Field=tenancy,Value=Shared \
    Type=TERM_MATCH,Field=preInstalledSw,Value=NA \
    Type=TERM_MATCH,Field=capacitystatus,Value=Used \
  --output json > ondemand-source.json

jq --arg retrieved "$SNAPSHOT_UTC" '
  .PriceList[] | fromjson |
  . as $p | .terms.OnDemand[][] |
  . as $term | .priceDimensions[] |
  select(.unit == "Hrs") |
  {
    retrievedUTC: $retrieved,
    publicationDate: $p.publicationDate,
    effectiveDate: $term.effectiveDate,
    region: $p.product.attributes.regionCode,
    instanceType: $p.product.attributes.instanceType,
    operatingSystem: $p.product.attributes.operatingSystem,
    tenancy: $p.product.attributes.tenancy,
    preInstalledSw: $p.product.attributes.preInstalledSw,
    capacitystatus: $p.product.attributes.capacitystatus,
    description: .description,
    USD_per_hour: .pricePerUnit.USD
  }' ondemand-source.json > ondemand-rates.json

aws ec2 describe-spot-price-history --region eu-west-2 \
  --instance-types m6i.xlarge \
  --product-descriptions 'Linux/UNIX' \
  --start-time "$SNAPSHOT_UTC" \
  --output json > spot-source.json

jq --arg retrieved "$SNAPSHOT_UTC" '
  [.SpotPriceHistory[] | {
    retrievedUTC: $retrieved,
    effectiveUTC: .Timestamp,
    region: "eu-west-2",
    AZ: .AvailabilityZone,
    instanceType: .InstanceType,
    productDescription: .ProductDescription,
    USD_per_hour: .SpotPrice
  }]' spot-source.json > spot-rates.json

The Pricing filters explicitly select Linux/shared tenancy. The Spot query selects Linux/UNIX; its response does not expose a tenancy field. Keep both source files with the worksheet.

Price the fallback envelope

These are modelling assumptions, not live AWS quotations: P_od=$0.22 and P_spot=$0.07 per node-hour. Set H=730 hours, three baseline nodes, ten equivalent burst nodes and F=fallback fleet-hours. Equal-size equivalents simplify this example; the heterogeneous YAML requires actual node-hour costs by instance type.

C = 3×H×P_od
  + 10×[(H−F)×P_spot + F×P_od]
  + C_replay + C_operations + C_other

Twenty-four fleet-hours means 240 burst node-hours. Fifty replay node-hours at the assumed Spot rate add $3.50. Separately assume $300 incident handling: the 24-hour scenario becomes $1,332.30 before other costs, saving $755.50 against compute-only all On-Demand.

Record EBS, EKS, ALB, NAT, data transfer, FIS and observability as additional or shared costs. This simplified comparison excludes VAT and commitments. Keep these visible in your cloud cost review.

Alert at 24 fallback fleet-hours; escalate when projected savings fall below the team’s minimum. A CPU limit is not a spend cap. Do not assume automatic return to Spot on a fixed timer.

For strict separate fallback control, use separately bounded Spot and On-Demand pools rather than one Karpenter mixed capacity NodePool. Retest placement. Hitting the fallback ceiling means accepting Pending Pods.

Cloudline offers a fixed scope: one EKS cluster, two representative workloads, baseline and placement review, queue-path audit, one interruption test, one forced On-Demand recovery test, measured recovery worksheet, cost model and go/no-go report. Discuss a Spot-readiness and failure-testing engagement.

Spot is an operating decision, not a discount setting. Approve workloads against data-loss and recovery limits, keep baseline capacity separate, and make fallback permission explicit. Then interrupt a worker, observe useful recovery and price the hours spent on On-Demand.

If those boundaries are still assumptions, Cloudline Consulting offers a fixed-scope Spot-readiness and failure-testing engagement: configuration review, bounded experiments, recovery evidence and a fallback cost model. Leave with a go/no-go decision and an owned runbook, not another untested NodePool.