AKS Node Pool Strategy: System/User/Spot/GPU/Windows Pools and Cost Optimization

An AKS Node Pool is a group of identically configured worker nodes (Linux or Windows VMs). A single AKS cluster typically hosts multiple node pools, separated by purpose (System, User, GPU, Spot). The System Node Pool is mandatory and runs kube-system namespace pods (CoreDNS, konnectivity, metrics-server, etc.), while User Node Pools run application pods. Each node pool can be configured independently for VM size, OS, autoscaling, taints, and labels, and pod placement is controlled via Node Affinity / Tolerations. In enterprise AKS, node pool design has a major impact on operational efficiency, cost, and performance.

System Node Pool: runs kube-system pods (CoreDNS, konnectivity, metrics-server, azure-cni-pod, etc.) and handles AKS management functions. Taint: CriticalAddonsOnly=true:NoSchedule (excludes app pods). Minimum 1 node (3 recommended in production for AZ redundancy). Use Standard VMs (D2s_v5 or higher) rather than Burstable (B2s_v3). User Node Pool: runs application pods with customizable taints, can be created multiple times by purpose (Spot, GPU, Memory-optimized), scales via Cluster Autoscaler, and now supports Auto Provisioning (Karpenter equivalent). The standard pattern for new AKS clusters is one System pool (D4s_v5 × 3 nodes, AZ-redundant) plus multiple User pools.

Spot Node Pools use Azure Spot VMs as nodes, cutting costs by up to 90%. Design considerations: 1) Spot nodes can be evicted at any time — pods need a Toleration for 'kubernetes.azure.com/scalesetpriority=spot:NoSchedule'; 2) intentionally schedule onto Spot via Node Selector or Node Affinity on 'kubernetes.azure.com/scalesetpriority=spot'; 3) use Delete as the Eviction Policy (not Deallocate); 4) optionally allow Max Price up to the pay-as-you-go rate to reduce eviction frequency. Use cases: batch processing pods (CronJob), stateless workers, ephemeral CI/CD pods. Never use Spot for stateful production workloads (databases, caches) — only stateless, eviction-tolerant workloads belong here.

Cluster Autoscaler (CA): the standard AKS node autoscaler. It detects Pending pods (those that can't be scheduled due to resource shortage) → launches new nodes → schedules the pods, and removes idle nodes. Enabled per node pool with Min/Max node counts. Typical scale-up latency is around 10 minutes. Node Auto Provisioning (NAP, Karpenter-based, GA in 2024): dynamically picks the optimal VM size from pod requirements (CPU/Memory/Affinity) and creates nodes — faster (1–2 min), more efficient, and more cost-optimized. Multiple node pool management is no longer needed. Microsoft adapted the AWS Karpenter project codebase for Azure, and NAP is becoming the default choice for new AKS clusters. That said, NAP is still new, so Cluster Autoscaler knowledge remains important.

GPU Node Pools are used for machine learning, graphics processing, and crypto mining (only if your organization permits it). They use Azure N-series VMs (NCv3, NDv4, ND H100 v5, etc.) as nodes. Microsoft's recommended pattern: 1) install the NVIDIA Device Plugin (the Azure GPU Operator is also an option); 2) request GPUs on pods via resources.limits with nvidia.com/gpu: 1; 3) explicitly schedule onto the GPU pool via Node Selector; 4) use a Spot GPU pool for cost optimization (great for ML training). NDv5 (H100 80GB GPUs, latest generation) is extremely powerful for LLM training and can easily run into millions of yen per month. A quota request is often required, so plan capacity ahead of time.

AKS supports Windows Node Pools alongside Linux ones (the System Node Pool must always be Linux). Workers run on Windows Server 2019 / 2022 and let you containerize .NET Framework apps and MSI-based legacy applications. Caveats: 1) Windows container images are much larger than Linux (several GB), so pull times are longer; 2) pod startup is also slower than Linux; 3) Windows and Linux must be in separate node pools (no mixing); 4) target via Pod Node Selector kubernetes.io/os=windows. This feature shines for legacy Windows app containerization projects, but for new projects the standard best practice is to containerize on Linux.

Standard pattern for production AKS: 1) one System Node Pool (D4s_v5 × 3 nodes, AZ-redundant, CriticalAddonsOnly taint); 2) User Node Pool (Regular): D4s_v5 × 3 nodes, autoscale Min 3 / Max 10, for production Web/API; 3) User Node Pool (Spot): D4s_v5 Spot, autoscale Min 0 / Max 20, for batch workers; 4) GPU Node Pool (when needed): NCsv3-series, autoscale Min 0 / Max 5; 5) Memory-Optimized Node Pool (when needed): E-series for in-memory cache. Roll upgrades pool by pool so the cluster as a whole has zero downtime. Combining AZ placement, Pod Disruption Budgets, and Cluster Autoscaler / NAP delivers both cost optimization and high availability.

AZ-104 (Administrator) covers AKS basics in Domain 3; AZ-204 (Developer Associate, note the July 2026 retirement) covers AKS deployment from a developer perspective; AZ-400 (DevOps Engineer Expert) covers AKS deployment from CI/CD pipelines and GitOps (Argo CD / FluxCD); AZ-305 (Solutions Architect Expert) covers architectural decisions; SC-100 (Cybersecurity Architect Expert) covers AKS security. Outside Microsoft: combining CKA (Certified Kubernetes Administrator), CKAD (Application Developer), and CKS (Security) is a powerful pairing for DevOps engineers. AKS is a core technology on the Azure DevOps engineer career roadmap.