Senior DevOps / Platform Engineer in New Delhi with 7+ years on AWS. I build and run Kubernetes platforms, mostly for GPU inference: scale-to-zero LLM and model serving on EKS, the NVIDIA stack underneath it, and the cost and compliance work around it.

What I've built Scale-to-zero inference on Amazon EKS (vLLM, TEI, Karpenter, KEDA); GPU capacity peaked at 116 NVIDIA GPUs with no idle cost A KV-cache capacity model for sizing vLLM concurrency per GPU; GPU image pulls cut from 94 s to 43–48 s Whisper and gpt-oss served on TensorRT + Triton Infrastructure for HIPAA, SOC 2 and ISO 27001 environments, and three years of PCI-DSS audits A GCP-to-AWS migration that took GCP spend down ~93% in three months

Reported upstream, then fixed or shipped NVIDIA Container Toolkit #2026: a hook failure that broke GPU container creation in v1.20.0; fixed in v1.20.1 Amazon EKS AMI #2790: the AL2023 NVIDIA AMI shipped the toolkit version that broke GPU containers Amazon Linux 2027 #17: the NVIDIA driver guide used package names that don't exist on kernel 7.2; AWS fixed the docs within a day Karpenter Blueprints #24: Kubernetes 1.31 support Sentry self-hosted #1097: the upgrade off PostgreSQL 9.6

Currently exploring GPU nodes on Amazon Linux 2027 with EKS 1.34 and Karpenter (discussion) Package-signature defaults for the NVIDIA container toolkit repo (discussion)

Stack: AWS (EKS, EC2, IAM, VPC) · Kubernetes · Karpenter · KEDA · Terraform · Helm · Argo CD · GitHub Actions · NVIDIA drivers, Container Toolkit, device plugin, DCGM · vLLM · Triton / TensorRT · Prometheus · Grafana · OpenTelemetry · Langfuse · Python · Bash

Certifications: NVIDIA-Certified Associate: AI Infrastructure and Operations · Google Cloud Associate Cloud Engineer

Writing at tanmay.dev · LinkedIn