- vLLM cheat sheet: vllm serve flags, OpenAI-compatible endpoints, health checks, metrics and benchmarksAI & GenAI · 11 Nov 2026
- Runbook: rolling out a new vLLM version or model on Kubernetes without dropping in-flight streamsAI & GenAI · 9 Nov 2026
- Monitor vLLM in production: Prometheus metrics, PromQL and alerts for queue, TTFT and KV cacheAI & GenAI · 6 Nov 2026
- GPU inference node out of disk: model caches, huge images and ephemeral-storage evictionsAI & GenAI · 2 Nov 2026
- Introduction to GPU scheduling on Kubernetes for LLM inference: requests, taints, node selectors and sharingAI & GenAI · 2 Nov 2026
- Kubernetes GPU node shows 0 allocatable nvidia.com/gpu: device plugin, drivers, container runtime and Xid errorsAI & GenAI · 2 Nov 2026
- Autoscaling self-hosted LLMs on Kubernetes: KEDA on queue depth and KV cache, plus GPU node autoscalingAI & GenAI · 29 Oct 2026
- Checklist: upgrading NVIDIA GPU drivers on Kubernetes inference node pools without an outageAI & GenAI · 29 Oct 2026
- GPU basics for LLM inference: VRAM, model size math, precision and which GPU you actually needAI & GenAI · 29 Oct 2026
- nvidia-smi cheat sheet for GPU inference nodes: inventory, memory, processes, clocks, errors and topologyAI & GenAI · 29 Oct 2026
- Sev1: spot GPU nodes were reclaimed at once and the self-hosted inference service went downAI & GenAI · 29 Oct 2026
- Sev1: vLLM pods crash-loop with CUDA out of memory and the self-hosted LLM endpoint is downAI & GenAI · 29 Oct 2026
- vLLM memory tuning in depth: KV cache sizing, max-model-len, max-num-seqs and preemptionAI & GenAI · 29 Oct 2026
- LLM pods take 20 minutes to become ready: slow model weight downloads and loading on Kubernetes, and how to cut cold startsAI & GenAI · 28 Oct 2026
- Multi-GPU inference hangs or dies with NCCL timeouts: shared memory, P2P, topology and network interface problemsAI & GenAI · 28 Oct 2026
- Serving many LoRA adapters on one vLLM server: adapter load errors, rank limits, latency spikes and memory sizingAI & GenAI · 28 Oct 2026
- Monitoring GPU inference nodes with DCGM Exporter and Prometheus: the metrics and alerts that matterAI & GenAI · 27 Oct 2026
- NVIDIA GPU Operator on Kubernetes - installing it, checking it works and fixing pods that can't see the GPUAI & GenAI · 26 Oct 2026
- Ollama is slow or running on CPU instead of GPU: model placement, context size and reloadsAI & GenAI · 26 Oct 2026
- Triton Inference Server model won't load - fixing repository layout, config.pbtxt, backends and GPU memoryAI & GenAI · 26 Oct 2026
- Optimization: getting more from GPUs when fine-tuning - mixed precision, batch sizing, data loading and checkpointsAI & GenAI · 25 Oct 2026
- Optimization: sharing GPUs on Kubernetes - MIG, time-slicing and bin packing for small modelsAI & GenAI · 25 Oct 2026
- Optimization: vLLM throughput tuning - batching limits, prefix caching, chunked prefill and parallelismAI & GenAI · 25 Oct 2026
- Sev1: GPU nodes won't scale up. AllocationFailed and InsufficientInstanceCapacity during peak loadAI & GenAI · 25 Oct 2026
- Sev1: GPU Xid errors and 'GPU has fallen off the bus'. Triage for inference and training nodesAI & GenAI · 25 Oct 2026
- Sev1: inference pods stuck Pending with 'Insufficient nvidia.com/gpu'AI & GenAI · 25 Oct 2026
- Transformation: moving self-hosted LLM inference from FP16 to FP8 or INT4 quantized modelsAI & GenAI · 25 Oct 2026
- GPU capacity and model quota planning: sizing, reservations and headroomAI & GenAI · 22 Oct 2026
- Highly available self-hosted LLM inference on KubernetesAI & GenAI · 22 Oct 2026
- NVIDIA driver and CUDA version mismatch: GPU not detected in VMs and containersAI & GenAI · 22 Oct 2026
- PyTorch 'CUDA out of memory': find what fills the GPU and fix itAI & GenAI · 22 Oct 2026
- Running self-hosted LLMs with vLLM: GPU memory, KV cache and the metrics that matterAI & GenAI · 22 Oct 2026
- Standalone LLM server on one GPU with vLLM: install, serve, secure and measureAI & GenAI · 22 Oct 2026