- Checklist: vetting open-weight models before you serve them - safetensors, pinned revisions, hashes, remote code and a private mirrorAI & GenAI · 11 Nov 2026
- vLLM cheat sheet: vllm serve flags, OpenAI-compatible endpoints, health checks, metrics and benchmarksAI & GenAI · 11 Nov 2026
- Amazon SageMaker real-time endpoints CLI cheat sheet: status, test calls, variants, autoscaling, logs and metricsAI & GenAI · 9 Nov 2026
- Hugging Face Text Embeddings Inference (TEI) cheat sheet: images, launch flags, batching limits, API calls and metricsAI & GenAI · 9 Nov 2026
- Runbook: rolling out a new vLLM version or model on Kubernetes without dropping in-flight streamsAI & GenAI · 9 Nov 2026
- vLLM structured outputs: requests that are slow, rejected with 400, or still return broken JSONAI & GenAI · 9 Nov 2026
- Monitor vLLM in production: Prometheus metrics, PromQL and alerts for queue, TTFT and KV cacheAI & GenAI · 6 Nov 2026
- Hugging Face hf CLI cheat sheet: download, verify and clean up model cachesAI & GenAI · 5 Nov 2026
- Azure Machine Learning online endpoints CLI cheat sheet: deployments, traffic, scaling, logs, metrics and errorsAI & GenAI · 4 Nov 2026
- GPU inference node out of disk: model caches, huge images and ephemeral-storage evictionsAI & GenAI · 2 Nov 2026
- Introduction to GPU scheduling on Kubernetes for LLM inference: requests, taints, node selectors and sharingAI & GenAI · 2 Nov 2026
- Kubernetes GPU node shows 0 allocatable nvidia.com/gpu: device plugin, drivers, container runtime and Xid errorsAI & GenAI · 2 Nov 2026
- Runbook: blue-green model rollout on Azure Machine Learning online endpoints with mirrored traffic and instant rollbackAI & GenAI · 2 Nov 2026
- Sev1: self-hosted LLM returns gibberish, never stops or leaks special tokens after a deploy — chat template and tokenizer fixesAI & GenAI · 31 Oct 2026
- Checklist: upgrading NVIDIA GPU drivers on Kubernetes inference node pools without an outageAI & GenAI · 29 Oct 2026
- nvidia-smi cheat sheet for GPU inference nodes: inventory, memory, processes, clocks, errors and topologyAI & GenAI · 29 Oct 2026
- LLM pods take 20 minutes to become ready: slow model weight downloads and loading on Kubernetes, and how to cut cold startsAI & GenAI · 28 Oct 2026
- Multi-GPU inference hangs or dies with NCCL timeouts: shared memory, P2P, topology and network interface problemsAI & GenAI · 28 Oct 2026
- Same prompt, different answer at temperature 0: where LLM non-determinism comes from and how to live with itAI & GenAI · 28 Oct 2026
- Serving many LoRA adapters on one vLLM server: adapter load errors, rank limits, latency spikes and memory sizingAI & GenAI · 28 Oct 2026
- Monitoring GPU inference nodes with DCGM Exporter and Prometheus: the metrics and alerts that matterAI & GenAI · 27 Oct 2026
- NVIDIA GPU Operator on Kubernetes - installing it, checking it works and fixing pods that can't see the GPUAI & GenAI · 26 Oct 2026
- Triton Inference Server model won't load - fixing repository layout, config.pbtxt, backends and GPU memoryAI & GenAI · 26 Oct 2026
- Advisory: choosing a self-hosted LLM serving stack - vLLM, SGLang, TensorRT-LLM/Triton, TGI, Ollama or llama.cppAI & GenAI · 25 Oct 2026
- Optimization: speculative decoding for self-hosted LLMs - faster generation without changing the outputAI & GenAI · 25 Oct 2026
- Optimization: vLLM throughput tuning - batching limits, prefix caching, chunked prefill and parallelismAI & GenAI · 25 Oct 2026
- Sev1: managed model endpoint times out after idle periods or scale-out (Azure ML, SageMaker)AI & GenAI · 25 Oct 2026
- Sev1: time to first token jumps from 300 ms to 20 seconds on self-hosted LLM inferenceAI & GenAI · 25 Oct 2026
- Transformation: moving models from hand-built GPU VMs to managed inference endpointsAI & GenAI · 25 Oct 2026
- Hugging Face model download fails: 401, 403 gated repos, timeouts and offline modeAI & GenAI · 24 Oct 2026
- PyTorch 'CUDA out of memory': find what fills the GPU and fix itAI & GenAI · 22 Oct 2026
- Running self-hosted LLMs with vLLM: GPU memory, KV cache and the metrics that matterAI & GenAI · 22 Oct 2026
- Standalone LLM server on one GPU with vLLM: install, serve, secure and measureAI & GenAI · 22 Oct 2026