- Putting Open WebUI in front of vLLM or Ollama: an internal chat UI with sign-in, roles and no open signupAI & GenAI · 12 Nov 2026
- vLLM cheat sheet: vllm serve flags, OpenAI-compatible endpoints, health checks, metrics and benchmarksAI & GenAI · 11 Nov 2026
- Runbook: rolling out a new vLLM version or model on Kubernetes without dropping in-flight streamsAI & GenAI · 9 Nov 2026
- vLLM structured outputs: requests that are slow, rejected with 400, or still return broken JSONAI & GenAI · 9 Nov 2026
- vLLM tool calling not working: fix auto tool choice errors, parsers and tool calls returned as textAI & GenAI · 7 Nov 2026
- Monitor vLLM in production: Prometheus metrics, PromQL and alerts for queue, TTFT and KV cacheAI & GenAI · 6 Nov 2026
- Introduction to GPU scheduling on Kubernetes for LLM inference: requests, taints, node selectors and sharingAI & GenAI · 2 Nov 2026
- Load testing an LLM endpoint before go-live: TTFT, tokens per second, concurrency and where 429s startAI & GenAI · 31 Oct 2026
- Sev1: self-hosted LLM returns gibberish, never stops or leaks special tokens after a deploy — chat template and tokenizer fixesAI & GenAI · 31 Oct 2026
- Autoscaling self-hosted LLMs on Kubernetes: KEDA on queue depth and KV cache, plus GPU node autoscalingAI & GenAI · 29 Oct 2026
- Sev1: vLLM pods crash-loop with CUDA out of memory and the self-hosted LLM endpoint is downAI & GenAI · 29 Oct 2026
- vLLM memory tuning in depth: KV cache sizing, max-model-len, max-num-seqs and preemptionAI & GenAI · 29 Oct 2026
- LLM pods take 20 minutes to become ready: slow model weight downloads and loading on Kubernetes, and how to cut cold startsAI & GenAI · 28 Oct 2026
- Multi-GPU inference hangs or dies with NCCL timeouts: shared memory, P2P, topology and network interface problemsAI & GenAI · 28 Oct 2026
- Serving many LoRA adapters on one vLLM server: adapter load errors, rank limits, latency spikes and memory sizingAI & GenAI · 28 Oct 2026
- Advisory: choosing a self-hosted LLM serving stack - vLLM, SGLang, TensorRT-LLM/Triton, TGI, Ollama or llama.cppAI & GenAI · 25 Oct 2026
- Kill switches for self-hosted models on Kubernetes: ingress, network policy, scale to zeroAI & GenAI · 25 Oct 2026
- Optimization: speculative decoding for self-hosted LLMs - faster generation without changing the outputAI & GenAI · 25 Oct 2026
- Optimization: vLLM throughput tuning - batching limits, prefix caching, chunked prefill and parallelismAI & GenAI · 25 Oct 2026
- Sev1: time to first token jumps from 300 ms to 20 seconds on self-hosted LLM inferenceAI & GenAI · 25 Oct 2026
- Transformation: from long prompts to a LoRA fine-tuned small model for a narrow taskAI & GenAI · 25 Oct 2026
- Transformation: moving self-hosted LLM inference from FP16 to FP8 or INT4 quantized modelsAI & GenAI · 25 Oct 2026
- Transformation: when and how to move from a hosted LLM API to self-hosted open-weight modelsAI & GenAI · 25 Oct 2026
- Hugging Face model download fails: 401, 403 gated repos, timeouts and offline modeAI & GenAI · 24 Oct 2026
- Highly available self-hosted LLM inference on KubernetesAI & GenAI · 22 Oct 2026
- Running self-hosted LLMs with vLLM: GPU memory, KV cache and the metrics that matterAI & GenAI · 22 Oct 2026
- Standalone LLM server on one GPU with vLLM: install, serve, secure and measureAI & GenAI · 22 Oct 2026