Ollama cheat sheet: models, the REST API, Modelfiles and server settings

#Check the install

ollama --version
systemctl status ollama            # Linux install script sets up a systemd service
journalctl -u ollama -n 100 --no-pager
curl -s http://localhost:11434/api/version

The server listens on port 11434 by default. If curl fails, the server isn't running or is bound to another address (see OLLAMA_HOST below).

Docker with an NVIDIA GPU:

docker run -d --gpus=all --name ollama \
  -v ollama:/root/.ollama -p 11434:11434 ollama/ollama
docker exec -it ollama ollama ps

The ollama volume keeps downloaded models across container restarts. Without it, every new container downloads the models again.

#Models

Tags matter. llama3.2 without a tag means llama3.2:latest, which may not be the size or quantization you expect. Pin the full tag in scripts and configs.

Tip

ollama ps is the first check for any performance complaint. The PROCESSOR column shows 100% GPU, 100% CPU or a CPU/GPU split. A split means the model plus its context didn't fit in VRAM.

Inside ollama run, these slash commands help:

/set parameter num_ctx 8192
/set system "You are a terse Linux SRE. Answer with commands first."
/show info
/bye

#REST API

#Generate and chat

# Single prompt, no streaming
curl -s http://localhost:11434/api/generate -d '{
  "model": "llama3.2:3b",
  "prompt": "Explain the Linux OOM killer in two sentences.",
  "stream": false
}' | jq -r .response

# Chat with roles
curl -s http://localhost:11434/api/chat -d '{
  "model": "llama3.2:3b",
  "messages": [
    {"role": "system", "content": "Answer in one short paragraph."},
    {"role": "user", "content": "What does a 502 from a load balancer usually mean?"}
  ],
  "stream": false,
  "options": {"temperature": 0, "num_ctx": 8192}
}' | jq -r .message.content

Streaming is the default. Without "stream": false you get one JSON object per line.

#JSON output

curl -s http://localhost:11434/api/chat -d '{
  "model": "llama3.2:3b",
  "messages": [{"role": "user", "content": "Classify: \"Disk full on db01\". Return severity and team."}],
  "format": {"type": "object",
             "properties": {"severity": {"type": "string"}, "team": {"type": "string"}},
             "required": ["severity", "team"]},
  "stream": false
}' | jq -r .message.content

"format": "json" also works when you only need valid JSON and not a fixed schema. Schema support depends on your Ollama version.

#Embeddings

curl -s http://localhost:11434/api/embed -d '{
  "model": "nomic-embed-text",
  "input": ["disk full on /var", "nginx returns 502"]
}' | jq '.embeddings | length'

#Admin calls

Preload a model and keep it in memory, or unload it, by sending an empty request with keep_alive:

curl -s http://localhost:11434/api/generate -d '{"model": "llama3.2:3b", "keep_alive": -1}'   # load, keep forever
curl -s http://localhost:11434/api/generate -d '{"model": "llama3.2:3b", "keep_alive": 0}'    # unload now

#OpenAI-compatible endpoint

Existing OpenAI SDK code can point at Ollama:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")  # key is required by the SDK, ignored by Ollama
r = client.chat.completions.create(
    model="llama3.2:3b",
    messages=[{"role": "user", "content": "Give me a one-line kubectl command to list pods not Running."}],
)
print(r.choices[0].message.content)

Compatibility covers the common chat, completions and embeddings calls, not every OpenAI feature. Test the parameters you rely on.

#Modelfile: a custom model with fixed settings

Bake your system prompt and parameters into a named model so every caller gets the same behaviour:

# Modelfile
FROM llama3.2:3b
PARAMETER temperature 0.2
PARAMETER num_ctx 8192
PARAMETER stop "<|eot_id|>"
SYSTEM """
You are an on-call assistant. Answer with the command first, then one line of explanation.
If you are not sure, say so.
"""

ollama create ops-assistant -f Modelfile
ollama run ops-assistant "How do I find the biggest directories under /var?"
ollama show --modelfile ops-assistant

Keep Modelfiles in Git and rebuild with ollama create in your deployment script. That gives you a versioned, reviewable "prompt plus settings" artifact.

#Server settings (environment variables)

Defaults differ between releases; check journalctl -u ollama at startup, where the server prints its effective config.

On Linux with systemd, set them in an override, not in the unit file:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_HOST=127.0.0.1:11434"
Environment="OLLAMA_KEEP_ALIVE=30m"
Environment="OLLAMA_MODELS=/data/ollama/models"

sudo systemctl daemon-reload && sudo systemctl restart ollama

If you move OLLAMA_MODELS, the ollama service user must own the new directory (sudo chown -R ollama:ollama /data/ollama), or the server fails to start or "loses" its models.

Caution

Ollama's API has no authentication. Setting OLLAMA_HOST=0.0.0.0 exposes model pull, delete and generation to anyone who can reach the port. Keep it on localhost and put a reverse proxy with authentication and TLS in front, or restrict the port with a firewall or network security group.

#Quick troubleshooting