Home / AI & GenAI

AI & GenAITroubleshootingIntermediate

LLM tool calling fails: wrong tool, bad arguments or endless loops

Fix the common tool/function calling failures in agent apps, including vague tool descriptions, invalid JSON arguments, missing tool results and loops, with validation and limits in code.

Deepak Bhardwaj2 min read

Applies toOpenAI / Azure OpenAI, Amazon Bedrock Converse, Anthropic, Gemini, OCI Generative AI tool calling

Symptoms and causes

Symptom Usual cause
Model answers from memory instead of calling the tool Tool description doesn't say when to use it; tool_choice not set where a call is required
Calls the wrong tool Two tools with overlapping names and descriptions
Arguments invalid (missing field, wrong type, made-up IDs) Loose schema; no validation; the model never saw a valid ID
API error: tool result missing / unexpected message order Your code didn't send back a result for every tool call id
Agent loops calling the same tool Tool returns an error the model can't act on; no step limit

1. Write tool definitions for the model

{
  "name": "get_ticket_status",
  "description": "Get the current status, assignee and last update of ONE support ticket. Use when the user asks about a specific ticket number like INC0012345. Do not use for searching tickets.",
  "parameters": {
    "type": "object",
    "properties": {
      "ticket_id": {"type": "string", "pattern": "^INC[0-9]{7}$", "description": "Ticket number, e.g. INC0012345"}
    },
    "required": ["ticket_id"],
    "additionalProperties": false
  }
}
  • Say when to use it and when not to.
  • Use enums, patterns and required fields. Enable strict schema mode where the provider supports it.
  • Fewer tools work better. With more than ~15–20, group them or route to sub-agents.

2. Validate every call in code

import json, jsonschema
args = json.loads(call.arguments)
jsonschema.validate(args, TOOLS[call.name]["parameters"])

On failure, don't crash. Return a clear error as the tool result so the model can correct itself:

{"error": "ticket_id must look like INC0012345; got '12345'"}

3. Return results correctly

  • Every tool call id gets exactly one result message, in the same turn, including when parallel calls happen.
  • Keep results small and structured. Don't return a 2 MB API response.
  • Include enough for the next step (IDs, status) and nothing secret.

4. Stop loops

MAX_STEPS = 8
for step in range(MAX_STEPS):
    resp = llm.chat(messages, tools=tools)
    if not resp.tool_calls:
        break
    ...
else:
    return "I couldn't complete this automatically; a human will follow up."

Also detect repeats: the same tool with the same arguments twice in a row → stop and escalate.

5. Test

Create a set of user requests with the expected tool and arguments, and measure tool selection accuracy and argument validity on each model or prompt change.

Warning

Never execute a tool call that changes something (refund, restart, delete) just because the model asked. Validate permissions for the end user, and require confirmation for impactful actions.

Related fixes