LLM Tool Calling (Function Calling) Explained: The Loop, the Schema and Why You Must Validate Everything

A language model is a text predictor. On its own it cannot check today's weather, look up a customer in your database, run a calculation reliably or send an email. It can only produce words. Tool calling (also called function calling) is the trick that changes that, and it is the base of every AI agent you will meet in Part VI.

The important thing to understand is what the model does not do: it never runs anything itself. It writes a structured request ("please call get_weather with city = Lahore"), and then your code decides whether and how to run it. This session shows the loop with a real model running on this laptop, then the part most tutorials skip: how it fails, and how to defend against it. It builds on the API ideas of Session 21 (messages, stop reasons) and the local model of Session 23.

How tool calling works

1. You describe each tool to the model: a name, a plain-language description and a JSON schema of its parameters.
2. The user asks something. If the model decides a tool would help, it replies not with an answer but with a structured tool call (tool name plus JSON arguments) and stops with the stop reason tool_use (or tool_calls).
3. Your code checks the call, runs the function, and sends the result back as a new message.
4. The model reads the result and writes the final answer, or asks for another tool.

That cycle, request, call, result, answer, is the whole idea. Here are the two tools we use, as ordinary Python. The code in the rest of the session imports them as REGISTRY and TOOLS:

import ast, operator

WEATHER = {"lahore": {"temp_c": 31, "sky": "clear"}, "karachi": {"temp_c": 34, "sky": "humid"},
           "peshawar": {"temp_c": 27, "sky": "cloudy"}}

def get_weather(city: str) -> dict:
    """Get the current weather for a city."""
    return WEATHER.get(city.strip().lower(), {"error": f"no data for {city!r}"})

def calculator(expression: str) -> float:
    """Evaluate a plain arithmetic expression such as '23 * 47'."""
    ops = {ast.Add: operator.add, ast.Sub: operator.sub, ast.Mult: operator.mul, ast.Div: operator.truediv}
    def ev(n):
        if isinstance(n, ast.Constant) and isinstance(n.value, (int, float)): return n.value
        if isinstance(n, ast.BinOp) and type(n.op) in ops: return ops[type(n.op)](ev(n.left), ev(n.right))
        raise ValueError("only + - * / on numbers is allowed")
    return ev(ast.parse(expression, mode="eval").body)

REGISTRY = {"get_weather": get_weather, "calculator": calculator}      # name -> function

Notice the calculator does not use eval. It walks the parsed expression and allows only numbers and four operators. Handing the model's text to eval would let it run any code. The tool descriptions the model sees, TOOLS, are JSON:

TOOLS = [
  {"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.",
     "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}},
  {"type": "function", "function": {"name": "calculator", "description": "Evaluate an arithmetic expression.",
     "parameters": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]}}},
]

One full round trip

Now the loop, with the local 1.5-billion-parameter model from the last session. Read the four numbered steps in the code against the four steps above:

import json
from llama_cpp import Llama

llm = Llama(model_path="data/qwen2.5-1.5b-instruct-q4_k_m.gguf", n_ctx=2048, n_threads=8, verbose=False,
            chat_format="chatml-function-calling")           # a local model, but the loop is identical for any provider

messages = [{"role": "user", "content": "What's the weather in Lahore right now?"}]

# 1) the model sees the tool descriptions and decides to call one. It RUNS NOTHING itself.
first = llm.create_chat_completion(messages=messages, tools=TOOLS, tool_choice="auto", temperature=0, max_tokens=100)
choice = first["choices"][0]
print("stop reason :", choice["finish_reason"])
call = choice["message"]["tool_calls"][0]
print("model asked :", call["function"]["name"], call["function"]["arguments"])

# 2) OUR code runs the function and sends the result back as a new message
args = json.loads(call["function"]["arguments"])
result = REGISTRY[call["function"]["name"]](**args)
print("our code ran:", result)

# 3) the model reads the result and writes the final answer.
#    Real provider APIs have a proper "tool result" message type. This small local runtime handles it poorly (it asks for
#    the tool again), so here the result is passed back as plain text. The idea is identical.
followup = {"role": "user", "content": f"Tool result from {call['function']['name']}: {json.dumps(result)}\nUsing only this result, answer the first question in one sentence."}
final = llm.create_chat_completion(messages=[messages[0], followup], temperature=0, max_tokens=80)
print("final answer:", final["choices"][0]["message"]["content"])

stop reason : tool_calls
model asked : get_weather {"city":"Lahore"}
our code ran: {'temp_c': 31, 'sky': 'clear'}
final answer: The current temperature in Lahore is 31 degrees Celsius, with clear skies.

The model was asked "What's the weather in Lahore right now?". It did not answer. It stopped with the reason tool_calls and asked for get_weather with the arguments {"city":"Lahore"}. Our code ran the function and got {'temp_c': 31, 'sky': 'clear'}, and then the model wrote the final answer from that result: "The current temperature in Lahore is 31 degrees Celsius, with clear skies." The model never touched the weather data. It only chose the tool and phrased the answer.

A practical note from writing this: with the local runtime used here, sending the result back as a proper "tool" message made the small model ask for the tool again, so the code passes the result back as plain text. Hosted APIs have a well-behaved "tool result" message type, and the idea is the same either way.

Tools from plain Python functions

You do not have to write JSON schemas by hand. A tool description is just the function's name, docstring and type hints, and libraries (and, as you will see next session, the MCP SDK) build it automatically:

import inspect, json

def make_tool(fn):
    """Build a tool description from a plain Python function: the name, the docstring and the type hints."""
    kinds = {str: "string", int: "integer", float: "number", bool: "boolean"}
    props, required = {}, []
    for name, p in inspect.signature(fn).parameters.items():
        props[name] = {"type": kinds[p.annotation]}
        if p.default is inspect.Parameter.empty: required.append(name)
    return {"name": fn.__name__, "description": inspect.getdoc(fn),
            "input_schema": {"type": "object", "properties": props, "required": required}}

def search_students(city: str, min_marks: int, active_only: bool = True) -> list:
    """Find students by city and minimum marks."""

print(json.dumps(make_tool(search_students), indent=2))

{
  "name": "search_students",
  "description": "Find students by city and minimum marks.",
  "input_schema": {
    "type": "object",
    "properties": {
      "city": {
        "type": "string"
      },
      "min_marks": {
        "type": "integer"
      },
      "active_only": {
        "type": "boolean"
      }
    },
    "required": [
      "city",
      "min_marks"
    ]
  }
}

The function name became the tool name, the docstring became the description the model reads, and the type hints became the JSON schema (str to string, int to integer, bool to boolean). Parameters without defaults, city and min_marks, are marked required. This is why clear docstrings and precise type hints matter: they are literally the instructions the model uses to decide when and how to call your function.

It does not always go well

Here is the part that matters most in real projects. A small model is not a reliable decision-maker. Let's offer the same two tools and ask three things, two of which need a tool and one of which does not:

import json
from llama_cpp import Llama

llm = Llama(model_path="data/qwen2.5-1.5b-instruct-q4_k_m.gguf", n_ctx=2048, n_threads=8, verbose=False,
            chat_format="chatml-function-calling")

print("question                           what the small local model did")
for q in ["What is 23 * 47?", "Say hello.", "Tell me a joke."]:
    out = llm.create_chat_completion(messages=[{"role": "user", "content": q}], tools=TOOLS, tool_choice="auto", temperature=0, max_tokens=100)
    m = out["choices"][0]["message"]
    action = f"called {m['tool_calls'][0]['function']['name']}({m['tool_calls'][0]['function']['arguments'].strip()})" if m.get("tool_calls") else "answered in text: " + m["content"][:40]
    print(f"{q:34s} {action}")

question                           what the small local model did
What is 23 * 47?                   called calculator({ "expression": "23 * 47" })
Say hello.                         called get_weather({ "city": "New York, NY, USA" })
Tell me a joke.                    answered in text: functions.calculator:

"What is 23 * 47?" gets a correct call to the calculator. But "Say hello." (which needs no tool at all) made the model call get_weather for "New York, NY, USA", a city nobody mentioned. And "Tell me a joke." produced a broken half-formed functions.calculator: text. Larger and better models are far more reliable, but the lesson holds for all of them: the model's output is untrusted input. It can pick the wrong tool, invent arguments, or call a tool when it should simply talk.

Treat arguments as untrusted input

Everything that comes back from the model must be validated before anything runs. At minimum:

• An allow-list of tools. Only names you registered can run.
• Schema validation. The arguments must match the JSON schema exactly (right fields, right types).
• Safe execution. Use parameterised queries, never string-built SQL. Run code only in a sandbox. Give database tools read-only access wherever possible.
• Human confirmation before anything destructive or expensive (delete, pay, send an email).
• Timeouts and rate limits.
• Treat tool results as data, not instructions. A web page or document can contain hidden text telling the model to do something else, which is prompt injection (Session 21).

import json
from jsonschema import ValidationError, validate

SCHEMAS = {t["function"]["name"]: t["function"]["parameters"] for t in TOOLS}
ALLOWED = {"get_weather", "calculator"}                           # an allow-list: nothing else can ever run

def safe_run(name, raw_arguments, user_id):
    """Treat everything the model sends as untrusted input."""
    if name not in ALLOWED:
        return {"error": f"unknown tool {name!r}"}
    try:
        args = json.loads(raw_arguments)
        validate(args, SCHEMAS[name])                              # must match the JSON schema exactly
        return REGISTRY[name](**args)
    except (json.JSONDecodeError, ValidationError) as e:
        return {"error": "bad arguments: " + str(e).splitlines()[0]}
    except Exception as e:                                         # the tool itself refused
        return {"error": str(e)}

tests = [("get_weather", '{"city": "Karachi"}'),
         ("get_weather", '{"town": "Karachi"}'),                   # wrong field name
         ("get_weather", '{"city": 123}'),                         # wrong type
         ("calculator", '{"expression": "__import__(\'os\').system(\'ls\')"}'),   # an attack
         ("delete_all_users", "{}"),                               # a tool that does not exist
         ("get_weather", "not json at all")]
for name, raw in tests:
    print(f"{name}({raw[:44]})".ljust(66), "->", safe_run(name, raw, user_id=42))

get_weather({"city": "Karachi"})                                   -> {'temp_c': 34, 'sky': 'humid'}
get_weather({"town": "Karachi"})                                   -> {'error': "bad arguments: 'city' is a required property"}
get_weather({"city": 123})                                         -> {'error': "bad arguments: 123 is not of type 'string'"}
calculator({"expression": "__import__('os').system('ls')           -> {'error': 'only + - * / on numbers is allowed'}
delete_all_users({})                                               -> {'error': "unknown tool 'delete_all_users'"}
get_weather(not json at all)                                       -> {'error': 'bad arguments: Expecting value: line 1 column 1 (char 0)'}

Six calls, as if produced by a model. The good call runs. A call with the wrong field name (town) is rejected with "'city' is a required property". A wrong type (123 where text is expected) is rejected. An attack, __import__('os').system('ls') sent to the calculator, is stopped by the tool itself, which allows only numbers and four operators. A made-up tool, delete_all_users, is refused because it is not on the allow-list. And invalid JSON is rejected before anything runs. Notice that the error messages are returned as results: the model can read them and try again, a pattern we use heavily in agents.

Scope and permissions

Tools run with someone's authority, so decide whose:

• User-level tools may only touch that user's own data. The user ID must come from your authenticated session, never from the model (look at user_id=42 in the code above, supplied by us, not by the model).
• Admin-level tools (bulk updates, settings) are exposed only to verified admin roles.
• Filter the tool list per role before the request reaches the model, so a normal user's model never even sees the admin tools. Log every tool call for auditing.

Common kinds of tools

• Web search (DuckDuckGo for learning, or Tavily, Brave or Serper for fresher results).
• Database tools, either fixed functions such as get_user_marks(user_id) (safest) or text-to-SQL (flexible, riskier).
• Calculators, code execution and file readers.
• Internal APIs.

Structured output: the sibling idea

Tool calling is closely related to structured output, where you force the model to reply in a strict JSON shape (using JSON mode or a schema) so your program can parse it instead of scraping free text. Both rely on the same fact: a model can be steered to emit a precise structure, and your code must then check it.

Here is the same loop against a hosted API (the shape, not code run here). The message types differ slightly between providers:

tools = [{
    "name": "get_weather",
    "description": "Get the current weather for a city.",
    "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
}]
messages = [{"role": "user", "content": "What's the weather in Lahore?"}]

reply = client.messages.create(model="claude-sonnet-5-5", max_tokens=300, tools=tools, messages=messages)
if reply.stop_reason == "tool_use":                              # the model asked for a tool
    call = next(b for b in reply.content if b.type == "tool_use")
    result = get_weather(**call.input)                           # OUR code runs it (after validation!)
    messages += [{"role": "assistant", "content": reply.content},
                 {"role": "user", "content": [{"type": "tool_result", "tool_use_id": call.id, "content": str(result)}]}]
    final = client.messages.create(model="claude-sonnet-5-5", max_tokens=300, tools=tools, messages=messages)
    print(final.content[0].text)

Try it yourself

1. Add a third tool, get_time(city), and re-run misfire.py. Does the small model start calling the wrong tool more often?
2. In validate_calls.py, add a test where the calculator receives "10 / 0". Where does it fail, and does your guard catch it?
3. Make average_marks(student) a tool (use the marks data from the next session's server) and chain it with the calculator.
4. Write a guard that lets only users with the role "admin" call a reset_password tool.

What comes next

Every app wires its tools in its own way, which means rewriting the same integration again and again. Session 25 introduces the Model Context Protocol (MCP), one open standard for exposing tools and data to any AI app, and we will build a real MCP server and talk to it.

Frequently asked questions