Concept Beginner · 3 min read

What is function calling in LLMs

Quick answer

Function calling in LLMs is a feature that allows language models to invoke external functions or APIs during a chat or completion request. This enables the model to perform specific tasks, retrieve real-time data, or execute code by generating structured calls instead of plain text responses.

Function calling is a capability in large language models (LLMs) that enables them to invoke external functions or APIs to perform tasks beyond text generation.

How it works

Function calling lets an LLM generate a structured request to invoke a predefined external function during a conversation. Instead of only producing text, the model outputs a function call with a function name and JSON arguments. The application then executes this function and returns the result to the model or user. This is like having a smart assistant that can "call" tools or APIs to get precise data or perform actions on demand.

Think of it as the LLM writing a function call in code, which your app runs to extend the model's capabilities beyond language understanding.

Concrete example

Here is a Python example using the OpenAI SDK v1+ to demonstrate function calling. The model is instructed about a get_weather function, and when the user asks for weather, the model returns a function call with parameters. The app then parses and executes the function.

python

import os
import json
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

# Define the function metadata
functions = [{
    "name": "get_weather",
    "description": "Get weather for a location",
    "parameters": {
        "type": "object",
        "properties": {
            "location": {"type": "string"}
        },
        "required": ["location"]
    }
}]

# User message
messages = [{"role": "user", "content": "What's the weather in New York?"}]

# Request chat completion with function calling
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    functions=functions
)

choice = response.choices[0]

if choice.finish_reason == "function_call":
    func_call = choice.message.function_call
    func_name = func_call.name
    args = json.loads(func_call.arguments)
    print(f"Function to call: {func_name}")
    print(f"Arguments: {args}")
    # Here you would call your actual function, e.g., get_weather(args["location"])
else:
    print(choice.message.content)

output

Function to call: get_weather
Arguments: {'location': 'New York'}

When to use it

Use function calling when you want your LLM-powered app to perform precise, structured tasks like querying databases, calling APIs, or executing commands dynamically. It is ideal for applications requiring real-time data, tool integration, or safe execution of external logic.

Avoid function calling if your use case only needs free-form text generation or if you cannot securely handle external function execution.

Key terms

Term	Definition
Function calling	LLM feature to invoke external functions with structured arguments.
Tools	Predefined functions or APIs the model can call during generation.
Tool call	The model's generated request to invoke a specific function with parameters.
Finish reason 'function_call'	Indicates the model output is a function call, not plain text.

✅

Key Takeaways

Function calling lets LLMs extend beyond text by invoking external APIs or functions dynamically.
Use the OpenAI SDK's functions parameter to define callable functions and parse model function calls.
Function calling is essential for real-time data retrieval, automation, and safe external execution.

Verified 2026-04 · gpt-4o-mini

Verify ↗