LLM Function Calling & Tool Use: Comprehensive Reference (2024-2026)

A production-grade reference for implementing function calling and tool use with Large Language Models. This document synthesizes best practices from Anthropic, OpenAI, MCP specification, and industry research.

---

Table of Contents

1. Tool Schema Design Checklist
2. Description Writing Guidelines
3. Tool Selection Optimization
4. Parallel Tool Calls
5. Error Handling Patterns
6. Tool Composition & Chaining
7. Security Considerations
8. Sandboxing & Isolation
9. MCP (Model Context Protocol)
10. Structured Outputs & Constrained Decoding
11. Performance Optimization
12. Common Pitfalls
13. Real-World Examples
14. Sources

---

1. Tool Schema Design Checklist

Required Elements

{
  "name": "get_weather",
  "description": "Get the current weather in a given location",
  "input_schema": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "The city and state, e.g. San Francisco, CA"
      },
      "unit": {
        "type": "string",
        "enum": ["celsius", "fahrenheit"],
        "description": "Temperature unit"
      }
    },
    "required": ["location"]
  }
}

Schema Design Best Practices

Automatic Schema Generation

Recommended: Generate schemas automatically from code to avoid drift.

# OpenAI SDK with Pydantic
from openai import OpenAI
from pydantic import BaseModel

class WeatherParams(BaseModel):
    location: str
    unit: str = "celsius"

# Auto-generate schema
client.chat.completions.create(
    tools=[{"type": "function", "function": pydantic_function_tool(WeatherParams)}]
)

Tools for schema generation:

• Python: pydantic, inspect module
• OpenAI SDK: pydantic_function_tool() utility
• Anthropic: JSON Schema with input_schema

---

2. Description Writing Guidelines

The "Intern Test"

"Can an intern/human correctly use the function given nothing but what you gave the model?"

If not, add more context.

Writing Effective Descriptions

Function Description:

• Explain what the function does and when to use it
• Include context about expected inputs and outputs
• Mention limitations or edge cases

Parameter Descriptions:

• Describe format expectations (e.g., "IANA timezone like America/Los_Angeles")
• Explain relationships between parameters
• Include examples for complex inputs

Good vs. Bad Examples

// BAD
{
  "name": "search",
  "description": "Search for things",
  "input_schema": {
    "properties": {
      "q": { "type": "string" },
      "n": { "type": "integer" }
    }
  }
}

// GOOD
{
  "name": "search_products",
  "description": "Search the product catalog by keyword. Returns matching products with prices. Use for inventory queries or price checks.",
  "input_schema": {
    "properties": {
      "query": {
        "type": "string",
        "description": "Search keywords. Supports AND/OR operators. Example: 'laptop AND gaming'"
      },
      "max_results": {
        "type": "integer",
        "description": "Maximum results to return (1-100, default 10)",
        "minimum": 1,
        "maximum": 100
      }
    },
    "required": ["query"]
  }
}

System Prompt Integration

Use system prompts to guide tool selection:

You are a customer service agent. You have access to:
- order_lookup: Use when customer asks about order status
- product_search: Use when customer asks about products
- escalate_to_human: Use when issue cannot be resolved

Always verify order numbers before looking them up.
Never use order_lookup for general questions.

---

3. Tool Selection Optimization

Why Tool Selection Fails

The "Less-is-More" Approach

Research shows that dynamically reducing available tools improves accuracy by up to 89% and reduces execution time by 80%.

Implementation:

1. Pre-filter tools based on user intent
2. Present only 3-5 relevant tools per turn
3. Use a routing layer to select tool subsets

Improving Selection Accuracy

1. Fine-tuning: Train on task-specific tool usage examples
2. In-context learning: Provide examples of correct tool selection
3. Chain-of-thought prompting: Force reasoning before tool calls

Claude-specific prompt for better selection:

Before calling a tool, analyze:
1. Which tool is relevant to the user's request?
2. Are all required parameters available?
3. Can missing parameters be reasonably inferred?

If parameters are missing, ask for them instead of guessing.

---

4. Parallel Tool Calls

When to Use Parallel Calls

Use parallel tool calls when:

• Operations are independent (no data dependencies)
• You need results from multiple sources
• Latency optimization is important

Implementation

OpenAI: Control with parallel_tool_calls parameter

response = client.chat.completions.create(
    model="gpt-4",
    messages=[...],
    tools=[...],
    parallel_tool_calls=True  # Default: True
)

Anthropic: Claude automatically decides when to parallelize

# Claude returns multiple tool_use blocks in one response
response.content = [
    {"type": "tool_use", "id": "id1", "name": "get_weather", "input": {...}},
    {"type": "tool_use", "id": "id2", "name": "get_time", "input": {...}}
]

# Return all results in one user message
{
    "role": "user",
    "content": [
        {"type": "tool_result", "tool_use_id": "id1", "content": "72F"},
        {"type": "tool_result", "tool_use_id": "id2", "content": "3:00 PM"}
    ]
}

LLMCompiler Framework

Academic research on parallel function calling:

• 3.7x latency speedup
• 6.7x cost savings
• ~9% accuracy improvement

Components:

1. Function Calling Planner
2. Dependency Graph Builder
3. Parallel Executor

Available in LangGraph/LangChain.

---

5. Error Handling Patterns

Error Response Format

Anthropic:

{
  "type": "tool_result",
  "tool_use_id": "toolu_abc123",
  "content": "Error: Location 'xyz' not found. Please provide a valid city name.",
  "is_error": true
}

OpenAI:

{
  "role": "tool",
  "tool_call_id": "call_abc123",
  "content": "Error: Invalid API key. Please check credentials."
}

Error Handling Best Practices

1. Validate before execution: Check parameters match schema
2. Return structured errors: Include error type, message, suggestions
3. Let model retry: Provide error context, model may self-correct
4. Log all failures: Enable debugging and pattern detection
5. Set timeouts: Prevent hanging on slow tools

---

6. Tool Composition & Chaining

Sequential Chaining

One tool's output feeds the next:

get_location() -> "San Francisco, CA"
                        ↓
get_weather("San Francisco, CA") -> "72F, Sunny"

Key considerations:

• Output format must match next tool's input
• Handle intermediate failures gracefully
• Track state across the chain

Workflow Patterns

Anthropic's Recommendation

"Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents are systems where LLMs dynamically direct their own processes."

Start with workflows for predictable tasks; use agents for open-ended problems.

Implementation Tips

1. Test each step individually before chaining
2. Define clear interfaces between tools
3. Handle partial failures (some tools succeed, some fail)
4. Add checkpoints for long-running workflows
5. Consider state persistence for resumability

---

7. Security Considerations

OWASP Top 1: Prompt Injection

Prompt injection is the #1 vulnerability in LLM applications (73% of deployments affected).

Types:

• Direct injection: User crafts malicious prompts
• Indirect injection: Malicious content in external data (websites, documents)

Defense Strategies

Input Validation Example

def validate_tool_input(tool_name: str, params: dict) -> dict:
    schema = get_tool_schema(tool_name)

    # Type validation
    for param, value in params.items():
        expected_type = schema["properties"][param]["type"]
        if not isinstance(value, TYPE_MAP[expected_type]):
            raise ValidationError(f"Invalid type for {param}")

    # Range validation
    if "maximum" in schema["properties"].get(param, {}):
        if value > schema["properties"][param]["maximum"]:
            raise ValidationError(f"{param} exceeds maximum")

    # Enum validation
    if "enum" in schema["properties"].get(param, {}):
        if value not in schema["properties"][param]["enum"]:
            raise ValidationError(f"Invalid value for {param}")

    return params  # Validated

Tool Permission Model

TOOL_PERMISSIONS = {
    "read_file": {"requires_approval": False, "scope": "read"},
    "write_file": {"requires_approval": True, "scope": "write"},
    "execute_command": {"requires_approval": True, "scope": "execute"},
    "send_email": {"requires_approval": True, "scope": "external"},
}

def execute_tool(tool_name: str, params: dict, user_approved: bool = False):
    perms = TOOL_PERMISSIONS[tool_name]

    if perms["requires_approval"] and not user_approved:
        raise PermissionError(f"{tool_name} requires explicit user approval")

    return tools[tool_name](**params)

---

8. Sandboxing & Isolation

Why Sandboxing Matters

AI-generated code can:

• Access filesystem (SSH keys, credentials)
• Make network requests (data exfiltration)
• Execute arbitrary commands
• Consume unlimited resources

Isolation Technologies

Recommended Architecture

┌─────────────────────────────────────────┐
│              Host System                │
│  ┌───────────────────────────────────┐  │
│  │       Sandbox Controller          │  │
│  │  (orchestrates, validates)        │  │
│  └───────────────────────────────────┘  │
│                    │                    │
│  ┌─────────────────┴─────────────────┐  │
│  │      Isolated Sandbox             │  │
│  │  ┌─────────────────────────────┐  │  │
│  │  │  Code Execution Environment │  │  │
│  │  │  - No network by default    │  │  │
│  │  │  - Read-only filesystem     │  │  │
│  │  │  - Resource limits          │  │  │
│  │  └─────────────────────────────┘  │  │
│  └───────────────────────────────────┘  │
└─────────────────────────────────────────┘

Layered Defense

1. Application layer: Input validation, output filtering
2. Container layer: Namespace isolation, resource limits
3. Network layer: Firewall rules, VPC isolation
4. Kernel layer: gVisor or VM isolation

Claude Code Sandboxing

Claude Code supports multiple sandbox modes:

• Docker-based: Full container isolation
• macOS Sandbox: Apple's seatbelt framework
• Permission-based: Tool-level access control

---

9. MCP (Model Context Protocol)

Overview

MCP is an open standard (November 2024) for connecting LLMs to external tools and data sources. Adopted by Anthropic, OpenAI, Google, and Microsoft.

Architecture

┌────────────────────┐
│     MCP Host       │  (Claude Desktop, IDEs)
│  (LLM Application) │
└─────────┬──────────┘
          │
┌─────────┴──────────┐
│    MCP Client      │  (Connection manager)
└─────────┬──────────┘
          │ JSON-RPC 2.0
┌─────────┴──────────┐
│    MCP Server      │  (Tool provider)
│  - Resources       │
│  - Prompts         │
│  - Tools           │
└────────────────────┘

Server Primitives

Security Principles

1. User Consent: Explicit approval for all data access
2. Data Privacy: No transmission without consent
3. Tool Safety: Untrusted descriptions require verification
4. LLM Sampling Controls: User approves all sampling requests

Building an MCP Serve