AI APIs & Tool Use: Messages, Tools, Streaming & MCP

Q: Should I let the SDK run the tool loop or do it manually?

Use the SDK's tool runner for the common case — it calls the API, executes your tool functions, feeds results back, and loops until the model is done, with far less code. Run the loop manually when you need fine-grained control: human-in-the-loop approval before executing a tool, custom logging/auditing, conditional execution, or special error handling. Both hit the same endpoint; the choice is about how much control you need over each step.

AI APIs & Tool Use

An AI API lets your code send a prompt to a hosted LLM and get a response — and, increasingly, lets the model call tools to fetch data and take actions. Modern APIs like Anthropic's Messages API (and similar offerings) converge on the same shape: a list of messages in, a structured response out, with optional system prompts, streaming, tool definitions, and output constraints layered on the same endpoint.

The mental leap beyond "text in, text out" is tool use (a.k.a. function calling): you describe tools the model may use, and when it decides to, the API returns a tool-call request; your code runs it and feeds the result back. That loop is what turns a chatbot into something that can search, query databases, and act.

TL;DR

Everything goes through a messages endpoint — messages in, structured response out.
Use system prompts for role/rules; stream long responses.
Tool use / function calling lets the model call your code to fetch data and act.
Structured outputs guarantee JSON shape; MCP standardizes tool/data connections.

Quick Example

A single call to Claude via the Anthropic SDK:

Core Concepts

The messages model

You send a messages array (alternating user/assistant turns) plus optional system instructions, and get back content blocks. The API is stateless — you resend the conversation history each call. max_tokens caps the output; set it generously so responses aren't truncated.

Streaming

For long outputs, stream the response so tokens arrive incrementally — this shows progress and avoids request timeouts. SDKs expose a streaming helper and a way to get the final assembled message.

Tool use (function calling)

Describe tools with a name, description, and JSON-schema inputs. The model decides when to call one; the API returns a tool-call request; your code executes it and returns the result, and the model continues. SDKs provide a tool runner that automates this loop, or you can run it manually for fine-grained control (approval gates, logging):

Structured outputs

When you need machine-readable JSON, use structured outputs to constrain the response to a schema, instead of hoping the model formats it correctly and parsing free text.

Model Context Protocol (MCP)

MCP is an open standard for connecting models to external tools and data sources through a uniform interface — so a tool/integration written once works across MCP-compatible clients, rather than re-implementing connectors per app.

Best Practices

Keep secrets server-side — never ship API keys to the browser; call the AI API from your backend.
Set a sensible max_tokens and stream long responses.
Use the SDK's tool runner for the standard loop; the manual loop when you need approval/audit hooks.
Gate destructive tools (send email, write data) behind validation or human confirmation.
Use prompt caching for large, stable prefixes (system prompt, tool list) to cut cost and latency.

Common Mistakes

Exposing your API key in client code

Auto-executing every tool call without guards

FAQ

What's the difference between a chat completion and tool use?

A plain completion is text in, text out — the model responds with prose. Tool use lets the model request an action: you declare tools, and when the model decides one is needed, the API returns a structured tool-call (name + arguments) instead of (or alongside) text. Your code runs it and returns the result, and the model continues with that information. Tool use is what lets an LLM fetch live data, query systems, and take actions rather than only generate text.

Should I let the SDK run the tool loop or do it manually?

Use the SDK's tool runner for the common case — it calls the API, executes your tool functions, feeds results back, and loops until the model is done, with far less code. Run the loop manually when you need fine-grained control: human-in-the-loop approval before executing a tool, custom logging/auditing, conditional execution, or special error handling. Both hit the same endpoint; the choice is about how much control you need over each step.

What is MCP (Model Context Protocol)?

MCP is an open standard for connecting AI models to tools and data sources through a consistent interface. Instead of writing bespoke integration code for each app, you expose a capability once as an MCP server (e.g. GitHub, a database, a file system) and any MCP-compatible client can use it. It standardizes the "how do I give the model access to X" problem, making tools portable across applications and reducing per-integration glue code.

How do I keep AI API usage secure?

Keep API keys on the server — never in client-side code or public repos (rotate immediately if leaked). Call the model from your backend and expose only your own endpoints to the frontend. Treat tool use as a security boundary: validate tool inputs, gate destructive actions behind confirmation, and assume prompt injection is possible (untrusted text trying to hijack the model). Add rate limiting and monitoring on your AI endpoints. See API Security and Secrets Management.