Calling LLM APIs

When you type into a chat assistant, a program somewhere packages your words into a small parcel of data, sends it across the internet to a model, waits, and unpacks the reply. That parcel is an API call, and by the end of this page you will be writing them yourself. Once you can, the chat window stops being the only way to use a model: you can run it over a thousand documents overnight, wire it into a spreadsheet, or build an assistant that knows your company’s rules. The code is short. The craft is in knowing what you are sending, what you are paying for, and what can go wrong.

What an API call is

An API (application programming interface) is a documented way for one program to ask another for something. A language-model API is a web address that accepts a request — which model, your instructions, the conversation so far — and returns the model’s reply as structured data. Running the model to produce that reply is called inference, and you pay per token processed. Tap through the journey of one request:

An SDK (software development kit) is a library that wraps these web requests in friendly Python. This page teaches one SDK in depth — Anthropic’s, for its Claude models — and shows the OpenAI equivalent where you would want it. Other providers, including Google Gemini, Mistral and open-weight models served locally through Ollama or Hugging Face, follow the same request → response shape, so the ideas carry over almost unchanged.

Your first call

If you have not yet made a project folder, a virtual environment and a safe home for your key, do that first: Setting up your workshop covers it, including why keys belong in environment variables or an ignored .env file and never in code. Then install the SDK and set your key for the session:

pip install anthropic
# macOS / Linux
export ANTHROPIC_API_KEY="paste-your-key-here"
# Windows PowerShell
$env:ANTHROPIC_API_KEY = "paste-your-key-here"

Now the smallest useful program:

import anthropic

MODEL = "claude-opus-5-5"   # check the provider's docs for current model names

client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY from the environment

response = client.messages.create(
    model=MODEL,
    max_tokens=16000,
    system="You are a friendly tutor. Answer in at most three sentences.",
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)

text = "".join(block.text for block in response.content if block.type == "text")
print(text)
print("stop_reason:", response.stop_reason)
print("tokens in/out:", response.usage.input_tokens, response.usage.output_tokens)

Example output — your text will differ
Sunlight contains all colours, but air molecules scatter short blue wavelengths
far more strongly than long red ones, so blue light reaches your eyes from every
direction in the sky. At sunrise and sunset the light travels through much more
air, most of the blue is scattered away, and the reds and oranges remain.
stop_reason: end_turn
tokens in/out: 31 412

The request has four parts:

• model — which model to run. Model names change often; claude-opus-5-5 (Claude Opus 5.5) is Anthropic’s current default As of October 2026. Keeping the name in one variable at the top means one edit when it changes.
• max_tokens — the most the reply may contain. It is a safety ceiling, not a target. Set it too small and replies are cut off mid-sentence; 16,000 is a sensible default for ordinary, non-streamed calls.
• system — optional standing instructions that frame the whole conversation (more on this below).
• messages — the conversation as a list of turns, each with a role ("user" or "assistant") and content.

The response is richer than a string. response.content is a list of blocks. Current Claude models can return thinking blocks (the model’s working, usually hidden) as well as text blocks, so the code joins only the blocks whose type is "text". response.stop_reason says why the model stopped: end_turn (it finished), max_tokens (it hit your ceiling — the reply is incomplete), tool_use (it wants to call one of your functions, covered in Tool Use and Agents) or refusal (it declined). Check it before trusting the text. response.usage records the tokens you are billed for.

Tokens and cost

Providers charge per token, separately for input (everything you send, including the system prompt and the whole conversation history) and output (everything the model writes, including any thinking it does). Prices are quoted per million tokens. You can ask how many input tokens a request will use before sending it:

import anthropic

MODEL = "claude-opus-5-5"   # check the provider's docs for current model names
client = anthropic.Anthropic()

messages = [{"role": "user", "content": "Summarise the history of Melaka in 200 words."}]
count = client.messages.count_tokens(model=MODEL, messages=messages)
print("This request would send", count.input_tokens, "input tokens")

count_tokens takes the same model, messages (and system, if you use one) as the real call, and is free to use. Output tokens cannot be known in advance, so estimate them from typical replies. The arithmetic is then simple enough to run without any key at all:

# Example prices in US dollars per million tokens (Anthropic list prices, October 2026).
# Prices change: check the provider's pricing page before you rely on them.
PRICES = {
    "claude-opus-5-5":  {"input": 4.00, "output": 20.00},
    "claude-haiku-5-5": {"input": 0.10, "output": 0.50},
}

def cost_usd(model, input_tokens, output_tokens):
    p = PRICES[model]
    return (input_tokens * p["input"] + output_tokens * p["output"]) / 1_000_000

# A support bot: 2,000 questions a day, ~1,500 tokens in and ~300 out each
for model in PRICES:
    one = cost_usd(model, 1_500, 300)
    print(f"{model:<17} per call ${one:.5f}   per 30 days ${one * 2_000 * 30:,.2f}")

claude-opus-5-5   per call $0.01200   per 30 days $720.00
claude-haiku-5-5  per call $0.00030   per 30 days $18.00

The example prices are Anthropic’s list prices for Claude Opus 5.5 ($4 input, $20 output per million tokens) and Claude Haiku 5.5 ($0.10 and $0.50 for prompts up to 100,000 tokens) As of October 2026. Three lessons fall out. Output usually costs several times more than input. The same workload can differ in cost by a factor of forty depending on the model, so try the smallest model that does the job well — and measure “well” with a test set, as on Measuring Models. And because the whole history is input on every turn, long conversations get steadily more expensive. Every model also has a context window, a maximum number of tokens it can read at once; Working with Context explains what that means in practice.

Conversations: the API is stateless

The single most surprising fact for newcomers: the model remembers nothing between calls. Each request is answered from scratch. The “memory” of a chat app is the app resending the whole conversation every turn. So your code keeps a list, appends each user message and each reply, and sends the whole list each time:

import anthropic

MODEL = "claude-opus-5-5"   # check the provider's docs for current model names
client = anthropic.Anthropic()
messages = []                      # the whole conversation lives here, on YOUR side

print("Chat with Claude. Type 'quit' to stop.")
while True:
    user_text = input("You: ").strip()
    if user_text.lower() in {"quit", "exit"}:
        break
    messages.append({"role": "user", "content": user_text})

    response = client.messages.create(
        model=MODEL,
        max_tokens=16000,
        system="You are a patient Python tutor. Keep answers short.",
        messages=messages,          # resend the full history every turn
    )
    reply = "".join(b.text for b in response.content if b.type == "text")
    print("Claude:", reply)
    messages.append({"role": "assistant", "content": reply})

Example output — your text will differ
Chat with Claude. Type 'quit' to stop.
You: What is a list in Python?
Claude: An ordered, changeable collection, written with square brackets: [3, 1, 4].
You: How do I add to one?
Claude: Use .append(): numbers.append(5) adds 5 to the end of the list.
You: quit

The second question, “How do I add to one?”, only makes sense because the first exchange was sent again with it. Delete the line that appends the assistant’s reply and the model will lose the thread. Two practical consequences: the history must alternate user and assistant turns, and a long chat eventually needs trimming or summarising, both to stay inside the context window and to control cost.

Streaming, errors and retries

A long reply can take many seconds. Streaming sends the text in small pieces as it is generated, so your user sees words appear immediately — the familiar typing effect of chat apps.

import anthropic

MODEL = "claude-opus-5-5"   # check the provider's docs for current model names
client = anthropic.Anthropic()

with client.messages.stream(
    model=MODEL,
    max_tokens=64000,
    messages=[{"role": "user", "content": "Write a haiku about monsoon rain in Kuala Lumpur."}],
) as stream:
    for chunk in stream.text_stream:
        print(chunk, end="", flush=True)
print()

Example output — your text will differ
Afternoon thunder —
the Klang river swells and roars,
mamak stalls packed tight

client.messages.stream(...) takes the same arguments as create; the with block keeps the connection open and closes it cleanly. stream.text_stream yields pieces of text; end="" stops print adding a new line after each piece and flush=True shows it at once. Streaming also lets you set a larger max_tokens (64,000 here) without the request timing out.

Network calls fail in ways ordinary code does not. Some failures are worth retrying and some are not, so catch the SDK’s specific error classes, most specific first:

import time
import anthropic

MODEL = "claude-opus-5-5"   # check the provider's docs for current model names
client = anthropic.Anthropic(max_retries=2)   # the SDK already retries some failures

def ask(question, attempts=3):
    for attempt in range(1, attempts + 1):
        try:
            response = client.messages.create(
                model=MODEL, max_tokens=16000,
                messages=[{"role": "user", "content": question}],
            )
            return "".join(b.text for b in response.content if b.type == "text")
        except anthropic.AuthenticationError:
            raise SystemExit("Check ANTHROPIC_API_KEY - retrying will not help.")
        except anthropic.RateLimitError:
            wait = 2 ** attempt                     # 2, 4, 8 seconds
            print(f"Rate limited; waiting {wait}s (attempt {attempt})")
            time.sleep(wait)
        except anthropic.APIStatusError as e:
            if e.status_code >= 500 and attempt < attempts:
                time.sleep(2 ** attempt)            # server trouble: try again
            else:
                raise                               # 400-type errors: fix the request
        except anthropic.APIConnectionError:
            print("Network problem; retrying")
            time.sleep(2 ** attempt)
    raise RuntimeError("Gave up after several attempts")

print(ask("In one sentence, what is an API?"))

Run with an invalid key, this prints Check ANTHROPIC_API_KEY - retrying will not help. — the point of catching authentication separately. A rate limit error (HTTP 429) means you are sending too much too fast: wait and try again, waiting longer each time (“exponential backoff”). Status codes of 500 and above are the provider’s problem and usually pass. Other 400-type errors — a misspelt model name, a malformed message list — will fail identically every time, so retrying only wastes money and time. The SDK already retries connection errors, 429s and server errors twice by default (max_retries); your own loop is a second, visible layer you can log.

Prompt caching and system prompts

Prompt caching saves money when many requests begin with the same long text — a 50-page handbook, a big set of instructions, a fixed list of examples. The provider stores the processed prefix for a few minutes, and later requests that start with exactly the same text are billed at a fraction of the normal input price (for Claude Opus 5.5, cache reads cost 5% of the base input price As of October 2026). In the Anthropic SDK the simplest form is one extra argument, cache_control={"type": "ephemeral"}, on messages.create. Keep the stable text first and the changing question last: any change early in the prompt breaks the match, and very short prefixes are not cached at all. Check response.usage.cache_read_input_tokens to confirm it is working.

The system prompt deserves more thought than it usually gets, because it is where an application becomes a product. Two apps on the same model can feel entirely different because of a few paragraphs: who the assistant is for, what it should and should not do, its tone, the format of its answers, what to do when unsure. Treat it like product design: write it down deliberately, keep it under version control, and test changes against a fixed set of real questions before shipping them.

Consensus A system prompt shapes behaviour but does not guarantee it. Users, and text the model reads, can still push it off course, so anything that truly matters — not leaking data, not taking risky actions — needs checks in your code as well, a theme of Shipping an AI App.

The same call with OpenAI

Switching provider changes names, not ideas. Here is the basic call side by side; the OpenAI version follows the Responses API in OpenAI’s official quickstart As of October 2026.

import anthropic

MODEL = "claude-opus-5-5"   # check the provider's docs for current model names
client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY

response = client.messages.create(
    model=MODEL,
    max_tokens=16000,
    system="You are a friendly tutor. Answer in at most three sentences.",
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print("".join(b.text for b in response.content if b.type == "text"))
print("tokens in/out:", response.usage.input_tokens, response.usage.output_tokens)

from openai import OpenAI

MODEL = "gpt-6-astra"   # check the provider's docs for current model names
client = OpenAI()       # reads OPENAI_API_KEY from the environment

response = client.responses.create(
    model=MODEL,
    instructions="You are a friendly tutor. Answer in at most three sentences.",
    input="Why is the sky blue?",
)
print(response.output_text)
print("tokens in/out:", response.usage.input_tokens, response.usage.output_tokens)

Key terms on this page

Sources & further reading

• Anthropic, Using the Messages API (Claude Developer Platform documentation, 2026).
• Anthropic, Pricing and Token counting (documentation, 2026).
• Anthropic, Streaming messages, Errors and Prompt caching (documentation, 2026).
• Anthropic, Giving Claude a role with a system prompt (documentation, 2026).
• Anthropic, anthropic-sdk-python (GitHub; examples on this page checked against version 2.3.0).
• OpenAI, Developer quickstart and Text generation (OpenAI API documentation, 2026).