APIs and SDKs: How Code Talks to Models

One accidental standard

First, three words you will see everywhere, in plain language:

• API — a way for your code to ask another computer for something and get an answer back. You send a request, you get a response. Like a form on a website, except your program fills it in.
• Endpoint — the specific address you send that request to, like a URL.
• SDK — a ready-made library for your language that hides the raw HTTP plumbing. Without an SDK you build the request by hand; with one you write client.chat.completions.create(...) and it handles the rest.

That is genuinely all three of them mean. Everything below builds on them.

Now the best news in this whole track: when OpenAI released their chat API in 2023, the request format turned out to be so convenient that nearly the entire industry adopted it. Today an enormous number of providers — including open-model servers like Ollama and vLLM — expose an OpenAI-compatible endpoint: an address that accepts requests in that same shape.

What that means for you: learn one request shape, and you can talk to dozens of models. The shape is:

• a model name,
• a list of messages, each with a role (system, user, assistant, tool) and content,
• optional sampling parameters (temperature, max tokens — everything from the previous article),
• a response that includes the message and a usage object with token counts.

One more piece of vocabulary, before any code: that shape is usually written in JSON — a plain-text format for structured data. Curly braces hold an “object” (a set of named fields), square brackets hold a list, and every field is written as "name": "value". It looks like code, but it is just a form filled in with text — deliberately boring, so that any language and any server can read it. When you see {"role": "user", "content": "..."} below, that is JSON: a tiny form with two fields, role and content.

And here is the part that ties the whole course together: that request is the one call everything else is built on. One POST with text in, one continuation back — there is no second kind of request. The chat window, the agent, the RAG pipeline: each is a program making this same call again with different text. An SDK is only a wrapper that builds that text and runs the loop for you. The whole idea in one place: The one call: how every AI product actually works.

The OpenAI-compatible API in practice

1from openai import OpenAI
2 
3client = OpenAI()  # reads OPENAI_API_KEY from env
4 
5response = client.chat.completions.create(
6  model=#98c379">"gpt-4o-mini",
7  messages=[
8      {#98c379">"role": "system", "content": "You are a concise assistant."},
9      {#98c379">"role": "user", "content": "Explain prompt caching in 2 sentences."}
10  ],
11  temperature=0.7,
12  max_tokens=200,
13)
14 
15print(response.choices[0].message.content)
16print(response.usage.total_tokens, #98c379">"tokens")

Look past the syntax and notice the same skeleton everywhere: model, messages or contents, some parameters, a response with text and usage. The differences worth remembering:

• Claude: the system prompt is a separate top-level parameter, not a message with a role. You must always pass max_tokens.
• Gemini: flatter style — contents can be a plain string for simple cases; config carries parameters.
• OpenAI-compatible: system is just the first message. This is the dialect most third-party providers speak.

What is an API key, really?

An API key is a long secret string that identifies you to the provider — partly a username, partly a credit card. Every request carries it, and the provider bills whoever the key belongs to. That is the whole mechanism.

Two rules follow directly:

1. The key is money. Anyone who has it can spend your balance. There are bots that scan public GitHub repositories for accidentally committed keys around the clock; a leaked key is usually abused within minutes.
2. Keep it out of your code. Put it in an environment variable — a value stored on the machine itself, outside your files, that your program reads at startup. That is why the examples above create a client with no visible key: the SDK reads OPENAI_API_KEY from the environment. Your code stays shareable; the secret does not travel with it.

Watch one request go out and come back

The code above is nice, but what actually travels over the wire? Step through it with “send next →”: this is a single request/response cycle, the same one your code performs every time. The first line, POST /v1/chat/completions, is just the address (the endpoint) plus the request type — POST means “create something with this data”, the verb your code uses when asking a model to generate. Nothing here is a chat in the human sense — the model has no idea a previous request ever happened.

Notice the two things people miss:

• The system message is just an item in the list, not a special setting — except for Claude, where it is a separate parameter.
• usage tells you exactly what you just paid for. It is the single most useful number in the whole response, and the easiest one to ignore.

Streaming in Node

For interactive UIs you want streaming (see the previous article). Here is how ordinary it looks in JavaScript:

import OpenAI from "openai";

const client = new OpenAI();

const stream = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Write a haiku about tokens." }],
stream: true,
});

for await (const chunk of stream) {
const delta = chunk.choices[0]?.delta?.content ?? "";
process.stdout.write(delta); // prints token by token
}

That loop is the entire UX trick behind “the AI is typing”: each chunk is a few tokens, appended to the UI as they arrive.

The Hugging Face ecosystem

Hugging Face is to open models what GitHub is to code. Three parts matter for an AI Engineer:

• The Hub — over a million model weights, datasets and demos. Llama, Mistral, DeepSeek, fine-tunes of everything: if weights are public, they are here.
• Inference — run hosted open models through a simple API (the huggingface_hub client or their chat-style endpoints), so you can try models without downloading anything.
• Transformers / Transformers.js — the library that loads a model locally and runs it in Python. Transformers.js even runs small models in the browser via WebGPU — no server at all.

A survival kit for working with APIs

Whatever the provider, the same four concerns come up in every real project:

• Rate limits — a cap on how much you may send per minute. You will hit them. Implement retries with exponential backoff from day one: if a call fails, wait 1s, then 2s, then 4s, instead of hammering the server. The official SDKs often do this for you.
• Errors and timeouts — models are reached over the network. Treat every call as fallible; decide what the UI shows when generation fails.
• Token accounting — read the usage field, log it, aggregate it. Cost surprises are a top reason LLM projects get cancelled.
• Version pinning — model names change and get deprecated. Pin the exact model version in config (a dated name like gpt-4o-mini-2024-07-18 rather than a moving label), and plan a migration path before the deprecation notice arrives.

Check yourself