From completions to native APIs: how an LLM call works now
If your code treats an LLM call as text going back and forth, with messages and tool definitions in and text and tool calls out, it’s missing a significant part of what the latest models offer. A response now contains encrypted reasoning, results of tools run by the provider and, optionally, conversation state held by the provider. Code that passes only the text along still works in some cases, but it gets less out of the model. According to OpenAI, its reasoning models perform better on its native Responses API than on Chat Completions and make better use of the prompt cache. In some setups, the model also loses its earlier reasoning between tool calls without any error.
This post shows what an LLM call looks like today, which parts only each provider’s native API supports and what to check in your own setup. I ran into these problems in my own system, which calls several providers and models through Pydantic AI in the application and Bifrost as the gateway, but these findings apply to other setups as well: parameters that worked with earlier models were dropped, and the model lost its earlier reasoning while the response looked normal. This first post stays at the provider level. My earlier article on provider fragmentation covers the economics behind the shift.
In short: for the latest models, and especially with tools, use each provider’s native API. Chat Completions still works for plain generation, but it has no standard field for the model’s reasoning between tool calls, and most hosted tools and all server-side conversation state need the native APIs.
From completions to chat completions #
OpenAI’s API started in 2020 with a completions endpoint: one prompt string in, a continuation of that string out. Any structure, such as a dialogue between a user and an assistant, lived inside the prompt as plain text, and the application had to parse the output itself.
In March 2023, three months after ChatGPT’s launch, OpenAI released Chat
Completions together with gpt-3.5-turbo, the model behind ChatGPT.
Instead of a single string, the model consumed “a sequence of messages together
with metadata”, each message with a role such as system, user or
assistant. Function calling followed in June 2023. The application sends the
messages and a list of tool definitions, and the model answers with the tool
calls it wants. The application runs the tools and sends the results back with
the full history, and the model writes the final answer. The provider uses
nothing from an earlier request, so every request contains the whole conversation. That
state is what most developers and users in general still have in mind.
Anthropic and Google had their own versions of more or less the same behavior.
Anthropic moved from Text Completions, a single prompt with Human: and
Assistant: turns, to the Messages API at the end of 2023, and Google launched
the Gemini API with generateContent in December 2023.
Which endpoint, when, and where each one is heading
OpenAI
/v1/completionssince 2020, legacy/v1/chat/completionssupported; newest models need Responses for toolsFirst model on Chat Completions.3GPT-5.4 · Mar 2026
Tools on Chat Completions only with reasoning off.4GPT-6 Astra · Sep 2026
Tool calling only on Responses.
/v1/assistantshosted tools and threads, shut down Aug 2026Requests to
/v1/assistants and /v1/threads no longer work. The migration path is the Responses API./v1/responsesMAIN APIsince Mar 2025, recommendedFirst model only on Responses.
Anthropic
/v1/completeText Completions, legacyText Completions removed.
/v1/messagesMAIN APIsince Dec 2023, all featuresAccepts
temperature, top_p and top_k only at their defaults.8API version 2026-10-21 · Oct 2026Sampling parameters rejected on every model.
/v1/chat/completionsadapter since Feb 2025, "for testing"Thinking on by default; the adapter doesn't return it.
…/models/{model}:generateContentsince Dec 2023, legacy since Jun 2026First model on
generateContent.10Gemini 3 Pro · Nov 2025Function calls need thought signatures.
/v1beta/openai/chat/completionsadapter since Nov 2024/v1beta/interactionsMAIN APIbeta Dec 2025, default since Jun 2026Only on Interactions.
Each provider fragmented in its own way. OpenAI added endpoints, first the
Assistants API with hosted tools and server-side threads, then Responses, and
moved its newest capabilities there: o1-pro was the first model only on
/v1/responses, and GPT-6 Astra and GPT-6.1 Sol accept tool calls only
there. Anthropic kept
/v1/messages and changes the rules per model and API version instead, which
is why its red markers are on the same bar. Google moved from generateContent
to the Interactions API, where its agents run. Responses, Messages and
Interactions are the APIs each provider recommends or treats as its default,
and the 2027+ zone is my forecast that providers will add new capabilities only
there. The faded orange bars still reach today, so a Chat Completions request
is accepted by all three providers.
Why it felt like a standard #
For a while, Chat Completions seemed to me like a standard the industry had converged on, but in late 2025 it became clear that it isn’t. The format felt universal because almost everything around the providers adopted it. OpenAI shipped first and had the largest developer base, so SDKs, tutorials and frameworks were written against its API. Newer providers such as Mistral offered compatible APIs, so switching meant changing a base URL, and gateways like Bifrost put the same format in front of everything else.
Google and Anthropic accept it too, through adapters on top of their own APIs. Google’s arrived in November 2024 and Anthropic’s in February 2025, and both treat it as a secondary option next to their own API. Chat Completions became the shared format through the ecosystem around the providers.
Translating between the formats worked because a conversation consisted of text, function calls and function results, which every format could express one to one. A gateway could turn a Chat Completions request into a Messages request and back without losing anything that mattered.
Reasoning becomes the default #
Reasoning models changed what an assistant turn contains, and they did so in three steps.
Hidden reasoning: OpenAI’s o1, released in September 2024, reasons before it answers. The reasoning tokens are billed as output but not returned. The application has nothing extra to store or send back, so Chat Completions still fits.
Optional thinking with a budget: Anthropic introduced Claude 3.7 Sonnet in February 2025 as “the first hybrid reasoning model”. Thinking is off by default and limited by a token budget. When it’s on, the response contains thinking blocks with a signature, and those blocks have to be sent back unchanged when a tool call follows. Since thinking was disabled by default, the shared format kept working after upgrading.
Reasoning enabled by default, and it has to round-trip: with the latest models, thinking is enabled by default, so the same upgrade now breaks the shared format, in a different way for each provider.
- Anthropic: adaptive thinking is on by default on Claude 5 models, and Opus 5.5 doesn’t accept switching it off. Adaptive means the model decides per request whether and how long to reason, guided by the effort setting, so a simple question can still get no reasoning at all. Thinking blocks are “Required” within a tool-use turn.
- Google: Gemini 3 and 4 models require the thought signatures from a function-calling response to be sent back in the current turn and return a validation error otherwise.
- OpenAI: from GPT-5.4, Chat Completions supports tool calling only with
reasoning_effort: "none", and GPT-6 Astra and GPT-6.1 Sol need the Responses API for any tool calling. Responses can return reasoning as encrypted items for applications that don’t keep state at OpenAI.
The newest models add a further condition: the conversation before the
reasoning has to stay unchanged. Anthropic’s newest models check that a
replayed thinking block still follows the conversation prefix that produced
it, and depending on the model and account they reject the request when it
doesn’t. On the Responses API, OpenAI’s newer models drop the reasoning that
follows a changed prefix and list it in input_transformations. Rewriting the
instructions, changing the tool definitions or shortening an earlier tool
result each counts as a change.
In all three cases the assistant turn now contains an opaque, provider-signed blob that only that provider can read and that has to be sent back unchanged. While Anthropic and Google returned the raw reasoning with their first reasoning models, none of the three does anymore. Anthropic cites misuse as the reason, and OpenAI cites safety monitoring and competitive advantage. Since the model still needs that reasoning in the next request, it’s returned encrypted, and the signature lets the provider detect any change.
One tool loop, then and now #
The clearest way to see what changed is one question that needs tools, sent once to Chat Completions and once to Anthropic’s Messages API.
Two requests per tool call
get_weather tool, and the news needs a web search./v1/chat/completionsThe whole conversation so far plus the tool definitions.
The model asks for a tool call. The API keeps nothing after sending it.
Repeats every message and adds the tool result, because the API stored nothing.
There's no search tool on this side, so the answer has the weather but no news.
The API keeps nothing between 2 and 3. Your app holds the whole conversation.
On Chat Completions with a model like GPT-4o, a question that needs one local
tool takes two requests. The first returns tool_calls, and the second contains
the full history plus the tool result and returns the answer. The provider
uses nothing from the first request to answer the second.
/v1/messagesOpenAI and Google work similarlyYou declare it, Anthropic runs it inside the turn.]
On Claude 5 the text is empty by default. The signature contains the reasoning, encrypted., tool_use, server_tool_useSearch call
The model asked for both tools at once, so the search waits for request 2.
Sent back as received, after an unchanged system prompt, tools and history. Otherwise the request fails, or thinking is dropped for it. + tool result
The search from response 1 runs now, with no extra step in your code. Results are returned as
web_search_tool_result blocks.Built from your tool result and the search results Anthropic fetched.
Still two requests, and your app still holds the history, now including signed thinking it can't read, unless you opt into server-side state.
The current version still takes two requests, and the application still holds the history, but three things were added:
- Hosted tools: the provider can run tools such as web search or code execution inside the turn and returns their results as part of a response.
- Signed thinking: when the model reasons, the response contains a thinking block that has to be part of the second request exactly as it was returned. On Claude 5 models the block usually contains no readable text at all, only the signature with the encrypted reasoning. If the block is modified, Anthropic’s API rejects the request. If it’s missing, the request still succeeds, but the model continues without its earlier reasoning.
- Optional server-side state: the provider can keep the conversation
itself, through
previous_response_idon OpenAI’s Responses API,previous_interaction_idon Google’s Interactions API or message threads on Anthropic, so the second request only sends what’s new.
To see what this looks like in practice, I sent the example directly to Anthropic’s API. Here’s the first request and response, with the IDs changed and the signature and code shortened:
The first round trip, from a real run
- 1No
thinkingparameter, so Opus 5.5 runs adaptive thinking and decides by itself whether to think. Switching it off returns a 400, as does a non-defaulttemperature. At the default effort,medium, it didn't reason for this question in four of four runs, so this request setshigh. - 2A function tool: name, description and JSON Schema. Your app runs it, as on Chat Completions.
- 3A server tool, declared only by its type. The provider runs it inside the turn.
- 4The whole conversation so far, here just the question.
- 5Empty by default on Claude 5 models. With
"display": "summarized"in thethinkingparameter, it holds a summary of the reasoning. The model did reason here: 69 of the 278 output tokens are thinking. - 6The reasoning itself, encrypted. Request 2 has to send this block back unchanged.
- 7Your tool, the block closest to Chat Completions'
tool_calls. - 8The search from 3, requested but not run yet. The model asked for your tool at the same time, so the search runs during request 2 and its results are part of response 2. This tool type searches from Python, hence
code_execution. - 9Your app runs
get_weatherand sends request 2: the history, this content unchanged and the tool result.
Most of what matters in the response can’t be read. The thinking text is empty (5), although 69 of the output tokens are reasoning, which is encrypted in the signature (6). The search (8) is only requested here. Since the model asked for the weather at the same time, the search runs during the second request, and its results are part of the second response.
In the second request, my application sent that content back together with
the weather result. To see how strict the round trip is, I also sent the second
request with the thinking block left out, with one character of its signature
changed and with text added to its empty thinking field, each case directly
and through Bifrost, my gateway, in front of the same API. The full example
with the search doesn’t get through Bifrost, because it adds an empty code execution
result for the pending search to the conversation, and Anthropic’s API rejects
the second request with a 400. For a clean comparison, I repeated the loop with
only the weather tool:
The thinking block in request 2: four cases
The same response 1 with 36 reasoning tokens, weather tool only, sent back directly and through my gateway.
| What request 2 sends back | Anthropic's API, directinput tokens of request 2 | Through my gatewayBifrost v2.5.1, same API behind it |
|---|---|---|
| Thinking block unchanged | used37 tokens more than without the block: the reasoning from response 1. The model continues without reasoning again.HTTP 200 · 521 input tokens | reasons againBifrost deletes thinking blocks with empty text before it forwards a request, and Claude 5 returns them empty by default.HTTP 200 · 484 input tokens |
| Thinking block left out | reasons againAccepted, because in adaptive mode no assistant turn has to start with a thinking block. The earlier reasoning is gone.HTTP 200 · 484 input tokens | reasons againHTTP 200 · 484 input tokens |
| Signature changed in one character | 400 error"Invalid signature in thinking block", as documented: "Modified thinking blocks are rejected with a 400 error."HTTP 400 | reasons againDeleted by Bifrost for its empty text, like the unchanged block, so the changed signature never reached Anthropic's API.HTTP 200 · 484 input tokens |
Text added to the empty "thinking" field |
used, text ignoredSame count as the unchanged block: the reasoning from the signature is used, the added text isn't read. The one documented exception: text in the empty field of an omitted block is ignored rather than rejected.HTTP 200 · 521 input tokens | used, text ignoredWith text in it, the block isn't deleted. It reaches Anthropic's API with its signature, which uses the reasoning and ignores the text, as directly.HTTP 200 · 521 input tokens |
The input token count shows whether the block reached the model: with it, the count is 37 tokens higher, the earlier reasoning. On Anthropic’s API, all four cases behave as documented, including the 400 for the changed signature. Through the gateway, even the unchanged block was dropped. Claude 5 models don’t return their reasoning as text by default: the thinking block is returned with an empty text field, and the reasoning is only in the encrypted signature. Before Bifrost forwards a request, it removes every thinking block with an empty text field, so these blocks never reached Anthropic’s API, even unchanged. The block with the changed signature was removed the same way, so the 400 never happened. On this route, the model lost its reasoning in three of four cases, each time with a normal response. I added both gateway problems, with reproductions, to Bifrost’s open issues: the dropped thinking blocks to #7064 and the added result for the pending search to #9591.
Why no shared standard emerged #
My April post on this topic describes why the divergence is rational for providers: once base models converged in quality, providers competed on capabilities and shaped their APIs around their own products (Why it’s rational). Server-side tool loops, signed reasoning and long-running agents each need fields and states that a shared schema would first have to agree on. I don’t see a reason for any provider to wait for that agreement while it competes on these features.
The closest attempt is Open Responses, a specification derived from OpenAI’s Responses API. It was published in January 2026 and handed to the Agentic AI Foundation in October 2026. It standardizes the request and response format, while hosted tools remain vendor extensions. Neither Anthropic nor Google implements it natively, and Google calls its own Interactions API its primary interface. I expect Open Responses to become the common format for gateways and open-weights model servers, while the large providers keep shipping new capabilities on their own APIs first.
What still works on Chat Completions #
Chat Completions hasn’t gone away. OpenAI’s migration guide states that it “remains supported”, and every provider and gateway still accepts it in some form. What it supports has shrunk, though: text generation still works with most models, tool calls while the model reasons and prompt caching work only with limits, and hosted tools and server-side state mostly need the native APIs.
What Chat Completions still covers
OpenAI's own endpoint and the adapters from Google and Anthropic, compared with each provider's native API.
native: ResponsesGoogleadapter
native: Gemini, InteractionsAnthropicadapter
native: Messages
reasoning_effort: "none".none on GPT-6 Astra and GPT-6.1 SolProviders add new capabilities to the native APIs. OpenAI recommends Responses for all new projects, and xAI, one of the providers that adopted the format, now marks its Chat Completions endpoint as deprecated.
What I’d do #
What Chat Completions costs: for the latest models, it costs features and
response quality. From GPT-5.4, OpenAI’s models accept tools there only
with reasoning switched off, and GPT-6 Astra and GPT-6.1 Sol not at all.
Anthropic’s adapter never returns the thinking, so the next request can’t send
it back, and the model continues without its earlier reasoning. OpenAI’s own
numbers point the same way: according to its
migration guide,
reasoning models on Responses show “better model intelligence” and use the
cache 40% to 80% better in its internal tests. Most hosted tools and all
server-side conversation state need the native APIs. Where Chat Completions
keeps the reasoning, it uses extension fields, such as Google’s
thought signatures or the reasoning_details field of OpenRouter and Bifrost,
and only clients that know the extension send it back.
Recommended: one native API per provider. Send each request in the format
of the provider whose model answers it: Responses for OpenAI, Messages for
Anthropic and Interactions for Google. A framework with one model class per
provider does this for you, such as Pydantic AI with OpenAIResponsesModel and
AnthropicModel. Keep the reasoning parts of each response in the history
exactly as they were returned, and send them back with the next request.
What I wouldn’t do:
- Run agents through a compatibility adapter. Google and Anthropic treat theirs as a secondary option, and Anthropic’s never returns the thinking.
- Edit or drop reasoning blocks to save tokens. A changed block fails on Anthropic’s API, and a dropped one costs the model its earlier reasoning.
- Rewrite earlier turns, instructions or tool definitions during a tool loop, for example to shorten old tool results. Anthropic and OpenAI then reject or drop the reasoning that follows.
- Take a normal response as proof that the model received its reasoning.
How to measure: for each model, check which endpoint your SDK, framework or
gateway actually calls. To check whether the model receives its reasoning, send
the second request of a tool loop once with the thinking block and once without
it, and compare the input tokens. Send both through the same route as your
application, gateway included. Anthropic’s token counting endpoint gives the
count without running the model, but it rejects requests with server tools,
and a direct count doesn’t show what a gateway changes. If both counts are
equal, the reasoning doesn’t reach the model. On OpenAI’s Responses API,
input_transformations in the response lists the reasoning items it dropped.
For the broader picture, six common assumptions and what to check for each:
Six assumptions, checked against current behavior
2026-10-21 rejects them on every model, even at the defaults.Check: the current Anthropic SDK sends 2026-10-21 by default. If your requests set sampling parameters, remove them before upgrading.Conclusion #
Every LLM call still runs on the loop that Chat Completions made popular. What changed is what has to travel through it. Chat Completions still handles plain generation, but with the latest models it rejects tool loops, quietly drops the model’s reasoning between tool calls, or returns it only in extension fields that every layer has to keep. For tool loops, use each provider’s native API. New capabilities will likely only arrive there.