OpenAI WebSockets: why agents with many tool calls get faster
OpenAI's Responses API has a WebSocket mode. What it does, why agents with many tool calls get up to 40% faster, and when you can skip it.
A coding agent hunting down a bug rarely makes a single model call. It reads a file, runs a test, reads the error, changes something and runs the test again. Twenty or thirty round trips between model and tools are nothing unusual. And every round trip carries the same overhead: a fresh HTTP request, and a server that has to reassemble the state of the conversation before the model can continue.
That overhead is what the WebSocket mode of OpenAI's Responses API goes after. OpenAI announced it at the end of February 2026. The connection stays open, and each turn sends only what is new. According to OpenAI, workflows with more than twenty tool calls run roughly 40% faster end to end.
What is WebSocket mode in the Responses API?
WebSocket mode is a way to call the Responses API over a single long-lived WebSocket connection instead of one HTTP request per response. Over that connection you send response.create events with the same payload as a regular Responses call, minus the stream and background fields. The response comes back as a stream of server events.
Keep this separate from the Realtime API. That API has used WebSockets since it launched in October 2024 (and now supports WebRTC as well), and it is built for speech: audio in, audio out, with as little delay as possible. WebSocket mode in the Responses API targets agents that work with text and tools, such as coding agents and orchestration loops.
Where does the speed-up come from?
The model does not compute any faster over a WebSocket. The gain is in what happens between two model turns.
In an agent loop over HTTP, a turn looks like this: the model asks for a tool call, your code runs it, and you send a new request with the result. That request either points to the previous response with previous_response_id or resends the whole conversation. Either way, the server first has to retrieve the conversation state.
In WebSocket mode, OpenAI keeps the most recent responses in an in-memory cache tied to the connection. A follow-up turn does not need to load that state from disk, and you only send the new input. Often that is nothing more than the tool result:
{
"type": "response.create",
"model": "…",
"store": false,
"previous_response_id": "resp_…",
"input": [
{
"type": "function_call_output",
"call_id": "call_…",
"output": "{\"tests\": \"3 failed\"}"
}
]
}
How much that saves per turn depends on your context; OpenAI only publishes end-to-end numbers. What is clear is that the saving scales with the number of round trips. A chatbot answering one question will barely notice. An agent that calls a tool thirty times skips the same overhead thirty times. That is why OpenAI positions the mode for workflows with a lot of back-and-forth between model and tools.
How much faster is it in practice?
The numbers come from OpenAI and a few early adopters, not from independent measurements. OpenAI cites roughly 40% faster execution for workflows with more than twenty tool calls. According to InfoQ, Vercel reported up to 40% lower latency, Cline 39% on multi-file workflows and Cursor up to 30%. OpenAI says its own Codex moved most of its Responses traffic to WebSocket mode.
The key word is "up to". These are the best cases: long chains of many small steps. If your application makes one call per user question, a persistent connection mostly gives you extra code to maintain.
What changes in your agent architecture?
A connection lives for an hour at most
After 60 minutes you have to reconnect. An agent that runs longer needs recovery logic. The documentation lists three routes: continue with previous_response_id if the response was stored, start over with the full context, or continue from a compacted version of the conversation via the /responses/compact endpoint.
The cache belongs to the connection
With store: false or Zero Data Retention, the previous response exists only in that cache. If the connection drops, a follow-up turn returns a previous_response_not_found error. If you store nothing at OpenAI for privacy reasons, you need to keep the context on your side to resume after an interruption.
Several tasks over one connection
A single connection can handle up to 16 responses at once, spread over stream_ids. Responses with the same stream_id run in sequence; different ones run in parallel. Useful when an agent fans out subtasks, but you have to track which event belongs to which task.
You can warm up the connection
Send response.create with generate: false and OpenAI prepares the system prompt and tool definitions without generating anything. The first real turn builds on that via previous_response_id.
WebSockets or WebRTC?
That choice comes up with the Realtime API, and OpenAI is clear about it: browser and mobile clients should connect over WebRTC, which is more robust "in most cases". A backend talking server to server uses a WebSocket. A WebSocket from the browser works with a short-lived token, but it is not the recommended route.
With the Responses API, the choice is between plain HTTP requests and WebSocket mode.
What breaks when you put this in production?
- Reconnecting. Networks hiccup, and the connection ends after an hour regardless. Test what your agent does when the socket drops in the middle of a tool call.
- State. In long sessions you need to decide what to keep, summarise or discard, especially with
store: false. - Measuring. Log events with timestamps, without sensitive content, so you can see where the time goes: model, tool or network. Only then do you know whether switching pays off for your workload.
So count how many round trips your agent makes per task first. Two or three? Skip this. Twenty or more? Then this is a speed-up you get without changing your model or your prompt.
