Testing a 1M context window with 45 million free tokens
The short answer
A stealth model called stealth/ox-alpha went free for a week with a 1M token context window, advertised with the line "we have capacity for 100 trillion tokens per day, lets see what you can do". So we hid a gate code inside 14 novels and asked for it back 192 times, at 8 context lengths and 8 burial depths, three runs each.
It came back correctly 71.4% of the time, and the shape of the failures is nothing like the story usually told about long context. Accuracy does not fall away as the window fills. It collapses in the middle and partly recovers, so the full 939,330 token prompt scored better than the 64,000 token one. The single worst place to put a fact was not the middle of the document. It was the very top.
44,614,640 input tokens, 68 minutes of upstream time, $0.00. Every number and every quoted answer on this page comes from that run. The model is live on this gateway: pick it in the playground or see it on the models page.
Somebody left the door open
On the 20th of August, OpenCode announced that a stealth model was free for a week: 1M context, multi-modal, tool calling, "generous rate limits, near unlimited usage". Stealth means the lab behind it has chosen to stay anonymous during the preview, which is now a normal way to test a frontier model in public without the launch.
We run a free LLM gateway, so an offer like that is not a curiosity, it is inventory. It was serving through this API within the hour. The rest of the day went on the question that actually matters once you have a 1M window in hand and no bill attached to it: does the window work all the way to the end, or is the last 900,000 tokens of it decoration?
The test
Needle in a haystack, the standard shape. Take a lot of text, plant one fact in it that cannot be guessed or known, ask for the fact back, and score exactly: either the planted number is in the answer or it is not. No partial credit, no judging model, nothing to argue about.
- Haystack: 14 public domain novels, 1,310,432 words of them, concatenated. The Count of Monte Cristo, Pride and Prejudice, Les Misérables, Great Expectations, The Adventures of Sherlock Holmes, Crime and Punishment and 8 more. At the largest size the prompt is about 7.2 novels long.
- Needle: "REMEMBER THIS: the gate code at the Nairobi Arboretum that morning was 4471, and the bench was the third one from the jacaranda." The code changes on every single run, so a model cannot score by remembering the last one.
- Grid: 8 context lengths from 1,000 to 1,000,000 tokens, crossed with 8 depths from the top of the document to the bottom. Three runs per square, 192 calls in total.
- Question: "What was the gate code at the Arboretum that morning, and which bench was it? Answer only from the document above."
The map
Every square is three runs. Click one to see what the model actually said, how long it took and how much it read. Switch the measure to see the same grid as time rather than accuracy.
Click any square above to read the run.
What the map says
The window is real. A fact planted in 939,330 tokens of nineteenth century prose does come back, and that is not a small thing. The prompt at that size is about 7.2 novels, more than most people read in a year, and the model reads all of it before answering a question about one sentence.
It does not decay the way everyone assumes. The expected picture is a clean slide from reliable to useless as the window fills. That is not what happened:
| Context | 1k | 4k | 16k | 64k | 128k | 256k | 512k | 1000k |
|---|---|---|---|---|---|---|---|---|
| Found | 22/24 | 22/24 | 18/24 | 11/24 | 15/24 | 11/24 | 19/24 | 19/24 |
The worst column is 64k, a length nobody thinks twice about sending. The best columns after the two small ones are 512k and 1M. A prompt fifteen times larger than the one that failed most did better. I have no mechanism to offer for that and I am not going to invent one, but it is a useful warning about the intuition it breaks: "keep the context small so it does not get lost" is advice this data does not support.
The dangerous place is the top, not the middle. The lost-in-the-middle result is well known enough to be folklore. This model lost the beginning. A needle in the first line of the document came back 11/24 times, the worst of any depth, while the same needle at 43% of the way down came back 21/24 times.
There is an unglamorous candidate explanation. At depth zero the needle sits above the opening line of a novel, where it reads like a header, a title block, something belonging to the document's furniture rather than its content. If that is what is happening, it matters more than a benchmark curiosity: the top of the prompt is exactly where people put the instructions.
It fails honestly. Of the 55 misses, 43 were the model saying, clearly, that the fact was not there: "The provided document does not contain any information about a gate code." Confident, wrong, and completely readable. Not once did it invent a code. That is the failure you want: a model that hallucinates a gate code is dangerous, a model that denies one it was given only wastes your time, and asking twice catches it.
The other 12 ran out of room mid-thought, with 1,500 tokens to spend on a one sentence answer. Which is the next story.
The first run measured our own bug
The first pass through this grid scored 2 out of 62. Almost every answer was an empty string. The obvious conclusion was that the 1M window was a marketing number and the model could not retrieve anything at all. Thirteen million tokens went into producing that conclusion.
The obvious conclusion was wrong. Every one of those calls had
max_tokens: 120 and returned exactly 120 completion tokens
with finish_reason: length. The model was thinking, silently,
and the thinking was eating the entire budget before a single character of answer could be written. Raise the
budget and the same question, on the same document, answers correctly:
| Request | Finish | Completion | Answer |
|---|---|---|---|
| max_tokens 120 | length | 120 | empty string |
| max_tokens 120, reasoning low | stop | 66 | "The gate code was 4471, and it was the third bench from the jacaranda." |
| max_tokens 2000 | stop | 376 | "The gate code was 4471 and the bench was the third one from the jacaranda." |
This gateway already knew about that failure. It carries a rule that
when a reasoning model is given a tight token cap, it should be asked for the least thinking it offers, so the
budget reaches the answer. The rule keys off the model name, and a model called
stealth/ox-alpha does not look like a reasoning model. It had
also been checked, on short prompts, where it honestly reports zero reasoning tokens and behaves like an ordinary
one. Give it real work and it is not ordinary at all.
So the experiment's first result was a patch to the router rather than a fact about the model. That is the argument for running the experiment on your own infrastructure instead of reading someone else's chart: the chart cannot tell you that your client is the thing that is broken.
Free capacity, and what it costs you instead
The second pass produced a different kind of nothing. 11 of its 64 calls came back as HTTP 200 with no content, no usage, no error and no explanation. Not a rate limit, not a refusal: a successful response carrying absolutely nothing, mostly at the 512k mark, where the upload is largest.
This is the real price of free inference and it is not paid in money. A client that trusts a 200 hands your user an empty answer and logs a success. It is the one failure that never appears in an error rate, which is exactly why this gateway counts an empty completion as a failure and fails over to the next provider instead of returning it. The final run retried them and lost none, which is the whole argument for a retry loop in one line.
A big window is a slow window
Accuracy is not the only thing that moves when you fill the context. The answer is one short sentence in every square of the grid, so this is the cost of reading, not of writing:
Median of 24 runs at each length, wall clock, from Nairobi, including the upload of the prompt.
Between the smallest and largest prompt the input grows about 716x and the wait grows around 13.1x, which is the good news: it is nothing like linear. It is still the difference between an interface that feels alive and one that needs a progress bar and an apology.
Race it
Numbers this large stop meaning anything, so here is the same task at a size a person can attempt. Below is 1,346 words of Jane Austen with one gate code hidden in it, exactly like the ones the model was given, which means it is exactly as conspicuous as the ones the model kept failing to find. Find the four digit code and type it in. No searching, that is cheating and you will know.
The model was given this exact passage and answered in 3.8s. It was also given 939,330 tokens of the same kind of thing and answered in 53.6s.
What we did not pay
The final run was 44,614,640 input tokens and 100,823 output tokens. Counting the two false starts, the whole investigation spent 66,323,024 tokens, which is the part that would normally have hurt: being wrong twice is most of the cost of any experiment. Here is the final run priced against other models with a 1M window, at their listed rates on the day:
| Model | Context | Per 1M in | This experiment |
|---|
Listed prices from one marketplace on 2026-08-21, input and output priced separately. The spread across models at the same context length is more than two orders of magnitude, which is worth sitting with before choosing where a long document goes.
What this does not show
One needle is the easy version. Retrieving a single distinctive sentence is the simplest thing a long context can be asked to do. Reasoning across facts that are 600,000 tokens apart is the hard version, and this run says nothing about it.
The needle does not belong. A Nairobi gate code in the middle of Moby Dick is conspicuous. A fact that blends into the surrounding text is harder to find, and real documents are full of those.
Three runs is three runs. Enough to separate a pattern from a fluke, not enough to rank models. Treat a single square as an anecdote and the columns as the result.
It is a preview, and it is temporary. The model is anonymous, the free window was announced as a week, and the marketplace page notes that prompts are retained by the provider although not used for training. That is the actual trade being offered, and it is a reasonable one for public domain novels and a poor one for anything confidential.
Try it before it closes
stealth/ox-alpha is routable on this gateway right now, and so are the
other providers it fails over to. It is not first in the chain, because at
4.1s on a small prompt it is nowhere near the fastest thing here, but it is the one to
reach for when the input is enormous or has a picture in it. Point the OpenAI SDK at
https://dreamprompting.com/api/v1 and ask for it by name, or
open the playground and pick it from the model list. The
API docs have the two line version.
When the free week ends, this gateway stops offering it on purpose rather than quietly starting to spend money, which is its own small story about what "free" means in this business.