We Paid for 36 Images to Answer One Question
OpenAI released ChatGPT Images 2.5 on September 8, 2026, with two new API models: gpt-image-2.5-flare and gpt-image-2.5-sunburst. The launch post said Flare delivered images at "two to four times the speed" of gpt-image-2.
We ran it. Three models, identical prompts, three repetitions per cell, 36 generations and edits billed to our own account. The speed claim is wrong, and it is wrong in OpenAI's favour.
On our runs gpt-image-2.5-flare generated a 1024x1024 image in a median of 16.7 seconds against 117.8 seconds for gpt-image-2. That is 7x, not 2-4x. It also cost a quarter as much.
Here is the method, the numbers, and the two places where the older models still hold their own.
How We Ran It
Three models: gpt-image-1 (shutting down October 23, 2026), gpt-image-2, and gpt-image-2.5-flare. Every call used the same prompt string, size=1024x1024 and quality=high. The only variable was the model.
Three tests survived our pilot. We threw two others away, and it is worth saying why: a six-string pricing card and a transparent-background icon were both passed cleanly by all three models. A test everything passes measures nothing. The tests below are the ones that actually separated the generations.
Latency is wall-clock time around the API call. Cost is computed from the token counts the API itself returns, not from our own estimates. Every model is stochastic and none of them expose a seed parameter, so we ran three repetitions per cell rather than trusting a single sample.
Speed and Cost: The Result That Matters
Nine generations per model, medians below.
Two things stand out. The first is that gpt-image-2 is the slowest model in the lineup by a wide margin - four times slower than the model it replaced. Nine runs, every one of them between 111 and 160 seconds. That is not variance, that is the model.
The second is the cost. All three bill on the same published token rates, yet Flare comes in at a quarter of gpt-image-2. It is not cheaper per token; it simply spends about four times fewer output tokens to produce an image of comparable complexity. We checked that it was not buying speed by producing simpler images: every output was a true 1024x1024, and edge-density measurements on Flare's files sat within a few percent of gpt-image-2's.
One caveat we will not bury: one of our nine Flare generations stalled at 142.6 seconds against a 15 to 17 second baseline. Eight of nine ran fast. Budget for the occasional slow call.
Where the Older Models Still Compete
Flare's advantage is almost entirely in generation. When we ran the same edit against a single frozen source image, the picture changed completely.
Editing is fifteen times cheaper than generating on both current models, and gpt-image-2 edits as fast as Flare does. If your workload is mostly iterating on an existing asset rather than producing new ones, the upgrade case is much weaker than the headline suggests. gpt-image-1 is the one with no argument left: slower and twelve times more expensive per edit than either successor.
Counting Is Where the Generation Gap Shows
We asked for exactly seven identical mugs in a row, the third from the left in red, every handle pointing right. Counting has been the longest-standing failure in image generation and no lab claims to have solved it.
gpt-image-1. Eight mugs, running off both edges of the frame, red in the fourth position. Two of the three requirements missed.
gpt-image-2. Seven mugs, third from left red, all handles right. Correct on all three.
gpt-image-2.5-flare. Also correct on all three, in a seventh of the time.
Counting was fixed between generation one and generation two. Flare holds the gain rather than extending it. If you are still on gpt-image-1 for anything involving a specific number of objects, that is a real reason to move, independent of speed.
The Test That Produced the Most Interesting Failure
We asked for a photorealistic cylindrical coffee bag with three exact strings on the label, type following the curve of the bag.
gpt-image-1 rendered all three strings correctly. It also drew a flat-fronted gusseted bag rather than a cylinder, which meant the type never had to wrap around anything. It got full marks on the text by quietly avoiding the hard part of the brief.
gpt-image-2.5-flare produced an actual cylinder with the type arched correctly around it, consistent lighting, and all three strings intact. This is the finding that will not show up on any benchmark table: the older model satisfies the letter of a prompt by simplifying the geometry, and you only notice when you compare them side by side.
Watch for prompt requirements a model quietly drops rather than fails. Those are far harder to catch in review than an obvious mistake, and they are the ones that reach a client deck.
What We Changed in Our Own Stack
We generate header images for this blog and asset packs for client work through the API, so this is not academic for us. Three changes:
- Default to gpt-image-2.5-flare for generation. Seven times faster and four times cheaper is not a close call. The header on this post was generated with it, in 18 seconds.
- Do not bother switching for edit-heavy work. Flare and gpt-image-2 edit at the same speed and the same price.
- Retire gpt-image-1 now, not in October. It loses on counting, it dodges hard geometry, and it costs twelve times more per edit. The shutdown date only forces what the numbers already argue for.
We have argued for a long time that image generation is a distraction relative to the boring automation that actually moves a P&L, and we still think that is broadly right. What has changed is the failure rate. When a model gets the object count right, wraps type around a cylinder, and does it in sixteen seconds for five cents, image generation stops being a demo and starts being a production step you can put in a pipeline without a human checking every frame.
The wider point is the one we keep making about every model launch this month: vendor benchmark claims are a starting hypothesis. It took us 36 API calls and $4.47 to find out that OpenAI understated its own model by roughly half. Run your own five tasks. If you want help building that evaluation into how your team ships, that is what we do.
Sources
- Introducing ChatGPT Images 2.5 - OpenAI
- Image generation guide and model pricing - OpenAI Platform docs
- Deprecations, including the gpt-image-1 shutdown date - OpenAI Platform docs
- Generating images with the GPT Image models - OpenAI Cookbook
- openai-python SDK, used for every call in this benchmark - GitHub