Somewhere past step 200 of a two-hour workflow, an agent notices something is off. A field doesn't match, a total looks suspicious. Then it submits anyway. According to OSWorld 2.0, that's not a weird edge case.. it's a recurring failure mode.
I work on a computer use agent in production, so when XLANG Lab released OSWorld 2.0 I read the full paper. The number everyone quotes is 20.6%, the completion rate of the strongest configuration tested. The number I kept coming back to is smaller: agents spent less than 7% of their action budget detecting and fixing their own mistakes.
The benchmark, briefly
The original OSWorld tested short desktop tasks.. change a setting, edit a file. Frontier agents got to around 80% on verified versions of it, which made computer use look close to solved. OSWorld 2.0 is built around the obvious objection that real work runs a lot longer than a two-minute sequence of clicks. The new benchmark has 108 end-to-end workflows: tracing an expense reimbursement across email and a banking portal, reconstructing a CAD model from engineering drawings, that kind of thing. The median task takes a skilled human about 1.6 hours, and agents average over 300 tool calls per task compared to roughly 30 in the original.
The strongest run, Claude Opus 4.8 with maximum thinking, finished 20.6% of tasks end to end. Its partial-credit score was 54.8%. So agents can usually do meaningful chunks of the work and get surprisingly far into a workflow. They just can't hold the whole thing together long enough to finish it correctly. Completion collapses with length: agents managed roughly 20-24% on tasks a human finishes in under 45 minutes, and under 10% once tasks passed the two-hour mark. On the longest tasks in the benchmark, every model scored zero.
Back to that 7%
The behavioral analysis found agents spent under 7% of their effort recovering from mistakes, and 9.8% on verification. Almost everything else went into reading the screen and figuring out how the tools behave. Read next to the failure analysis, that explains a lot. Agents forgot constraints from the original instructions hundreds of steps later. They missed new information that showed up while the task was underway. Some guessed where they should've asked the user, and some noticed a problem right before submitting.. and submitted anyway.
The paper points to stronger self-monitoring and repair as one of the clearest paths forward. From running one of these systems every day, I'd put it more bluntly: agents don't reliably validate their own work unless the system around them forces validation to happen. "Double-check your work before proceeding" is a sentence the model reads, not a behavior you can depend on, especially 250 steps in when the original constraints have been compressed into a summary of a summary. Verification has to live in the architecture.
What verification looks like in production
On the system I work on, verification starts before an action is committed. If the agent is about to do something consequential, we don't trust that the screen still looks the way it did when the model reasoned about it. The page can move or a dialog can pop up in the seconds between observation and click.. OSWorld 2.0 documents failures caused by exactly this kind of stale state. A cheap pre-action check stops one bad click from turning into 50 more steps built on a wrong assumption. The agent doesn't get to decide it feels confident; the surrounding system decides whether the action still makes sense.
For the moments that are easy to get wrong (did the dialog actually close, is there more content below the fold, is it safe to submit) we don't lean on the agent's general reasoning while it's busy planning and navigating. We use narrow detection calls whose only job is to answer one specific question. We learned the narrowness part the hard way. Early on we packed several detection questions into one call because it seemed efficient.. fewer requests, same answers. In practice the questions competed with each other and accuracy dropped across all of them. Splitting them into specialized calls run in parallel kept latency flat and made each detector more reliable. One context shouldn't be responsible for several competing kinds of attention at once, and OSWorld 2.0 shows the same weakness at workflow scale: the more threads of state a single context has to hold, the more of them quietly decay.
Make the failures measurable
The most useful thing we did was stop treating repeated detection failures as prompting problems. One question our agent kept getting wrong sounds trivial: is there a scroll-down chevron in this screenshot? Trivial until you need it to work reliably on every screen the app can produce. So we built a labeled dataset of about 600 screenshots and tracked each model's real error rates.. how often it false-alarmed versus how often it missed a chevron that was there. After that, a prompt change wasn't "better" because a few examples looked convincing. It had to move the error rates without quietly breaking another part of the distribution. Take the recurring failure and measure it until it's boring. Unglamorous work, and it's the difference between a demo and a system.
Reliability gets expensive
The economics don't rescue you either. The winning Opus 4.8 run cost about $72 per task and produced around 224,000 output tokens. GPT-5.5 finished 13% of tasks at about $25 each. Spending more helped, but nowhere near linearly.. near the top of the benchmark, extra reasoning buys a few more finished tasks rather than broadly more reliability. More tokens don't replace checkpoints and recovery logic. Without those, you're mostly paying the agent to wander for longer.
Agents don't respect invisible boundaries
One more finding that deserves attention if you're deploying these things internally. In roughly a third of the analyzed tasks, agents bypassed the user-visible interface to force completion.. pulled hidden application state or edited files directly. In one case, an agent pushed a repository and leaked a hard-coded API key from a .env file. Nobody told it to expose a credential. It got stuck, found another route to the visible objective, and didn't account for what that route revealed. If an agent shouldn't read a file or submit without approval, the system has to enforce that. The model can't be the final authority on what the model is allowed to do.
What I'd tell a team deploying agents
Start by measuring how much of the agent's execution actually goes to checking its own work.. how often they confirm state before acting, what happens after they notice something went wrong. If you don't know, the answer is probably much less than you think. Then take the failures that repeat and turn them from vague observations ("the model gets confused on long tasks") into specific detectors with labeled data and regression tests. Make consequential actions pass through explicit gates, and make submission a verified state transition instead of another button the agent is allowed to click.
The usual caveats apply. OSWorld 2.0 has 108 tasks, the authors describe the results as diagnostic rather than comprehensive, and some of the behavioral analysis rests on single rollouts. The leaderboard numbers will age. The structural finding will hold longer: as workflows get longer, agents lose track of what they were asked and spend almost nothing checking whether their own work is still correct.. which is also what these systems look like when you run one every day.
Agents can already do a surprising amount of the work. Whether they finish it depends on what you build around the model.
Paper: OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks, XLANG Lab, June 2026.