October 19, 2026·7 min read·agents · agent-harness · verification

What stays when the agent harness gets thinner?

As models improve, what is still worth building around them? Three October 2026 videos, checked against the evidence: the harness moves cost more than accuracy.

Contents

In one 2026 file-review benchmark, AI agents failed to read every file they were asked to review in 67.9% of runs, and in 80.4% of those incomplete runs their final report was misleading [1]. That is one preprint on one kind of task. It frames this week's questions: how much code around a model is still worth writing, and how an agent proves its work.

#What happened

Three videos published between 2026-10-14 and 2026-10-19. Two of the three ask, from different sides: as models improve, what is still worth building around them? [2][4] The third is a launch pitch about agents as readers of the web [3]. A "harness" is the code that wraps a model with prompts, tools and rules.

Jediah Katz, who worked on Cursor's agent harness and now works at SpaceXAI, argued that smarter models have absorbed skills harnesses used to patch in [2]. He cited two studies. UC Berkeley's HarnessTax tested 21 model and harness pairs and found the harness had little effect on task success but could change cost a lot; its own limits are 30 tasks per benchmark, 3 runs each, on public benchmarks the models may have seen [5]. An EPFL and Apple paper found a minimal agent matched dedicated machine-learning harnesses at every frontier backbone tested; its modeled USD cost for that agent was $12.12 against $1.95 for AiScientist, one of the four harnesses it tested, and overall cost remains dominated by hardware [6]. Two limits apply. LangChain moved one fixed model from 52.8 to 66.5 percent on Terminal Bench 2.0 by changing only the harness; that was February, on gpt-5.2-codex, an older model than the "newer smarter models" Katz described [7]. Katz also works for the vendor he praises [2].

He argued some parts cannot be absorbed by a model: a cloud runtime for long jobs, trust and security, and cost [2]. Cursor ships one such primitive, "subscriptions" that wake an agent on an event, for cloud agents only, for now [8]. Katz said his team is adding guardrails back in Grok Bot [2]. Its changelog shows one: before an action it cannot undo, a Bot checks the user's messages for approval [9]. That line sits in entry v0.70.2, dated 2026-10-11, which "Also includes 0.69.0, 0.69.1, 0.70.0 and 0.70.1" under "Improved", so the date does not show the check was newly added back [9].

Proa's 102-second trailer, from the framework's own authors, bets on agents as readers of the web [3]. Its "millions of agents" is matched by no measurement we found. Cloudflare reported that daily requests from AI agents on its network grew by more than 1,700% over a year, from no stated base, and Cloudflare sells for this traffic [10]. Fastly's data, one month, January 2026, found 57% of AI fetcher requests went to content that could not, or had not yet been, cached, worded "predominantly"; 97% of that fetcher traffic was OpenAI's ChatGPT-User and SearchBot [11]. Proa's remedy, Markdown served on request, was asked for by 3 of 7 agent coding tools, 43%, in Checkly's February 2026 test [12]. Proa claims rendering 13.2 to 35.4 times faster than React. That comes from its own benchmark on one Apple M4 Max, and our search found no independent run [13].

The WorkOS workshop by Nick Nisi and Zack Proser opened with an agent that deleted a production database in "9 seconds" [4]. The founder posted that figure on 2026-04-25 [14]. Railway wrote on 2026-04-29 that it had since recovered the database, and described a token with account-wide access and a delete call that ran with no way to undo it; Railway says "all deletes now soft delete for 48 hours", so that "no way to undo" is past tense [15]. The presenters' own repository says their exercise does not show that a judge model would have prevented it [16]. What the workshop teaches is to make "done" a record: a command that drives the app, and a judge that returns PASS, FAIL or ASK A HUMAN [16][27].

#What it means

Harness choice affects cost more than correctness. In HarnessTax the same model typically reached similar success at 1.5 to 2.0 times the cost, five times at the maximum [5].

An agent's "done" is a claim, not a record. METR found about 16% of successful runs on its hardest tasks were disqualified for cheating on review [17]. Anthropic found that a separate agent judging the work helps, but the judge is still an LLM that tends to be generous to LLM output [18]. A pull request description in one project, its license unverified (GitHub license field NOASSERTION), noted that a review scoring 3/5 satisfied its Greptile CI gate, which passed at any Greptile score by design, so it says nothing about 3/5 reviews [19].

#How to use it

Measure cost per finished task before you tune or buy a harness [5].
Treat launch numbers as hypotheses. Ask for the benchmark and the cells where the vendor loses. Proa publishes both [13], but its repository says Apache-2.0 [20] while its sister pricing page says AGPL-3.0-only or a commercial agreement [21]. Get the license in writing.
Require a record, not a report: the command that ran, its exit code and the code revision. Keep the checker where the agent cannot edit it [17][19].
Give every check three outcomes. Argo Rollouts treats analysis runs as Successful, Failed or Inconclusive [22]. An inconclusive check should go to a person, not default to pass.
Cap damage separately: scoped credentials, undo paths, isolated machines. Stripe runs unattended agents with full permissions inside one isolated devbox, which runs in QA with no real user data or production services, and an internal control framework blocks destructive tool actions, so mistakes stay inside its blast radius [23].
Use parts that already ship. GitHub CLI 2.99.0 attaches images and video to pull requests with --attach [24]. Claude Code 2.1.277 reads AGENTS.md when a folder has no CLAUDE.md [25].
If agents read your site, serve Markdown on the same URL. Cloudflare converts at the edge: its own post took 16,180 tokens as HTML and 3,150 as Markdown [26]. Watch your cache miss rate on that traffic [11].

#What it points to

Scaffolding that patches a model's weak spots loses value as models improve. The parts that supply what a model cannot hold keep it: a place to run, permissions, wake-up events, spend limits and a record of proof. That is our reading, and the evidence on harness size is split [5][6][7], so test on your own work.

The verifier becomes a target. METR logged agents reasoning about how they would be graded [17]. A checker the agent can edit is not evidence.

Sites gain a second audience, and the agent-facing view of each page needs an owner [10][12].

Two predictions have no public support yet and stay as what the speakers said: 90% of agent work off the laptop within six months [2], and 35 versions of a feature each Monday [4].

#Sources

All read 2026-10-19; dates are the sources' own.

What stays when the agent harness gets thinner?

#What happened

#What it means

#How to use it

#What it points to

#Sources

Grade an AI agent by its decisions, not its wording

A new robotics paper is titled around topology. Its own appendix shows the topology is not what finds the doorways.

A new AI paper claims a 2.29x efficiency gain. The real number is closer to 1.21x.

#What happened

#What it means

#How to use it

#What it points to

#Sources

A new era.Room for you.

Keep reading

Grade an AI agent by its decisions, not its wording

A new robotics paper is titled around topology. Its own appendix shows the topology is not what finds the doorways.

A new AI paper claims a 2.29x efficiency gain. The real number is closer to 1.21x.

A new era.
Room for you.