ena pragma
·5 min read·ai-market · ai-verification · ai-vendor-selection

Astra for Law is real. The benchmark a buyer actually needs is not on the page.

OpenAI's Sept 17 legal AI launch is genuine and well-partnered, but its only benchmark compares itself to the same model with plain web search, not to any competitor.

Contents

On September 17, 2026, OpenAI launched Astra for Law: GPT-6 Astra configured with a legal search index, legal-specific tools, and named design partners at real law firms. It is a genuine, well-documented product. It is also the third nearly identical launch from a major AI lab into the legal vertical in five months, and the one number a law firm buyer would actually want, how it stacks up against a competing lab's model, does not appear anywhere on the announcement.

#What shipped, and who is actually using it

Astra for Law is not a new base model. In OpenAI's words it "combines GPT-6 Astra, our latest and most powerful model, with settings, tools, and context tailored for professional legal work," and it appears in the model picker as "GPT-6 Astra Law." Its search index covers "more than 230 million URLs, with sources added daily," paired with the Free Law Project's CourtListener case-law collection. The named partners are specific, not anonymous logos: Sullivan & Cromwell, Ropes & Gray, Cooley, Skadden, Latham & Watkins, and Wachtell, Lipton, Rosen & Katz, three of them (Sullivan & Cromwell, Cooley, Latham) quoted by named partner and title and the other three by firm statement, plus API customers Harvey and Legora and ecosystem integrations with Thomson Reuters, iManage, Intapp, and others. The launch page itself frames the tool as complementary, not a replacement, naming Thomson Reuters directly as a provider firms "rely on" alongside the new tool.

#The benchmark that never compares to a competitor

OpenAI's headline claim is that Astra for Law reaches "54.0%" overall correctness on a private validation set of Vals AI's Legal Research Bench, against "38.7%" for plain GPT-6 Astra using web search alone, a 40% relative improvement. Every comparison on the page is Astra-for-Law against OpenAI's own prior setup. The one moment the page invokes a competitor, two hand-picked worked examples against Claude Fable 5.1, one litigation and one transactional, is OpenAI narrating its own prompts, not benchmark data.

The public version of that same benchmark family tells a different story. On Vals AI's public Legal Research Bench leaderboard, Anthropic's Claude Opus 5 and Claude Fable 5.1 share the top spot at 55.29% strict all-pass accuracy, alongside Meta's Muse Spark 1.3 Max; the leaderboard's own summary says the three "share the lead on strict accuracy at 55.29%." Plain GPT-6 Astra sits on the same public board at 39.4%. These are not the same test conditions as OpenAI's private validation set, so this is not proof OpenAI's own number is wrong. It is proof of something else: the one comparison a buyer evaluating "which AI is best for legal research" would actually want to see is available on the public record, and OpenAI's launch page does not cite it.

#The throughline: this is the third lab into this exact vertical, not OpenAI's alone

Astra for Law looks like a standalone bet until you place it next to the rest of 2026. Anthropic launched Claude for Legal on May 12, four months earlier, alongside a separately announced Freshfields legal-tools partnership three weeks before that. Google launched Gemini Enterprise for Legal on August 25, three weeks before OpenAI's move. Thomson Reuters, the incumbent whose CoCounsel Legal connector Astra for Law lists as "forthcoming," had already expanded a Claude/CoCounsel Legal integration via MCP four months earlier: "New MCP integration brings CoCounsel Legal into Claude workflows." The same legal-tech incumbent wiring itself into two competing labs on nearly identical terms, rather than picking one, is itself the clearest evidence that no single lab has locked up this market.

There is a second tension the page does not resolve. Harvey, the standalone legal-AI vendor OpenAI's own page lists as an API customer cleared to build on Astra for Law, raised at a $15.5 billion valuation eight days before this launch. OpenAI is simultaneously treating Harvey as a partner and shipping a first-party product, sold to the same law firms, into the market Harvey just raised over a billion cumulative dollars to serve.

#The risk category none of the three launches name

Legal AI is launching into a market with an actively tracked, growing sanctions problem: a public database of court decisions addressing AI-hallucinated content in legal filings had identified 2,041 cases as of its September 14, 2026 update. None of the three labs' 2026 legal launches, Astra for Law included, name this risk by its own tracked scale. It is not a reason to distrust any of these products specifically; it is the baseline risk against which "the model is accurate enough" claims should be read, regardless of which lab is making them.

#What a law firm or business buyer should do with this

The real story is not "OpenAI solved legal AI." It is that three frontier labs appear to have concluded the same thing: legal is a well-documented, high-value professional vertical worth a specialized retrieval layer, named design partners, and a governance story, and all three built nearly the same shape of product within five months of each other. That convergence is the signal worth taking seriously, more than any single lab's launch. Before committing to one vendor, ask for the same benchmark family's public numbers across labs, not just the private number the vendor chose to publish; a vendor's launch page can be accurate on every line and still leave the finding that matters in a document it merely links to. And ask what change in your own review process the accuracy gain is meant to justify, since, as one independent read of the launch put it, "a fifteen-point gain changes how much checking is needed. It does not change whether checking is needed."

#Sources

Keep reading