---
title: "A video said a 4B model matches the best AI at 368 times less cost. The paper is credible; the comparison is narrower"
description: "We traced a 23-minute video on MIT's Prospector paper to its source and graded 32 claims. The lesson that lasts is giving agents search tools, not shrinking the model."
publishedAt: 2026-10-30
author: Ena Pragma
url: https://enapragma.co/field-notes/ai-4b-prospector-video-checked-the-paper-is-credible-the-368x-comparison-is-narrower
tags: ["ai-verification", "knowledge-graphs", "agent-design", "research-method"]
---

**The short version.** The paper is credible and the video reports almost all its numbers correctly; we did not run the code, so we cannot say it replicates. A 4B-parameter Qwen3.5 model with small LoRA adapters (a lightweight add-on trained on top of a frozen model) scores 0.839 on a 195-question semiconductor knowledge-graph test, against 0.846 for Claude Opus 5.5 and 0.877 for GPT-6 Astra, when every model drives the same three graph tools. Four of the video's 32 claims are overstated: that the 4B matches Astra (it is 3.8 points behind on one pass), that standard graph RAG is broken (the paper beat one kind of pre-selected subgraph, not full GraphRAG), that decision-level training is the only fix, and that the 4B is "comparable with the best LLMs that you can buy" (newer flagship models were not in the comparison). A fifth, "368 times lower cost," holds with a caveat: here 368 became about 170 against the nearest-accuracy model (Opus 5.5), and about 19 against a model scoring 0.59 (our arithmetic). The lesson that lasts is not the small model. It is that handing an agent search tools beat pre-loading it with retrieved text for every model tested, and that the biggest error left is finding the right answer and then giving a different one.

On October 30, 2026, at 9:30 AM Central, the channel Discover AI posted a 23:01 video called *Big Tech's Nightmare: MIT's 4B "Prospector" AI* ([YouTube](https://youtu.be/1ATJa_6dRoE)). It covers a paper submitted three days earlier ([arXiv 2610.37121](https://arxiv.org/abs/2610.37121)), by Haiqian Yang and Markus Buehler at MIT. The promise: a small model on a laptop that is "almost as good or maybe even better" than the biggest models, at "368 times lower cost." A video like that is a lead, not a source. This note shows how we checked it, what held, what did not, and what it means for how we build agents.

## How we checked it

We did not summarize the video. We treated it as a set of claims and went looking for the owner of each one.

1. **We kept the full transcript.** We pulled the whole text through a transcript library and cross-checked it against a second copy. Auto-captions garble names: the video's "QLoRA 3.4" is Qwen 3.5, and "Cloud Opus 5.5" is Claude Opus 5.5. The audio is all we had; we did not watch the slides.
2. **We split 23 minutes into 32 testable claims.** Each got an owner: the paper, the code, a vendor page or another study.
3. **We went to the paper, the code and the vendors.** We read the paper's full text, the repository README, the Hugging Face model card, and Anthropic's and OpenAI's own pages for model names and prices.
4. **We matched every quote against the source string.** Each row carries the exact words and a link, and each quote was checked against the one source it is credited to, not just any source. The checker was run against a planted wrong quote first, and it rejected it.
5. **We read the field around it.** A paper on its own tells you little about whether it matters, so we looked at other 2026 work on agents that search graphs.
6. **We asked what the video could not tell us.** What each number is compared with, what was held equal, and what the authors say against themselves.

Our tally of the 32 video claims: 13 hold, 14 hold with a caveat, 4 are overstated, 1 cannot be verified, and none is wrong.

## The breakdown: what the video does, in order

| Time | Part | What the host does |
|---|---|---|
| 0:00 to 0:32 | Hook | Promises a 4B model on a MacBook M4 Pro that matches the top models at "368 times lower cost" |
| 0:32 to 3:40 | Setup | Knowledge graphs ground a model; graphs are too big to read; classic graph RAG picks the subgraph before reasoning |
| 3:40 to 7:21 | The paper and the tools | A model chooses what to read next with three tools: find, expand, connect. Worked example on a semiconductor graph |
| 7:21 to 8:53 | Aside | Compares the idea to his own 2022 physics paper (not graded) |
| 8:53 to 11:28 | Why pre-selecting fails | Says pre-selecting costs the model "up to 40% in accuracy," and names a failure mode, decision misses |
| 11:28 to 14:34 | The fix | A four-stage training recipe, with a restart from the exact turn where the answer was decided |
| 14:34 to 17:10 | Results | Three test conditions; the local 4B against Opus 5.5 and Astra; 91.3% with sampling and a verifier |
| 17:10 to 18:46 | Transfer | The same model on three graphs it was not trained on |
| 18:46 to 23:00 | Code, speed, cost, privacy | Repo, MacBook against H200, price per 1,000 questions, keeping data local |

## What held up

The numbers are reported correctly. With the same three tools, the paper's Prospector-4B "reaches an accuracy of 0.839" against "0.877 for gpt-6-astra" and 0.846 for claude-opus-5.5. On three graphs from other fields it scored 0.803, 0.766 and 0.809, second on each to Astra at 0.857, 0.846 and 0.865. The tool definitions, the training order, the reward rule, the repo and license (Apache-2.0), and the worked example all match the paper.

The cost table is real too: $0.57, $96.73 and $209.76 per thousand questions for the 4B, Opus 5.5 and Astra. We confirmed the provider rates on the vendors' own pages ($4 and $20 per million input and output tokens for Opus 5.5; $10 and $50 for GPT-6 Astra). The tokens per question are the paper's recorded numbers, and we did not re-measure them.

The method also sits in a pattern, not alone. Search-on-Graph-R1 (July 20, 2026) "internalizes this navigation into a compact 8B model" with supervised fine-tuning and then reinforcement learning ([arXiv 2607.18481](https://arxiv.org/abs/2607.18481)). Small trained graph searchers beating prompted frontier models is a 2026 line of work, not one paper.

## What was smaller than it looked

**Matching Astra.** On one pass the 4B scored 0.839 and Astra 0.877, a gap of 0.038 (our arithmetic). With 195 questions, one score has a standard error near 0.026 (our arithmetic), so the gap is not proven, and a tie is not proven either. That 0.026 is our unpaired estimate; a paired comparison on the same questions would give a smaller gap, and the plus or minus 0.015 is the paper's own figure. The authors write that the model "comes close to the strongest frontier model" and "matches the second." The video says it matches both.

**"I can beat Astra and I can beat Opus 5.5."** That figure needs 32 sampled attempts per question and a trained verifier. The paper says "when we sample a pool of 32 episodes per item," and the score rises "to 0.913, against 0.839 for the single pass." Opus and Astra got one pass. The host says himself, "Now, it would be fair to do the same to Astra." The paper does not report that run.

**"368 times lower cost."** The arithmetic is right: $209.76 divided by $0.57 is 368 (our arithmetic). But that is the gap to the most expensive model only.

| Compared with | Accuracy with tools | Cost per 1,000 questions | The 4B's $0.57 is cheaper by (our arithmetic) |
|---|---|---|---|
| GPT-6 Astra | 0.877 | $209.76 | about 368 times |
| Claude Opus 5.5, the model nearest the 4B's 0.839 | 0.846 | $96.73 | about 170 times |
| deepseek-v4-pro, the cheapest off-the-shelf model that drove the tools | 0.590 | $10.72 | about 19 times |

The fair comparison is Opus 5.5, which scores within 0.007 of the 4B, so the honest gap is about 170 times. The 19 times compares the 4B with a model that scores 0.590, a weaker job, so we keep it only with that label.

The $0.57 is one pass on a rented GPU, priced "on a rented GH200" (a data-center GPU) with episodes run at the same time. The paper does not price the 32-sample and verifier setup that reaches 0.913. And the number leaves out the cost of building the graph, writing test questions with a larger model, and training.

**"Up to 40% in accuracy."** These are accuracy points, not percent. The paper says "the tools add between 0.16 and 0.40 over retrieval." The 0.40 is Astra: 0.877 with tools minus 0.477 with a pre-selected subgraph (our subtraction).

**Standard graph RAG is broken.** The paper's baseline is a subgraph around the right entity, chosen by a k-hop rule (everything within a set number of links) or a PageRank rule (a link-popularity score), not a community-summary GraphRAG system. On its own benchmark that baseline loses to tool use. A separate 2026 benchmark finds that "agentic search substantially improves dense RAG and narrows the performance gap to GraphRAG," while concluding that "GraphRAG remains advantageous for complex multi-hop reasoning" (questions that need several linked facts) ([RAGSearch, arXiv 2604.09666](https://arxiv.org/abs/2604.09666)).

**"Zero hallucinated edges."** The paper has a scoring gate: "an answer passes the gate only if every edge it cites" was one the tools returned. That checks provenance, not truth, and the trained policy still had 4 such citation misses, up from 3 before decision training.

**"Replicate this for your knowledge graph easily at home."** The repository README says `--graph <name>` on the evaluation command "serves the released policy on your graph without retraining." That is one documented command. Building the graph, writing test questions (in the paper's Section 6.2, "A teacher model then phrases a question that names the start entity and no intermediate node, in the form of a domain exam question") and training your own policy are not easy. On a Mac the first run "peaked at 18 GB of memory."

**"The best LLMs that you can buy."** Anthropic's docs list Claude Fable 5.1 above Opus 5.5, and OpenAI's page points to a newer model, GPT-6.1 Sol. Neither was in the paper's eight-model comparison. The result is one graph task with tools handed to every model.

## Claim by claim

Grades: **Holds**, **Caveat** (true with a named gap), **Overstated** (wider than the source), **Unverifiable**. "Paper" is [the full text](https://arxiv.org/pdf/2610.37121) of the October 27, 2026 preprint. "README" is the [GitHub repository](https://github.com/lamm-mit/Prospector).

| # | Video claim | Grade | What the source says | Source |
|---|---|---|---|---|
| 1 | A local 4B is almost as good as or better than Astra and Opus 5.5 for a fraction of the cost | Caveat | 0.839 against 0.846 and 0.877 on one pass; better only with 32 samples and a verifier | [Paper](https://arxiv.org/pdf/2610.37121) |
| 2 | It runs on a MacBook M4 Pro | Caveat | 1.2 questions per minute on a MacBook M4 Pro; the first run peaked at 18 GB | [Paper](https://arxiv.org/pdf/2610.37121), [README](https://github.com/lamm-mit/Prospector) |
| 3 | 368 times lower cost | Caveat | 368 against Astra only; about 170 against Opus 5.5 (the nearest in accuracy), about 19 against deepseek-v4-pro, which scores 0.590 | [Paper](https://arxiv.org/pdf/2610.37121) |
| 4 | The graph fixes stale model knowledge | Caveat | Not tested by the paper. Its problem is specialized, often proprietary knowledge, not recency; the host framed freshness as a general property of graphs | [Paper](https://arxiv.org/pdf/2610.37121) |
| 5 | Classic graph RAG selects evidence before reasoning | Holds | "The usual approach selects the evidence before reasoning begins" | [Paper](https://arxiv.org/pdf/2610.37121) |
| 6 | A 4B Qwen model with a rank-16 LoRA adapter, trained with RL | Holds | "the rank-16 adapter of stage 3 on Qwen3.5-4B" | [Paper](https://arxiv.org/pdf/2610.37121) |
| 7 | The recipe is SFT, full-episode GRPO, rejection sampling, decision-level GRPO | Caveat | SFT is supervised fine-tuning on examples; GRPO is a reinforcement learning method that scores groups of attempts. Order holds; the first stage copies a frontier model's demonstrations (gemini-3.7-flash), so the video's "we just need a rank 16 LoRA adapter" understates it | [Paper](https://arxiv.org/pdf/2610.37121) |
| 8 | An MIT paper dated October 27, 2026 | Caveat | Submitted Tue, 27 Oct 2026; an arXiv preprint, not shown as peer reviewed | [arXiv abstract](https://arxiv.org/abs/2610.37121) |
| 9 | The tools are find, expand and connect | Holds | Near word for word; without expand, accuracy falls from 0.841 to 0.398 | [Paper](https://arxiv.org/pdf/2610.37121) |
| 10 | Without the graph models mostly fail; a pre-selected subgraph helps but loses answers | Holds | Closed book 0.154 to 0.226; subgraph 0.328 to 0.508; with tools 0.549 to 0.877 | [Paper](https://arxiv.org/pdf/2610.37121) |
| 11 | Standard graph RAG is broken | Overstated | The baseline is a seeded k-hop or PageRank subgraph, not full GraphRAG | [Paper](https://arxiv.org/pdf/2610.37121), [RAGSearch](https://arxiv.org/abs/2604.09666) |
| 12 | Pre-selecting costs up to 40% | Caveat | 0.16 to 0.40 accuracy points, not percent | [Paper](https://arxiv.org/pdf/2610.37121) |
| 13 | A failure mode called decision misses | Holds | The model "retrieved the correct answer as a candidate but asserted a different entity" | [Paper](https://arxiv.org/pdf/2610.37121) |
| 14 | Retrieval misses fall with model strength; decision misses stay at 16 to 48 | Holds | Retrieval misses fall "from 41 of gpt-5-mini's 88 to 6 of gpt-6-astra's 24" | [Paper](https://arxiv.org/pdf/2610.37121) |
| 15 | Nobody had found this failure mode before | Unverifiable | A rhetorical aside, graded for completeness. The paper makes no priority claim, and we found no source that settles it; a 2026 paper on knowledge-graph question answering also sorts failures by type, on a different problem | [Paper](https://arxiv.org/pdf/2610.37121), [GRACE](https://www.mdpi.com/2073-8994/18/8/1340) |
| 16 | Decision-level GRPO restarts from the deciding turn; a gate guarantees zero hallucinated edges | Caveat | The restart holds; the gate checks provenance, and 4 citation misses remain, up from 3 before decision training | [Paper](https://arxiv.org/pdf/2610.37121) |
| 17 | Decision-level GRPO is the only way to stop decision errors | Overstated | Decision misses fell from 20 to 11 items, not to zero | [Paper](https://arxiv.org/pdf/2610.37121) |
| 18 | A correct grounded answer gets reward 1; others less | Holds | Other rollouts get a quarter of their gold-path recall, so up to 0.25 | [Paper](https://arxiv.org/pdf/2610.37121) |
| 19 | With tools, Opus 5.5 scores 84% and Astra 87% | Holds | 0.846 and 0.877 | [Paper](https://arxiv.org/pdf/2610.37121) |
| 20 | The local 4B matches Opus 5.5 | Caveat | 0.839 plus or minus 0.015 against 0.846; the gap is inside the noise of a 195-item test (our unpaired estimate of the error is about 0.026; a paired gap would be smaller; the plus or minus 0.015 is the paper's own) | [Paper](https://arxiv.org/pdf/2610.37121) |
| 21 | The local 4B matches Astra | Overstated | 0.839 against 0.877; the authors' own wording for Astra is "comes close", not "matches" | [Paper](https://arxiv.org/pdf/2610.37121) |
| 22 | 91.3% beats Astra and Opus 5.5 with sampling and a verifier | Caveat | The pool is 32, not 33; the baselines got one pass | [Paper](https://arxiv.org/pdf/2610.37121) |
| 23 | On three unseen graphs: 80, 76 and 80 against Astra 85, 84 and 86 | Holds | 0.803, 0.766, 0.809 against 0.857, 0.846, 0.865 | [Paper](https://arxiv.org/pdf/2610.37121) |
| 24 | The graphs are bio-inspired materials, PFAS and materials properties | Holds | Same wording in the abstract; all four graphs came from one extraction pipeline | [arXiv abstract](https://arxiv.org/abs/2610.37121) |
| 25 | Code, data and scripts are on GitHub | Holds | Apache-2.0 repository (the "MIT" is the university); weights on [Hugging Face](https://huggingface.co/lamm-mit/Prospector) | [README](https://github.com/lamm-mit/Prospector) |
| 26 | You can replicate it for your own graph easily | Caveat | Serving the released policy is one command; building the graph, test questions and training are not easy | [README](https://github.com/lamm-mit/Prospector) |
| 27 | Mac and H200 give the same accuracy; the Mac is slower, about 1.2 questions per minute | Caveat | The text says 1.2; the extracted figure data says 1.75. We could not tell which is right | [Paper](https://arxiv.org/pdf/2610.37121) |
| 28 | Per 1,000 questions: Astra over $200, Opus 5.5 about $96, the 4B 57 cents | Caveat | $209.76, $96.73 and $0.57; the $0.57 is one pass on a rented GPU | [Paper](https://arxiv.org/pdf/2610.37121) |
| 29 | The baseline prices are the posted rates | Holds | $4 and $20 (Opus 5.5); $10 and $50 (Astra) per million tokens | [Anthropic docs](https://platform.claude.com/docs/en/models/opus-5-5/overview), [OpenAI](https://openai.com/index/gpt-6-astra/) |
| 30 | Train on public data, then run on private data locally | Caveat | "a private graph can be served by a model that has never read it," but only public graphs were tested | [Paper](https://arxiv.org/pdf/2610.37121) |
| 31 | The 4B is comparable with the best LLMs you can buy | Overstated | Newer models from both vendors were not tested | [Anthropic docs](https://platform.claude.com/docs/en/models/opus-5-5/overview), [OpenAI](https://openai.com/index/gpt-6-astra/) |
| 32 | The worked example ends at the wafer-holding system | Holds | Matches the paper's Figure 1 and Appendix B | [Paper](https://arxiv.org/pdf/2610.37121) |

## What it means for how we build agents

This is our reading, labeled as ours. The small model is the least useful part for us. The useful part is in the paper's own tables.

**Tools beat pre-loaded text, for every model tested.** Opus 5.5 scored 0.508 when handed a pre-selected subgraph and 0.846 when it drove find, expand and connect itself, a gain of 0.338 (our subtraction). Across all eight off-the-shelf models the tools added between 0.16 and 0.40. We build agents on Claude. The question for any of them is whether it pulls evidence step by step or is handed a pile of retrieved text up front.

**The next error is not retrieval.** For the strongest models, most misses are a case of having the answer and giving another. In the paper's error table Opus 5.5 has 14 retrieval misses, 16 decision misses and 0 citation misses; Astra has 6, 18 and 0. More retrieval would not fix those. A last step that re-reads what was found and checks the answer against it may reduce them; we have not tested that.

**Training or serving a small model does not apply to us now.** Our agents run on Claude, so a price per thousand questions on a rented GPU is not our cost structure, and we have not tested the released adapters on our own hardware. We are watching, not building. The paper has no independent replication yet: the repository shows 7 stars and 1 fork on October 30, 2026, and our searches found no outside rerun. Search results are leads, not proof of absence.

**The limits are the authors' own too.** The test questions are generated from the graph by a model. It is one benchmark family. The paper also discloses that Claude Code (Opus 5, Opus 5.5, Fable 5 and Fable 5.1) helped write its software, which is the same stack we build on and not a flaw.

**A possible fit, not an established one.** Some client work may involve questions that cross several linked sources, which is the shape of a multi-hop graph question. We do not know whether any client faces enough of these to matter, and the privacy argument is untested on real proprietary data.

## What we plan to do with it

This is a plan, not something we have measured. We intend to build a bank of about 30 multi-hop questions from our own records, then score three arms with Claude: closed book, pre-loaded recall, and step-by-step tool traversal in the find, expand and connect style. We will label every miss as retrieval, decision or citation, and try one gate that lets an answer cite only items retrieved in that run, adopting it only if accuracy holds or rises. We have no results yet and will not claim any until we have a baseline.

## Six things to try with your own agent

None of these needs the paper's model. Each follows from a measured piece above, but the paper measured graph tools on a knowledge graph, not document search, so the transfer is our assumption. We have not run them on your system, so treat each as a test, not a promise.

1. **Give the agent lookup tools instead of a pasted pile.** Let it search, open one item and follow links, rather than receiving everything retrieved in advance. The paper's gain for tools over pre-selected text was 0.16 to 0.40 of accuracy (16 to 40 points).
2. **Run a three-arm test on 30 of your own questions.** Closed book, pasted context, and tools. The gap between the last two is your number, not the paper's.
3. **Label every miss.** Retrieval miss (never found it), decision miss (found it, answered differently) or citation miss (cited something it never retrieved). The fix depends on which dominates.
4. **Add a final check.** Before answering, have the agent re-read the evidence it retrieved and confirm the answer is in it. It may reduce decision misses; test it.
5. **Test a citation gate.** Accept an answer only if everything it cites was returned by a tool in this run. Measure accuracy with and without it, and the share of answers it rejects.
6. **Price a claim before you repeat it.** When you see a claim of *N times cheaper*, ask against which model, on which setup, and compare with a model that does the same job. Here 368 became about 170 against the nearest-accuracy model (Opus 5.5), and about 19 against a model scoring 0.59.

Here is a prompt block for ideas 3 to 5. It is our adaptation, not wording from the paper. Paste your material in place of the brackets.

```
Task: answer the question using only items you retrieve with the tools.
1. Search for the entities in the question. Open each one before you use it.
2. Follow links one step at a time. Note each item you retrieve.
3. Before answering, re-read the items you retrieved. Is the answer in them?
4. Cite only items you retrieved in this session. If you cannot cite it, say so.
5. If your draft answer differs from a candidate you retrieved, say why.

Question: [your multi-hop question]
```

## What we did not verify

- We did not run the repository or recompute any benchmark number. The numbers are the authors', checked against the paper's own text.
- Tokens per question, which drive the cost table, are the paper's. We confirmed only the vendors' posted rates.
- The Mac speed is unresolved: the text says 1.2 questions per minute and the extracted figure data says 1.75.
- We found no independent replication or critique. We ran a few search queries, and they returned the paper and the repository. Absence from one index is not proof.
- We did not find a source that settles who first named decision misses, so claim 15 stays unverifiable.
- The transcript is auto-captioned, so wording may differ from what was said where names were garbled. We had audio only, not the slides.
- The video is the host's work. We quote it sparingly and use none of its frames.

Related field notes: [A video said AI can now reason about time. The headline gain depends on which baseline you pick](/field-notes/ai-time-reasoning-video-checked-the-headline-gain-depends-on-which-baseline-you-pick) runs the same check on another video, and [Three AI videos mostly got their numbers right. The slip is in what each number is attached to.](/field-notes/three-ai-videos-mostly-got-their-numbers-right-the-slip-is-in-what-each-number-is-attached-to) checks three more.

## Sources

All read October 30, 2026, Central time.

- [YouTube, Discover AI, Big Tech's Nightmare: MIT's 4B Prospector AI, October 30, 2026](https://youtu.be/1ATJa_6dRoE)
- [arXiv, Prospector paper, full text (2610.37121), submitted October 27, 2026](https://arxiv.org/pdf/2610.37121)
- [arXiv, Prospector paper, abstract page (2610.37121), submitted October 27, 2026](https://arxiv.org/abs/2610.37121)
- [GitHub, lamm-mit, Prospector repository, last commit October 27, 2026](https://github.com/lamm-mit/Prospector)
- [Hugging Face, lamm-mit, Prospector model card, created October 28, 2026](https://huggingface.co/lamm-mit/Prospector)
- [Anthropic, Claude Opus 5.5 model docs, released September 22, 2026](https://platform.claude.com/docs/en/models/opus-5-5/overview)
- [OpenAI, GPT-6 Astra, page updated September 29, 2026](https://openai.com/index/gpt-6-astra/)
- [arXiv, RAGSearch (2604.09666), April 1, 2026](https://arxiv.org/abs/2604.09666)
- [arXiv, Search-on-Graph-R1 (2607.18481), July 20, 2026](https://arxiv.org/abs/2607.18481)
- [MDPI Symmetry, GRACE (18(8):1340), August 8, 2026](https://www.mdpi.com/2073-8994/18/8/1340)