Cheaper AI agents: what the savings depend on, and what to keep a person on

A YouTube video from the channel Discover AI, "JEV Mapped to 4B Qwen3 in Agent Harness: NEW CODE", uploaded October 18, 2026, walks through a paper submitted to arXiv three days earlier, JevAgentHarness, by eight authors from the University of Illinois Urbana-Champaign, Princeton University and Google. The idea is to let a small model make the routine multiple-choice decisions inside an AI agent, "such as which tool to call next," and to call the big model only when the small one is unsure.

The figures the video states match the paper, and near its end the video says "It is a first step. It is a prototype." What it leaves out is the plain LLM's score on the same tasks, the training bill, the license on the released model and data, and what the vendor's customer agreement says about the method. This note covers those, then gives four small tests an owner can run this week.

We read the video's auto-generated captions, not the picture, plus the paper, the authors' project page, code and model card, TypeSafe's documentation and customer agreement, and three independent preprints (abstracts, and one results section of the first), all on October 25, 2026, Central time. We ran no model. Every figure below is the authors' own unless we name someone else.

#What a decision model does

Jev, introduced by TypeSafe AI on September 15, 2026, does not write text. In the paper's words, "a decision model returns a probability for each listed option instead of generating text." The harness, meaning the program that gives the model its tools and runs the loop, asks it a question with a list of options. In the paper's cascade setup, "the decision model answers first, and the harness uses it when its confidence clears a threshold." We call that threshold a cutoff below. If the cutoff is not met, "the LLM decides as usual." The tests run on a simulated shop desk, DecisionGym, "a simulated customer service desk for a small shop," with Gemini 3.5 Flash as the agent LLM doing the other work.

Three words used below. A 4B model has about four billion parameters (Qwen3-4B has 4.0 billion), and the authors serve theirs on one graphics card. An adapter is a small add-on file that sits on a base model: AgentDecision-4B is a LoRA adapter on Qwen3-4B. A checkpoint is a saved version of a model during training, so v1 and the released model are two checkpoints.

#The numbers match. The question is what each is attached to.

The authors state most of these limits themselves. The paper has a section called "Where delegation fails," and the model card says "Under shift the threshold did not protect success." The video makes the same point near its end: "the strongest result really depend on the relevance and on the correctness of the environment-specific supervision," and "this does not demonstrate a universal capable decision model at all." What the captions never mention is the plain LLM's 100.0% on the shifted tasks, or the training bill, which the paper gives for the earlier checkpoint (see the break-even below).

#Where the savings came from

The paper's Table 2 splits the savings by part. Each row compares the agent without and with one part, against the plain LLM agent.

The first two rows cover different task sets, so they are not a like-for-like pair, though the paper's Finding 2 sets them side by side: "A fitted threshold protects success." Our reading: the big savings come with a small model trained on the agent's own states, which made 99.9% of the tool choices in a separate 500-task test. Jev's fitted cutoffs were 1.00 for tools and 0.99 for arguments, "so only near-certain answers are used," and it made 27.2% of the tool choices.

Whether the cutoff keeps protecting you when the work changes is mixed:

• Shifted tasks. Tools renamed, requests reworded, distractor tools doubled, new task shapes added. The in-domain cutoffs were carried over unchanged to 200 of them. Cost still fell, by 64.8% and 69.7% for the two 4B models, but success fell by 6.0 and 7.5 points from the plain LLM's 100.0%: "The two 4B models stayed confident and kept taking most tool choices, so success fell." The released model accepted 22 wrong tool choices and the earlier checkpoint 62; the exploratory 0.8B scoring head that learns to defer did worse still (64.5%, with 383 wrong). The authors conclude that "a new agent, new tools or a new task family call for refitting the thresholds and, for the best accuracy, training on that agent's states."
• A new business rule. A rule the small model had never seen, whether to cancel or ship by customer tier and order amount, left the agent solving 3.0% of tasks "against 88.0% for the LLM alone." After refitting, the cutoff "left every tool choice of AgentDecision-4B v1 to the LLM." For Jev, the paper's main text says "Refitting the threshold restored success with Jev," but its appendix reports two refits on hard tasks with different results: 19.0% against 42.0% for the plain LLM, and 89.0% against 87.0% once the final answer was reworded as a list per customer (+2.0 points on 100 tasks, interval 6 below to 10 above, in a table the paper labels exploratory). The main text sums up hard tasks as delegation that "kept or raised success but saved only" 4.6 to 30.5% and "ran slower."
• Coding agents. The paper reports cost falling 7.0 to 10.5% when Jev sent each step to a strong or a fast model, with "success varied by harness." The authors' alpha-stage proxy reports 28 to 64% less on three of four harnesses in a 74-task test, one run each, with a fast tier 4 to 7 times cheaper (the fourth, which its README calls inconclusive, cost 27% more), and 7 to 13% with a tier about 2.7 times cheaper.

Preprints independent of the paper's authors (we did not check their ties to TypeSafe), read by abstract and, for the first, its section 4.1, point different ways. One consulted Jev on whether a Claude Sonnet 5.5 terminal agent should stop, verify or change approach: "Jev raised cost per attempt by 19%" against the same agent with no controller (the abstract has Jev cheaper than a small-LLM controller, ratio 0.81) at "an identical pass rate (73.0% vs. 73.0%)" over 21 tasks. Another reports "95% success with 72.7% fewer strong-model calls" on 100 tasks and, on external BFCL and tau-style evaluations, "limited advantages over a cheap generative cascade when ordinary routing is already highly accurate." A third found that Jev "substantially reduces decision cost and latency across all five agents" but that "Mean cost per successful outcome increases in one agent even as spending per attempt falls," with task success "close to baseline levels on ALFWorld" but declining "more substantially in two mobile agents." They do not average into a forecast. For an owner the lesson is the last quote: count cost per finished task, not per attempt.

#The license limits, plainly

The video tells viewers "You have here the license part." It does not say what the license is. The authors' project page does: "The models and the training and coding data are released for non-commercial use under CC BY-NC 4.0. The benchmark is released under CC BY 4.0 and the code under MIT."

NC means non-commercial. The model card gives the reason for the data: it contains "outputs of commercial models: Jev's probabilities (TypeSafe) as soft labels, and agent text written by Gemini models (Google)." It tells readers to "Check those providers' terms before any use beyond non-commercial research, in particular before training a model that competes with theirs."

The second limit is the vendor's own contract. TypeSafe's Master Customer Agreement (the page reads "Last updated Sep 23, 2026") lists among its license restrictions that the customer will not "use the Services or any Output (defined below) to perform model distillation, train a model to imitate the output of the Services, or develop (or to facilitate the development of) a similar or competing product or service." The video's recipe trains a small model on Jev's probabilities. In our reading, for a customer who accepted that agreement, that is a listed restriction. Three limits on what we can say:

• It binds a customer who accepted it. The agreement says it is accepted by clicking a box, signing an order, or using or paying for the services.
• The paper's authors reached Jev through OpenRouter. The paper says "Jev is billed through OpenRouter," and section 7 of the agreement says third-party platforms are subject to the customer's agreement "with the relevant provider and not this Agreement." The paper says derived weights and data "are released only where the providers' terms permit," and lists Jev via OpenRouter under "provider terms" without saying which terms those are. We could not tell which terms covered the authors' calls. We are not saying anyone broke a rule.
• Owning output is not permission to train on it. Section 4.2 says TypeSafe "disclaims ownership of Output" and assigns what rights it has to the customer, while 2.3(b) restricts the use separately. That is our reading of the text, not legal advice.

The practical result for a business: the released adapter and training data carry a non-commercial license. The code is MIT and the benchmark is CC BY 4.0, which are not non-commercial licenses, and you should read them yourself. The benchmark data include every model's answers, Jev's among them, so the model card's providers'-terms warning may reach it too; we did not check. If you build your own, you would need your own checked outcomes as labels, which we have not tested, and you should check the terms before using any vendor's answers in that role.

#Our reading, and how well each part is supported

#Four small tests for this week

None needs code. Use only data you already send to that AI, and paste nothing private into anything new. If a test could cancel, send, refund or charge anything, switch those tools off first, or use a test account with a payment sandbox: a test account on a live payment setup can still move real money.

1. A shadow week. For five working days, every time your AI assistant picks from a menu (which template, which folder, who gets the ticket), add a row to a sheet, describing each situation in a few words with no customer details: what it saw, the options, what it picked, what was right. Add a column for what a plain rule would have picked, such as "anything from this sender goes to this folder." Change nothing. At the end, count how often the AI and the rule agree and how often the AI was wrong. Where the plain rule and the AI agree and the AI is rarely wrong, that pick is a candidate for the rule, no model needed.

Why it is not obvious: the paper's Finding 2 sets a cutoff fitted from shadow runs, which lost no task, against a fixed 0.5, which lost 21.5 points. They cover different task sets, so take the pair as a hint; a log is how you would see that trade on your own work before it costs you. The authors' proxy for coding agents has a shadow setting that "logs its answer next to what the agent did; nothing changes." Once a week, also read ten things the AI got through without a person and mark the wrong ones, because a steady approval rate proves little: under shift the cutoff still took 99.6 to 100% of tool choices, wrong ones included. Limits: the cutoffs were fitted on 150 calibration tasks, and the model card says "They hold only for that setting." A week of notes will not get you the paper's 96.0%, which came from 15,694 recorded states.

2. Shuffle the menu. Take ten items you sort with AI, such as emails into piles or leads into tiers. Run each item twice, once with the options in your order and once reversed, each time in a fresh chat so the AI cannot see its earlier answer. Run a few twice in the same order too, also in fresh chats, to see whether an answer flips on its own. Treat any item whose answer changes with the order, or on its own, as one you should not automate yet.

Why: TypeSafe's page on Jev's known weak spots says "the order of a Choice's options can affect the answer, and jev-1.13 leans toward the option that comes first," and its fix is to "Reorder the options and check the answer is consistent." That is a vendor's warning about its own model; we did not check whether other models lean the same way.

3. The ask-first list. List every action your AI can take that you cannot take back: send, cancel, refund, delete, publish, pay. Next to each, write "person" or "hard rule," meaning a plain rule in settings or code, such as "refunds over $50 always wait for me." Then test one case, only on a test account with a payment sandbox, or with the cancel, refund and send tools switched off, and send only to a test inbox. Give the agent an ambiguous request, such as "cancel my order" when the test customer has two orders, or "refund my order" without saying where the money goes, and see whether it asks before it acts.

Why: in the paper's failure analysis, with the decision model making every tool choice and no cutoff, "All 34 of Jev's failed tasks began with an irreversible wrong tool choice, usually cancelling where the agent should have asked." The paper adds that "Models trained on agent states made no such error" in that test, though under shift the two 4B models did accept wrong tool choices, as above. The model card says "Never make a decision model the only check on an irreversible action." The person-or-hard-rule list is our reading: no run in the paper tested a person or a plain rule in that role.

4. A two-minute contract search. Open the customer agreement of each AI service you pay for and search the page for three words: distill, imitate, competing. If you ever plan to train anything on a vendor's answers, read the sentence around the hits first. Zero hits proves little, because a contract may say "train" or "derive" instead; in TypeSafe's agreement those words also hit unrelated clauses, such as reverse engineering, derivative works, telemetry, and a promise that TypeSafe will not train on the customer's data without consent. For an open model, read the license name before it touches customer work: CC BY-NC means non-commercial.

Worked example: in TypeSafe's agreement the three words land in one sentence, 2.3(b), quoted above, and nowhere else. Not legal advice. Terms differ for someone using a reseller or gateway, and we did not read OpenRouter's. A separate signed agreement replaces the click-through one; the agreement's own opening says so.

#Before you build anything

The authors give their own break-even: "Training and calibrating AgentDecision-4B v1 cost $192.93, which this saving pays back after 5,759 episodes." That is the earlier checkpoint at its 81.1% saving, on simulated shop-desk runs. At 20 tasks a day, 5,759 task runs is about 288 days (our arithmetic, one episode as one run of one task, at about 3 cents saved per run, implied by the paper's two figures). Your tasks per day, cost per task and model-tier price gap set your number: multiply tasks per day by what a task costs now and by the saving you hope for, once low and once high, and divide the one-off cost by each result. The proxy's README advises: "Pick a fast tier that is much cheaper than the strong one, and measure on your own tasks."

Related field notes: Grade an AI agent by its decisions, not its wording covers the same month's decision-model work, Same task, twice shows how to test a savings claim on your own tasks, and Three AI videos mostly got their numbers right checks what a figure is attached to.

A cheap model can press the routine buttons. Which buttons are routine is something only your own log can tell you.

#What we did not check

• We ran no model, recomputed no table from the released data, and read the video's captions only.
• The three independent studies are arXiv preprints read by abstract (plus one results section of the first), and we did not check whether any has been peer reviewed.
• Every Jev figure in the paper is for Jev 1.13. TypeSafe's models page says "Jev 1.14 is the newest Jev release, available from October 14, 2026."
• We did not read OpenRouter's terms, and no lawyer has read any of this.
• Our arXiv abstract searches for the model and benchmark names found only the paper, so we found no independent replication. That is a search result, not proof that none exists.

#Sources

All read October 25, 2026.

• Discover AI (YouTube), JEV Mapped to 4B Qwen3 in Agent Harness: NEW CODE
• arXiv, JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs (2610.21412) and its full text
• JevAgentHarness authors, project page with the license table
• JevAgentHarness authors, code repository (MIT, status alpha)
• Hugging Face, AgentDecision-4B model card
• TypeSafe AI, Master Customer Agreement
• TypeSafe AI, Introducing System One Models and Jev, September 15, 2026
• TypeSafe AI docs, Jev 1.13 jaggedness, last reviewed October 2, 2026
• TypeSafe AI docs, Models
• arXiv, Good Judgment, Bad Advice: Decision-Model Controllers Raise Agent Cost Without Improving Success (2610.24649)
• arXiv, REFLEX with Jev for Efficient Selective Control in LLM Agents (2609.26532)
• arXiv, When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows (2610.13092)
• Creative Commons, Attribution-NonCommercial 4.0 International deed