A chat model writes an answer. Jev, the decision model John Lindquist demoed live on October 8, does not. You hand it a sentence and a menu. It picks one option and returns a score for every option on the list. Your code does the rest.
Joel Hooks hosted. About an hour of demos, then questions. This Drop is the part you can carry back to a codebase: what kind of question Jev is for, and where ordinary code should take back over.
#Three questions, one request
John's grocery example is the whole model in one picture.
A shopper types a sentence. The app asks Jev whether that sentence changes the cart. "Will it rain tomorrow?" comes back near zero. "Add two coconut yogurts" comes back near certain. Yes or no is not a boolean. It is a probability from 0 to 100, and the shop's code sets the cutoff.
The second question is pick one. The menu is the functions the app can actually run: search products, get cart, add to cart, remove from cart, and uncategorized for everything else. Jev can only land on a name that is on the list, so the code can branch on the tool name. Weather goes to uncategorized. Gluten-free goes to search. The yogurts go to add to cart.
The third is a score on a scale the app defined. Zero is not shopping. One is looking. Two is ready to buy. A score of 1.1 sits between levels because it is an average of the levels, weighted by probability. Jev returns those probabilities too.
One request can carry all of the questions at once. John calls the pattern speculative fan-out, from TypeSafe's docs: ask every question that might apply, then let code read only the answers that do. A support ticket can be scored for category, severity, refund intent, and frustration in one round trip. If the category is a bug, code reads severity and ignores the refund score. The other answers still came back. They are cheap to ignore.
Jev doesn't write an answer. Jev picks one of the options the app gave it and returns a score for every option.
#Confidence is not the top score
The number you gate on is not the raw top score. With three actions, an even split is about 33 percent each. John walks TypeSafe's confidence-routing example: confidence is how far the top score sits above that even split.
Below a floor, a person handles the message. Above the floor, each action has its own bar, set by the cost of a mistake. Checking a balance does no harm, so it can run immediately. Approving a transfer moves money, so it needs a higher bar, and a miss asks the customer to confirm first.
The same message does not always return the same confidence. John sent one message four more times and watched the number move a little. Keep the bar clear of where everyday messages actually land. And pin the model version you tested. latest moves when a new release ships.
#The menu is the product
The demos differ in where the menu comes from.
In the to-do app, the menu is a fixed list of actions: create, complete, set due, assign. A spoken sentence is split by ordinary code, then each piece is scored against that list. "Add" means create. "Move" means set due. The words "the dentist" are not pasted into the database. Jev picks the real task from the tasks you already have. Twelve questions against one sentence came back in well under a second.
In the flight demo, nobody wrote a list. A Chrome extension John called Reflex numbers the buttons and fields on the page. Jev picks a number: type here, click there. Plain JavaScript reads the page. A chat model only gets the step that needs new words typed. The page changes, so the next menu is new. Fifteen decisions, about eleven seconds, less than half a cent.
In the Wikipedia walk, each article is a menu of the first links the walk has not visited. The question stays the same on every page: which link leads closer to philosophy? The menu is new each time. John ran the same walk on TypeSafe's docs. Most questions reached the page that answered them. One ran out of hops, so he added a short block of links on the homepage. The rewalk landed in one hop. At that price you can walk the docs again every time they change.
#Narrow in code, then pick
A pick-one question takes at most 255 options. Even below that cap, a menu of every task in the app gets long, expensive, and muddy, because tasks start to sound like each other.
John's rule of thumb:
- If you can list the options and the list stays short, that is Jev.
- If the list grows with your data, narrow it in code first, then let Jev pick from the few that are left.
- Exact values, like dates, go to a parser. Jev can pull the month and the day out of "Friday." The calendar math stays in code.
- Anything that has to be right every time stays in plain code.
- If you need new words written, that is a chat model.
He hit this himself. The first to-do app handed Jev every item in the list. The picks got weird. A search that returns a few candidates, then a pick among those, behaved. The model was the same. The menu was the bug.
#Start with inference, pave what repeats
John's last move is the one to steal.
Put a Jev call where your code makes the decision. Log every pick. He sent a few hundred real support messages through a four-team router. Sixteen of them mentioned a refund, and all sixteen went to billing. That is a worn path. Pave it with one ordinary line, a text pattern for "refund" or "charge twice," and those messages never reach the model again. An odd message still falls through.
The same idea is a cache, except the thing you cache is an English sentence. If the same sentence on the same page should always do the same thing, store the answer. Jev is for the sentence you have not seen.
#What he is teaching next
John was explicit about the limit. Jev is fast and cheap. It does not count reliably, should not do arithmetic, and reads dates as text. It will not write the email or study the design. System one, the fast automatic kind of thinking, is the job. System two stays with a chat model or a person.
His week at MEGA is the factory around that: project setup where agents can ship and fix, feedback from the team and from users, video of user research cut down to the clips that matter, and harnesses you can change. The line he borrowed: there are many agent harnesses, but this one is yours. You are not training your own coding model. You are deciding what that model is allowed to call.
