Stop paying a poet to fill a dropdown

Most of the AI in business software is a switch statement with a fuzzy input. We keep paying chat models to write a paragraph about which case applies.

Look at what the feature actually is. A message comes in. Your code needs to know whether it goes to billing, support, sales or the bin. Four cases. The hard part is the input: a human wrote it at 11 pm, half in anger, with a screenshot attached. The output is a dropdown.

So what do we do? We hire a poet.

---

You've seen this code. I've written it. The prompt ends with "Respond with ONLY the category name, nothing else." The model replies with something like "Sure! This looks like a billing issue, since the customer mentions a double charge."

So you write a regex, a little pattern matcher, to fish the word out. Then a lowercase pass, because sometimes it says "Billing." with a period. Then a fallback for the day the regex catches nothing. Then a retry for the day the fallback guesses wrong.

That's the prose round trip. You start with an enum, a fixed list of allowed answers, turn it into English, pay a model by the token to write English back, and parse that English to recover the enum you started with.

The fair objection: nobody does that anymore, just use structured outputs. True, and good. In August 2024 OpenAI shipped Structured Outputs, which constrain the model so its reply matches a JSON schema you supply, and reported that its newest model matched the schemas 100% of the time in its own evals. The regex is dead. Nobody misses it.

But look at what that fixed. It fixed the shape. You now get {"category": "billing"} instead of a paragraph.

It didn't give you a number you can act on. The tempting move is to add a confidence field to the schema and let the model fill it in. That's a model grading its own homework, a check that takes its answer key from the thing it checks. It's one more thing the poet writes, formatted as a float.

And "how sure?" is the question that decides what happens next. Act on the answer, or put it in front of a human. Without a confidence you can trust, every answer arrives in the same flat voice and you can't tell which ones to check.

A model that can't write a sentence

On September 15, 2026, TypeSafe launched Jev and called it a "System One" model, after Daniel Kahneman's fast, intuitive mode of thinking. The pitch is in what it gave up. Jev doesn't generate text at all. You hand it an input, say a support ticket, plus a set of typed questions, and it returns typed answers with probabilities attached.

The docs list three kinds of question about text. A Choice picks one option from a list you wrote and returns the pick, a probability for every option, and a confidence. A Score places the input on an ordered scale you define. A Noul, their word for a yes-or-no question, returns the probability of yes. Then the docs tell you to combine the answers in your own code, with your own weights. TypeSafe prices input at $0.042 per million tokens. Output is free, because there's no output to speak of.

I like this more than I expected to, and mostly because of the homework.

TypeSafe's comparison table puts it in one line: "Possible outputs and structure are defined in advance." You can't call Jev until you've written the enum. A chat model will take a question nobody finished thinking about and answer it anyway, at length, in a confident tone. Jev makes you finish the thinking before you ask.

The number is the product

The best review of Jev I've read comes from outside TypeSafe: a pre-registered test by PrimeLine. Pre-registered means the author wrote down the pass and fail bars before looking at a single answer, which is more discipline than most vendor benchmarks get. The conclusion fits in two sentences: "you can get the same answer out of any LLM. You cannot get the same number."

On raw labels Jev didn't win cleanly. Sorting 800 commit messages by type, Anthropic's Claude Opus 5 was statistically indistinguishable from it. Filing 450 notes into categories, Claude Haiku 4.5 beat it. What held up was the confidence. On both jobs, including the one it lost, Jev's probabilities ranked its own mistakes usefully: the answers it was least sure about were the ones most likely to be wrong.

The same test found the limits, too. Calibration is whether "90% sure" turns out right about nine times in ten, and the expected calibration error is the average miss. It was far better on yes-or-no questions (0.012) than on pick-one questions (0.086), though the author warns the question types weren't measured on identical samples. And the confidence field on a Choice turned out to be a fixed formula over the top option's probability, rescaled by the number of options. It adds nothing the probabilities don't already say. To see whether the runner-up is close behind, read the distribution.

Valid is not true

Then there's the marketing. TypeSafe's launch post says Jev "can't hallucinate" and plots it at 0% hallucination. To its credit, the same post explains the number:

Our number is not empirical. Schema matching is guaranteed, thus we can confidently add 0% into the plots.

By that standard my JSON parser has never hallucinated either.

A type-valid answer can still be the wrong answer. choice: billing on a ticket about a login bug is perfectly valid and perfectly wrong. PrimeLine makes the same point, and found that an instruction planted in the text pushed Jev to the wrong answer in 22.5% of its injection tests. A schema doesn't stop the input from arguing with your question. It guarantees the label is one of yours. Only your own labeled data tells you whether it's the right one.

The label was the bug

Here's a hypothetical, but you'll recognise it.

Say you run support for a small software company. Tickets land in one inbox and you want each routed to one of four teams: billing, technical, account, sales. You write the obvious question, "which team owns this ticket?", as one Choice with four options.

Then a ticket arrives: "You charged me twice and now my account is locked." That's billing and account. The model splits its probability between the two, picks one, and half the problem lands on the wrong desk. Run it over a month of mail and you find a big share of tickets need two teams. Your dashboard shows a model with mediocre accuracy.

The model didn't fail. It answered the question you wrote.

The label was the bug.

A Choice assumes exactly one right answer. Its probabilities add up to one, so "both teams" can only show up as a split that looks like doubt. So ask a different question. Four yes-or-no questions: does billing need to see this? Does technical? Each gets its own probability, two can both be high, and that's the question type where PrimeLine found calibration strongest.

Because every answer is now a probability of yes, you can draw lines. Above a high bar, route it automatically. Below a low bar, skip that team. Everything in the murky middle goes to a review pile that a person works through every morning. Pick the bars from a sample you labeled by hand; PrimeLine is clear that thresholds don't transfer between question types.

That review pile is the system knowing what it doesn't know. It's the thing we were trying to buy all along.

The boring tool was already here

None of this is new, and that makes our habit look worse.

Zero-shot classification, sorting text into labels a model was never trained to sort, has sat on Hugging Face for years. facebook/bart-large-mnli uses a trick from a 2019 paper: turn each label into a sentence, "This text is about billing," and score whether the text supports it. Pass multi_label=True and it scores each label on its own. That's the four yes-or-no questions from the support inbox, available since before anyone said "prompt engineering" in a standup.

And if your labels hold still, you may not need a hosted model at all. Banking77 is a public dataset of 13,083 online-banking customer messages sorted into 77 kinds of request. A fine-tuned DistilBERT, 67 million parameters, reports 92.5% accuracy on its test set. Setups differ, so don't line that number up against other benchmarks. The point is that a small model trained on your own examples runs where you put it and costs what your hardware costs.

So here's the order I'd use. Rules first, wherever a rule works. A small trained classifier when the labels are stable and you have examples. A decision model like Jev when the categories are new and you have no labels yet. A chat model when you actually need words.

Whichever you pick, own the exit. Keep the questions and the thresholds in your code, not in a vendor's dashboard. Jev is proprietary and hosted. Kev, one letter off, is an open family of decision models under Apache 2.0, and its README says TypeSafe's Python SDK works against a Kev server unchanged. Once the interface is a list of typed questions, the model behind it is a line of config.

The poet was never the problem. We hired one because we hadn't decided what we were asking.

Write the enum first.