The Typesafe JEV Model: Is It a New Post-LLM AI?

Every number in this article comes from a file on disk, not from documentation. We built a compact state out of real captured sessions from a live tenant, POSTed it to the TypeSafe API, and wrote each run down with its model version, token count, latency, cost and raw answers. The probes, the ledger and the raw probabilities are reproduced at the end.

The question is badly framed

“Post-LLM” and “useless toy” share an assumption: that a new model either eats the old one or is worthless. That assumption is how the industry has read every model release for three years, and it is now actively misleading, because the interesting new systems are not competing with language models at all. They are occupying a layer language models were never good at.

Consider what we currently do with a chat model when we need a decision. We ask it to classify a support ticket, route an alert, tag an error, or label a session. We get back a paragraph, we bolt a JSON grammar constraint onto it, we parse it, and we hope the parse holds. Underneath a hundred lines of prompt engineering and schema validation, we are using a system that predicts the next token in order to answer a yes-or-no question.

That is a category error with a bill attached. A model that writes prose must emit the prose before you can read the decision buried inside it — you pay for the tokens, you wait for them all, and you inherit a text surface that can be manipulated by the text it is classifying. JEV is a different answer to the same question: a model whose entire output is the decision.

What it actually is

Mechanically it is almost nothing. You send one JSON state and a set of typed questions to POST /v1/systemone. You get back typed answers: no prose, no reasoning trace, no tool calls, no chance to negotiate. There are three question types, and the interesting thing about them is how literal the mapping to code is.

Note what is absent. There is no “explain your answer”. There is no temperature, because there is no sampling to control. There is no system prompt to jailbreak, because the output has no text surface: the only channel from your state into the world is a number in a field. For anyone who has watched a classifier get talked out of its own instructions by the document it was classifying, that is a security property, not a limitation.

noul    →  { "noul": 0.98 }
choice  →  { "choice": "backend_api", "confidence": 0.89,
             "probabilities": { "backend_api": 0.91, "site_frontend": 0.09 } }
score   →  { "score": 0.92, "confidence": 0.40,
             "probabilities": { "0": 0.34, "1": 0.43, "2": 0.21, "3": 0.02 } }

That last line is the part most people miss. choice and score return a probability distribution and a confidence that is separate from the winning probability. In the run above, the model picked backend_api at 0.91 and reported confidence 0.89 — agreement. In another run it picked third_party_marketing with a winning probability of only 0.48 and reported confidence 0.24: it had essentially no idea, and it said so.

That two-axis output is what makes confidence-gated routing possible rather than aspirational: high probability plus high confidence acts automatically, low confidence goes to a human queue, and you get to draw those two lines yourself. Compare that with the single boolean a keyword rule or a temperature-zero LLM call gives you, which has no way to represent “I am not sure” at all.

What we measured

Specifications are cheap, so we built a state document out of real captured sessions and ran three probes. The state builder is the discipline that makes this work: it reduces a session to a bounded aggregate — event-count histogram, console messages deduplicated by prefix and clipped, only failed or slow network requests, the first few form fields — landing at roughly 4.7 KB, about 3,300 input tokens. Never the raw event stream.

The claims that held up

The PII contrast: the single best result

Two real sessions from the same site, same state builder, same questions. One contained an IDENTIFY event carrying a name and an email; the other did not. Asked whether the captured state contained personal data that must be masked or purged before storage, the first came back at 0.98 and the second at 0.19.

That gap is the whole argument for a decision model in one number. A keyword rule cannot do it, because the failure mode is semantic — a field named dob, iban or passport, or a free-text comments box, in any of the languages a platform serves, and no substring list converges. An LLM can do it, but you would be paying for prose you throw away, in a path that runs on every session. And critically: the discriminating model was reading the state, not paraphrasing the prompt. A model pattern-matching the instruction would have returned the same number for both sessions.

Where it is a toy

This is the part a vendor post would omit, and it is the part that decides whether you can ship the thing. Four failures, in descending order of how much they should change your architecture.

The operational traps nobody mentions

This is where the difference between a demo and a shipped feature lives, and we found these by reading our own production services rather than by reading any model documentation. If you are integrating a decision model into a pipeline that already processes data, the model is the easy part.

And one non-negotiable piece of instrumentation: store the raw probability, not just the boolean you derived from it. If all you keep is “fired” and “did not fire”, no threshold you chose can ever be evaluated afterwards, because you threw away the number that produced it. Ledger every judgment with its model version, confidence, cost and latency, and run it read-only for two weeks before it is allowed to change anything.

What the integration looks like

The reason this is worth taking seriously is how small the code is. The entire decision surface is one call, and everything downstream of it is the ordinary code you already have. Here is our error-digest triage, which today ranks aggregated errors by occurrenceCount alone — so a customer’s real defect competes for attention with an advertising pixel complaining about an invalid event name.

// One POST, one call. The state is bounded; the questions are typed.
const res = await jev.ask(state, {
  actionable: {
    type: 'noul',
    instructions:
      'Could an engineer on the site\'s own team change code, configuration ' +
      'or content to fix this error? Third-party tags, marketing pixels, ' +
      'browser extensions and recorder-internal messages are not actionable.',
  },
  owner: {
    type: 'choice',
    instructions: 'Which owner should be given this error message?',
    criteria: {
      site_frontend:         'The site\'s own application code or markup',
      backend_api:           'A server-side endpoint erroring or too slow',
      third_party_marketing: 'Advertising, analytics or tag-manager scripts',
      recorder_sdk:          'The session-recording SDK itself',
      not_a_defect:          'Informational, not a defect',
    },
  },
  severity: {
    type: 'score',
    instructions: 'How much does this degrade a user\'s ability to finish?',
    criteria: [
      'Cosmetic or invisible to users',
      'Annoying, but the flow completes',
      'Blocks a step of the flow',
      'Blocks the whole task',
    ],
  },
});

// Everything below is ordinary code. No model in the loop.
const j = res.answers;
const show = j.actionable.noul >= 0.7 && j.owner.confidence >= 0.5;
const rank = Math.round(j.severity.score * 40 + Math.log1p(occurrences) * 10);

if (j.actionable.noul >= 0.4 && j.actionable.noul < 0.7) {
  queueForHumanReview(error);   // the "I am unsure" band, made explicit
}

Three questions, one request, one round trip per error group per company per day. No keyword list to maintain, no prompt-injection surface in the output, and the thresholds are three numbers you can argue about and tune — which is the actual deliverable. The 0.4–0.7 band is the design detail worth stealing: it is the region where the model is telling you it does not know, and it maps directly onto a human review queue instead of a coin flip.

The verdict

Neither option in the headline is right, and the useful answer is a third one. JEV is not post-LLM, because it does not replace language models and does not try to. It is not a useless toy, because a measured 763 ms for 19 parallel judgments at $0.000129, with a confidence field that says “I am unsure” on the genuinely ambiguous cases, is a capability no threshold rule and no chat model currently gives you at that price.

What it is, precisely, is a decision layer. And the reason the framing feels new is that we have spent three years using a generative model for decisions, then papering over the mismatch with structured-output schemas, validators and retry logic. JEV removes the mismatch by removing the generation. The output is the decision.

So the practical litmus test is one question, and it is not about the model at all: does the thing you want back map onto a branch in your code? If yes, a decision model is the right shape and an LLM is an expensive detour through prose. If the thing you want back is a sentence a human reads, keep the LLM and stop looking for a replacement. Most teams have far more of the first kind of problem than they think, which is exactly why this stopped looking like a toy to us.

Reproducing this

Nothing above required a special data set. If you have session-replay events, you have everything needed: reduce a session to a bounded aggregate, phrase each decision as a question a person answers at a glance, and write down the raw answer next to the version of the model that produced it.

# 1. A real session -> a bounded state (~4.7 KB, ~3.3 k input tokens)
node build-state.mjs session-user-events.json

# 2. Six judgments in ONE call: PII, owner, struggle, fidelity
node probe.mjs       session-user-events.json

# 3. The contrast that matters: same site, IDENTIFY vs none
node probe.mjs       server-events.json

# 4. Discrimination and confidence on a known-ambiguous session
node contrast.mjs    session-user-events.json

# 5. Nineteen URL-classification questions, one call
node url-discovery.mjs

# 6. The ledger: model, tokens, latency, cost, raw answers
cat live-runs.jsonl

Two notes on reproducing it honestly. First, the contrast probe compares a real captured session against a labelled synthetic counterfactual (a first-party TypeError plus HTTP 500s) — it demonstrates that the model responds to a clear defect, and it is not a real production defect. Second, the state document contains one real session’s IDENTIFY payload, which is why the ledger stores only models, tokens, timings and answers, and never the state itself.

The tools this connects to are worth reading in their own right: the primitives define the three answer shapes, and the confidence-routing pattern is the part that turns a probability into a workflow. If you want the vendor-neutral version of the same argument, the framing of a decision layer rather than another chatbot has been made independently as well.

Related reading

• How to Run an AI-Assisted Pentest on Your Website (Without the AI Attacking for Real)
• Local LLMs for Session Replay Debugging: Keeping User Data Off Someone Else’s Server
• An MCP Server for Session Data: Giving AI Agents Access to Replay
• Error Tracking with Session Replay: Debugging Production for Real