Jev Inside ngram: Faster Decisions in an AI Video Agent

AI agents spend a surprising amount of time asking large language models small questions.

Is this request allowed? Which voice ID best matches the name the user typed? Which music track fits this script? Which tags belong on this gallery item?

These are useful decisions, but they do not need an essay. They need one value from a known set, a score, or a probability. Sending every one of them through a general-purpose generative model adds latency and cost to the path between a user's request and the result they want.

At ngram, we replaced eight of those calls with the Jev model, a System One evaluation model from TypeSafe AI. The change gave us a narrower interface for bounded decisions while preserving generative models for the parts of the product that benefit from open-ended reasoning and creation.

The short version

• Jev accepts text or structured text state and answers typed Choice, Score, and boolean questions.
• It evaluates multiple questions independently against the same state in one request.
• We use it only where the valid outputs are known before the call.
• We moved eight text-only decisions: triage, three guardrails, two catalog selectors, and two gallery-tagging paths.
• We kept generative calls where the model must write, explain, plan, or reason across an open-ended output space.
• We deployed behind legacy, shadow, and jev modes, with schema validation, catalog checks, observability, and fallback to the previous model.
• Jev does not receive images or videos in any of these paths. It receives policy text, user text, scripts, product context, and catalog metadata.

The wrong tool for a small decision

A modern AI agent contains several kinds of model work.

Some calls create things: a script, a scene plan, an answer, an image prompt, or code. Some calls synthesize messy evidence and explain why it matters. Others choose among values the application already understands.

The third group has a different contract. Consider a music selector. The application already has a catalog of valid track IDs. It needs to select one of those IDs from the script, storyboard narration, research, and brand style. Free-form prose is incidental. The result the application can act on is a bounded choice.

A general-purpose LLM can produce that choice, especially with structured output. But its generation machinery is broader than the task. It still generates a sequence, carries an output-token cost, and can return a structurally valid identifier that the application cannot use.

Jev is designed around the smaller contract: give it state, define questions with allowed answers, and receive typed decisions.

What the Jev model is

TypeSafe describes Jev as its first “System One” model. The name refers to fast, bounded judgment rather than deliberate, multi-step reasoning. The public interface has three primitives:

The model receives a shared state as text, JSON, or an array of text. Each question is meant to be atomic. Application code then combines the answers, chooses thresholds, validates identifiers, and performs side effects. This split is central to the design: the model proposes bounded judgments; the application owns policy.

Vercel exposes the same evaluation interface through AI Gateway and AI SDK 7's experimental_evaluate. That gives us the usual Gateway routing, usage metadata, and observability while keeping the questions typed in TypeScript.

How it works under the hood

The high-level design is public. The complete architecture, training corpus, weights, and all training details are not.

According to TypeSafe's launch article, Jev uses a new model architecture with a parallel sampler. It produces structured decisions directly instead of emitting a chain of output tokens one after another. Multiple questions are evaluated independently against the same state, so one question's answer does not become another question's context.

TypeSafe says it trains System One models using reinforcement learning for calibrated decisions, or RLCD. The objective combines accurate classification with probability values that remain useful across many examples. If decisions assigned roughly 0.8 probability are correct roughly 80 percent of the time on representative data, the probabilities are calibrated. That still does not make an individual 0.8 answer “80 percent correct,” and it does not eliminate domain drift.

This design has three practical consequences.

1. Questions can run in parallel

One request can ask for a policy category, a content type, and whether an escalation is needed. TypeSafe's API evaluates these questions independently against the shared state. Its parallel questions cookbook reports that a 13-question document evaluation ran about 10 times faster and 12.2 times cheaper than 13 sequential calls. Those figures are TypeSafe's benchmark on that workload, not a universal guarantee.

2. More questions do not create a longer answer sequence

A generative model emits more output tokens as the answer grows. Jev's documented parallel sampler is built for a set of bounded outputs. TypeSafe says adding independent questions has little effect on latency, though input size, question complexity, option count, routing, and network conditions still matter.

3. Probabilities are inputs to policy, not policy themselves

A score of 0.72 cannot decide whether to block a user, page an engineer, or spend money. Those actions have different failure costs. TypeSafe's confidence guidance and Vercel's threshold guide both put the threshold in application code and recommend calibrating it with labeled, domain-specific data.

Why this is different from JSON mode

Structured output from a general-purpose LLM solves a valuable problem: it constrains the shape of generated output. A schema can require a trackId string and a reason string.

It does not change the underlying task into a native bounded decision. The model still generates tokens, and a schema-valid trackId can still refer to no track in the catalog. Questions placed in one generated object may also influence one another because they share an autoregressive output sequence.

Jev's interface starts with the decision. Choices are declared up front, questions are evaluated independently, and the response includes the full probability distribution. We still validate the result. Type safety prevents malformed values; it does not prove that the selected value is semantically correct.

Where the Jev model fits in an agent loop

Our agent still uses generative models extensively. The AI video generator still depends on them for scripts, scene plans, and other work where the result is not known before the call. Jev sits around that creative core in places where the next action depends on a compact decision.

User request
│
├── Triage and policy decision ─────────── Jev
│
├── Script, story, scenes, prompts ─────── Generative models
│
├── Voice and music catalog selection ─── Jev
│
├── Image and video generation ────────── Media models
│
└── Gallery classification ────────────── Jev

This placement gives us a simple rule. It also makes the model boundary easier to inspect inside an agentic workspace, where sources, intermediate work, and results need to remain legible:

Use Jev when the input can be represented as text, the valid outputs are known before the request, and the application does not need generated prose or a reasoning trace.

That rule is more useful than “replace cheap-looking LLM calls.” It prevents us from moving a task merely because it returns JSON.

The eight calls we replaced

We audited the LLM calls in the codebase and selected eight paths whose useful outputs are bounded decisions.

The Image Lab and Video Lab entries share a guardrail evaluator, but they are distinct product call sites. The online gallery classifier and offline backfill share the same decision contract, but they run in different operational paths.

A closer look at triage

Triage previously asked a general-purpose model for a structured object: whether the message was allowed, its policy category, an optional rejection reason, and whether it was a system question.

The Jev path asks two bounded questions against the same state:

1. Which content-policy category applies?
2. What is the probability that this is a product or system question?

Application code derives allowed from the category and uses a 0.5 threshold for the boolean answer. If the message is blocked, the product now returns a standard rejection sentence. If it is an allowed system question, a separate generative model can still write the helpful answer. Jev decides whether that second call is appropriate; it does not replace the writing call.

That boundary keeps behavior clear. The classifier classifies. The writer writes.

A closer look at voice selection

Voice selection shows why post-model validation still matters. The selected value feeds ngram's AI voiceover feature, but the selector itself receives only request and catalog text. We send the user's request and a compact view of the available voices. Jev chooses a match type and IDs for the best and alternate candidates.

After the response, our code checks that every chosen ID exists in the catalog. An ambiguous result must contain two distinct voices. A none result maps to an empty candidate list. The output is then parsed through the same result schema used by the previous path.

The model cannot invent a voice and silently pass it to the rest of the product.

A closer look at gallery tags

The gallery classifier asks five independent choice questions in one evaluation: primary, secondary, and tertiary use case; primary and secondary role. Optional positions include a sentinel value for “no additional tag.”

Code removes duplicates, rejects values outside the taxonomy, and requires at least one use case and one role. The same evaluator powers new items and the historical backfill, reducing the chance that the online and maintenance paths drift apart.

Text only means text only

Jev currently accepts text and structured text state. It does not accept images, audio, or video as input.

None of our eight replacements passed media to the previous LLM call. The Image Lab and Video Lab guardrails may sound multimodal because of the product surface where they run, but their evaluator receives only the policy and the formatted text request. Voice selection receives catalog metadata, not audio. Music selection receives track metadata and video context, not an audio waveform. Gallery tagging receives title, description, script, and prompt, not the rendered video.

If a future decision needs to inspect pixels, motion, speech, or music, it will need a multimodal model or a separate media-understanding step. We would not route the raw media to Jev.

Rolling out without a behavior cliff

Replacing a model behind an agent can change more than types. Different decisions can route the whole workflow down a different path. We added one migration wrapper with three modes for each capability:

Each capability has its own environment switch. This lets us compare or roll back triage independently from music selection or gallery tagging.

The shadow comparison is deliberately based on the product contract. Triage compares allowed, category, and system-question status. Voice selection compares match type and the selected IDs. Music compares the selected track ID. Gallery tagging compares the ordered use-case and role arrays. We are measuring whether the two implementations make the same application decision, not whether their prose resembles each other.

In jev mode, malformed or unusable results fail validation and use the legacy path. Abort signals are different: cancellation is rethrown rather than converted into a fallback call, so a stopped user request stays stopped.

Every evaluation goes through our common AI context. We record the model, provider, span, latency, input and output token usage, Gateway cost metadata, and trace identifiers in the same observability system as other model calls. This gives us the data needed to compare real production workloads instead of extrapolating from a demo.

What we deliberately did not replace

We left two broad groups on general-purpose models.

Split tasks that still need a generative call

Some calls both classify and generate useful content. If Jev performs the classification but the workflow immediately makes another model call to write the result, the extra boundary may add complexity without removing the dominant latency or cost.

Triage is a narrow exception: many messages need only the initial decision, and only allowed system questions continue to a writer. For tasks where generation always follows, we kept the combined path.

Evaluations that depend on open-ended reasoning

Some “evaluation” calls compare alternatives, explain tradeoffs, diagnose an issue, or synthesize evidence. A score alone would remove information the rest of the product uses. Jev does not generate prose or a chain of reasoning, so we kept those calls on models suited to that work.

We also left all creative generation in place. Scripts, story structures, scene directions, prompts, and user-facing answers remain generative tasks.

Speed and cost: claims versus measurements

TypeSafe reports typical Jev latency of 70-500 milliseconds and direct pricing of $0.042 per million input tokens with no output-token charge. In its published workflow benchmark, it reports a high-end result of 193.6× faster and 444.6× cheaper than the compared generative workflows. Vercel later summarized these as “up to 194× faster and 445× cheaper” in its AI Gateway launch post.

Those are vendor measurements. They are useful for forming a hypothesis, not for predicting an exact ngram result. End-to-end performance also includes network latency, Gateway routing, input size, retries, validation, fallbacks, and the percentage of calls that still lead to generation.

Our rollout is built to answer the production question directly:

• Compare legacy and Jev duration on the same shadow request.
• Measure decision equivalence by capability.
• Track fallback and validation-failure rates.
• Compare provider cost metadata on representative traffic.
• Watch the full user-visible workflow duration rather than only the isolated model span.

The goal is lower latency and cost without changing the decisions users depend on. The instrumentation makes that a testable claim.

Limits and open questions

Jev is a specialized model, and that specialization creates boundaries.

It does not explain itself. A probability distribution can support a policy, but it cannot replace a generated rationale when the rationale is part of the product.

Typed does not mean correct. The model can select an allowed option for the wrong semantic reason. Domain test sets, shadow traffic, and outcome monitoring still matter.

Calibration must be checked on our distribution. Published calibration behavior does not guarantee that a threshold transfers to our policies, catalogs, languages, or user mix.

Large choice sets require care. TypeSafe documents native Choice sets up to 255 options. Above that, its SDK uses a two-stage approach that scores options independently before selecting among finalists, which can add latency and alter the decision shape.

The implementation is not fully open. TypeSafe has described the sampler, decision primitives, and RLCD objective at a high level, but has not published the full architecture or weights. We should be precise about what is documented and avoid filling gaps with speculation.

Media still needs media models. Text metadata can be enough for our current selectors and guardrails. It is not a substitute for inspecting the actual media when a product requirement depends on it.

A checklist for bounded model decisions

Before moving an LLM call to Jev, we ask:

1. Can every valid output be declared before the request?
2. Is the useful result a choice, score, or probability?
3. Can the complete input be represented as text or structured text?
4. Does the product work without a generated explanation?
5. Are the questions atomic and independent?
6. Can application code validate the answer and own the side effect?
7. Do we have representative examples for equivalence and threshold testing?
8. Can we shadow the new path and roll it back independently?

If several answers are “no,” a general-purpose or multimodal model is probably the better fit.

Frequently asked questions

Is Jev just a smaller LLM?

TypeSafe presents it as a separate System One architecture for parallel, calibrated decisions rather than a smaller autoregressive chat model. The public material does not disclose enough implementation detail to independently characterize every internal difference.

Does Jev replace reasoning models?

No. It handles bounded decisions. We continue to use reasoning and generative models when the output space is open, when the model must synthesize evidence, or when users need written content.

Can Jev inspect an image or a video?

No. Its current documented inputs are text and structured text. Our Image Lab and Video Lab guardrails pass text requests, not media.

Why keep a legacy fallback if Jev is faster?

Availability and contract preservation matter more than a single fast span. A fallback lets the workflow continue if evaluation fails, the Gateway has an incident, or validation catches an unusable result. We separately track fallback frequency because frequent fallback would erase the expected gains.

Why use standard rejection messages?

The old guardrails could generate a custom rejection reason. We accepted a standard message for the four safety-related paths because the actionable product behavior is the same: the request is blocked under a known policy category. Standard copy is also easier to review and keep consistent.

What does “confidence” mean?

For Choice and Score, TypeSafe derives confidence from the shape of the probability distribution. It is distinct from the probability assigned to a boolean answer. Neither value chooses the application's threshold for us.

A smaller model boundary makes a faster agent

AI agents will continue to need powerful generative models. The opportunity is to stop making those models do every kind of work.

Jev gives us a compact boundary for a recurring class of tasks: shared context in, typed decisions out. In ngram, that boundary now covers eight calls around the creative pipeline. We validate the results, keep application policy in code, observe the rollout, and retain generative models where their broader capabilities matter.

The result is not a less capable agent. It is an agent that spends its expensive reasoning and generation budget on the parts users can see, and resolves the small decisions on the way there with a tool designed for them.

Try ngram to see that creative pipeline in action.

Sources and further reading

• Vercel AI Gateway: Evaluation models
• Vercel: What is Jev?
• Vercel: Jev model page
• Vercel: Jev on AI Gateway
• TypeSafe AI: Introducing System One models and Jev
• TypeSafe AI documentation: System One
• TypeSafe AI documentation: Primitives
• TypeSafe AI documentation: Confidence
• TypeSafe AI documentation: Machine learning primer
• TypeSafe AI cookbook: Parallel questions

The four diagrams in this article are original ngram illustrations based on the cited public documentation and the implementation described in this article. Product behavior and source links were last verified on September 22, 2026.