Opens in a new tab
Explainer

Jev can pick a support queue. The ticket still has to get there.

In brief

A typed answer saves an application from parsing a paragraph. Follow a hypothetical support ticket to see what Jev chooses, what its probabilities mean, and who still has to deliver the work.

8 min read

Is this the right starting point?
Before you read
Basic familiarity with APIs, structured data and a support-routing workflow.
What you will take away
Design and evaluate a Jev router that distinguishes schema-valid judgments from tickets reaching a usable queue.
Sources
A saffron square tile leans against a closed plum ceramic box beside an open cream tray.
On this page6 sections

The dashboard opens, but the application can no longer get answers from the service. Someone has just replaced its access token, the credential it sends with requests. Now those requests through the application interface, or API, come back with HTTP 401, a response indicating that valid authentication is missing. That response does not identify the underlying cause. In this hypothetical support ticket, the first job is modest: put the report in front of the right team. Nobody needs a polished paragraph about the situation yet. They need a destination.

That is the kind of small, bounded judgment TypeSafe built Jev to return. Introduced on September 15 as a System One model, it accepts information to assess and questions with specified answer types. The information is called the state. The reply contains structured answers and probability information, so application code can read a queue choice directly. That saves one awkward step, turning generated words into a field. Getting the ticket to a person is still the application's job.

Three questions hiding inside one ticket

The router already has the user's words. It wants a queue name, not another version of the complaint. TypeSafe’s System One documentation describes a deliberately narrow input and output: text, JSON objects and arrays of text can go in; images, audio and video currently cannot. Jev does not write code or explain its reasoning in prose. The application asks for a particular kind of judgment instead.

For a destination, it asks for Choice, which selects among supplied alternatives. This is one of the three documented primitives. For a separate question, such as whether the report concerns authentication, Noul returns a value from zero to one for a yes-or-no statement. To judge how well the user has described a reproducible problem, Score evaluates the report against levels the team defines. Choice and Score also return probabilities across the possible answers and a confidence field. These are different questions, even when they concern the same ticket:

Answer typeWhat the team asksWhat the application receives
ChoiceWhich queue should investigate?A selected option, probabilities across the options, and confidence.
ScoreHow completely does the report describe a reproducible failure?A score against the supplied levels, probabilities across those levels, and confidence.
NoulDoes the report describe an authentication failure?A zero-to-one value for that statement.

Put the queue question and authentication question in one request, and it is tempting to imagine a little conversation between them. There is none. TypeSafe says each question is evaluated independently against the shared state. A high authentication value does not instruct Choice to select the access queue. If the application must establish authentication involvement before choosing among access teams, it has to arrange those dependent steps explicitly.

A useful score starts with an agreement

Imagine two reviewers looking at the same bug report. One calls it good because the symptom is obvious; the other calls it poor because nobody can reproduce it. A number would only conceal their disagreement. In this constructed exercise, the team defines 0 as no reproducible description, 1 as some steps without both expected and observed results, and 2 as all three. Now a disputed score can lead to a specific missing detail rather than a debate about what good means.

That score does not pick the destination. Routing asks which team should take the report, scoring examines its reproduction description, and Noul assesses a particular statement such as authentication involvement. A beautifully described problem can still reach the wrong team. An unmistakable access problem can still omit the steps someone needs to reproduce it.

Give the ticket somewhere to go when it does not fit

Here is our hypothetical report in full: “My access token was rotated this morning. The dashboard still loads, but every API request returns 401.” The application wants a queue and, separately, an assessment of whether authentication is involved. It is not asking permission to reset a credential or change an account. Those actions are outside these routing questions.

The sketch below follows the request shape in TypeSafe’s quick start; its input is synthetic. Notice the last queue, triage. Without it, a sparse report or a problem outside the listed teams would still have to wear one of their labels. Giving uncertainty a destination is more useful than demanding a confident-looking choice.

{
  "model": "jev-latest",
  "state": "My access token was rotated this morning. The dashboard still loads, but every API request returns 401.",
  "questions": {
    "queue": {
      "type": "choice",
      "instructions": "Select the queue that should investigate this report.",
      "criteria": {
        "access": "Authentication, credentials, or authorization failures",
        "application": "Application behavior unrelated to access",
        "network": "Connectivity or transport failures",
        "triage": "Insufficient evidence or no suitable specialist queue"
      }
    },
    "authentication_related": {
      "type": "noul",
      "instructions": "The report describes an authentication failure."
    }
  }
}

The documented POST https://api.typesafe.ai/v1/systemone endpoint returns an answer for each question. This request has not been executed and does not predict Jev's actual choice for the ticket. A trial should retain the versioned model returned by the service alongside the requested alias. jev-latest is handy to request, but its name alone cannot show that two later runs used the same model.

Once a choice arrives, the application looks it up in the current queue directory and sends the original report. It has no reason to ask the model to invent an address. A renamed or unavailable queue goes through the established triage route. The label is a useful intermediate result; delivery is another step.

A synthetic ticket and supplied queue criteria enter Jev. Choice and Noul answers go to application checks. A suitable available queue receives the ticket; uncertainty or an unavailable destination goes to human triage. The diagram distinguishes a typed answer from verified completion.
A judgment leaves Jev; the application then checks the destination and records whether the original ticket reached a usable queue.

A perfectly valid answer can land on the wrong desk

TypeSafe's launch post describes schema matching as a guarantee. Here, the schema is the defined shape and allowed values of the answer, so Choice is constrained to the supplied options rather than an arbitrary string. That removes a genuine integration headache. The post also explicitly separates its zero figure for schema-related hallucination from an empirical claim about overall factual accuracy.

In our example, network fits the allowed answer space perfectly. It could also put an authentication problem on the wrong team's desk. Code can accept the value without complaint while the user waits for a transfer. The shape of the answer and the usefulness of the destination are two different things to check.

The accompanying numbers need that distinction too. TypeSafe’s confidence documentation defines confidence as a statistic derived from the returned distribution of probabilities. When the distribution is concentrated, the model expresses a more definite preference among the supplied options. Confidence is not simply the winning option's probability, nor a universal probability that this particular routing decision is correct.

The practical comparison is across groups of tickets whose correct destinations have been labeled: do stronger stated probabilities go with more correct choices? Keep the sparse reports, unfamiliar product names and conflicting symptoms visible in that comparison. A large pile of familiar requests can make the average look reassuring while those difficult tickets keep being passed around.

Count the transfer, not just the reply

Suppose the synthetic ticket gets that unhelpful network choice. A fast, cheap reply has started another transfer. TypeSafe reports large speed and cost improvements in its evaluations, but also discloses that several workflow comparisons use other models' predictions as reference probabilities. Agreeing with another model is not the same as getting this support team's ticket to the right place. Independently labeled local tickets can answer that second question.

They can also show whether Jev is the simplest useful option. Hugging Face’s zero-shot classification interface accepts text and candidate labels without training on those exact labels. A conventional classifier, a deterministic rule or an LLM with structured output could fit too. Neither the launch comparison nor an unrelated community benchmark decides which one will finish this team's routing work best.

For this particular report, a maintained rule matching token rotation and 401 responses might do the job nicely. The next person may describe the same symptom without mentioning a token, or mix it with a connection problem. That is where a model's ability to interpret varied wording and nuanced queue criteria becomes more interesting. Give the alternatives the same labeled examples, including the disagreements, and see whether that flexibility reduces work rather than merely replacing a rule that already works.

Follow the cost and elapsed time through an accepted routing outcome, including transfers, corrections and human triage. Our agent cost and quality guide uses that same accounting: the price of an attempted decision matters in relation to the work it completes. Saving on calls while creating more transfers can make the whole router more expensive to operate.

Let the existing router keep its job during the comparison

Users need not become part of the experiment. The existing router can keep serving them while Jev proposes destinations for sanitized examples labeled by the team. When a reviewer disagrees with a proposed access choice, put the input and queue definitions beside the model version, answer, probabilities, latency and reviewed destination. The discussion can then get somewhere: was a symptom missing, had the queue's remit changed, or did the model judge the same information differently? Include unclear reports, unfamiliar wording and cases with no suitable specialist queue, not only the easy ones.

Carry the comparison past the prediction. If the proposed access queue has been renamed, the current directory lookup should reveal that and send the original report to established triage, rather than ask the model to guess another address. Record delivery, transfers and triage alongside the answer. Otherwise a router can look dependable by sending almost everything to people who must make the decision again.

Back at the ticket, the dashboard still opens and the API still returns 401. Jev can make the first judgment easier for code to consume; the directory and delivery path can put that judgment to work. Keep it where the whole route takes less effort, with established triage for ambiguity. The useful result is a waiting report reaching someone who can help, not merely a tidier field in the router.

Sources & context

Sources linked in this article. Read alongside the author’s analysis; a citation does not independently verify a publisher’s claims.

Report an error or outdated detail

Keep reading

Where to go next