---
title: "TypeSafe instead of Claude Code: judgment, not generation"
description: "The same 80 messages triaged by an LLM agent loop and by a System One model returning typed answers. Compare wall time, cost, and malformed outputs live."
published: 2026-09-17
updated: 2026-09-17
canonical: https://protrailblazer.com/posts/typesafe-instead-of-claude-code/
tags: [typesafe, claude-code, demo, agents, structured-output, system-one, batch, developer-tools]
---

# TypeSafe instead of Claude Code: judgment, not generation

**Answer:** Some jobs you hand to a coding agent are not coding jobs. Labeling 80 messages, ranking candidates, flagging what needs a reply: each one is a decision, not a program. A System One model like TypeSafe's Jev answers those decisions directly as typed values with probabilities, so there is no text to generate and nothing to parse. The demo below races that against a generate-then-parse agent loop on the same batch and shows where the time, the cost, and the malformed outputs come from.

The interactive below drops the same 80 inbound messages into two lanes. The agent lane prompts a language model, waits for it to write out an answer token by token, parses the result, and occasionally has to retry a malformed one. The System One lane sends each message with its questions and gets typed answers back. Both lanes run three workers. Watch the Scoreboard, then add questions per item and see which lane notices.

*Interactive demo: [open the typesafe batch judgment demo](https://protrailblazer.com/demos/typesafe-batch-judgment/)*

## What you're seeing

The top-left window is the agent loop. Each of its three rows is a worker holding one message, with the phase it is in (prompt, gen with a token count, parse) and a progress bar. When a worker hits a malformed output, the phase label turns red and reads "malformed, retry", and the worker goes back to generating. Below it, the System One window has the same three workers, but each one only has two phases: request and typed answer. On an ordinary run, that lane finishes the whole batch while the agent lane is still in its first quarter, then its title bar turns red and its rows read "done".

The Scoreboard compares the two lanes on a shared scale per row: wall time so far, cost so far, and malformed outputs. The System One malformed count is zero by construction, because the answer arrives already in the schema you asked for. The Console on the right shows the most recent typed answer: the message as state, the questions that went with it, and each answer with its probability bar. The Tally keeps the running totals for both lanes side by side.

The questions are the interesting knob. At one question per item, each message only gets a label (lead, support, billing, spam, other). At six, it also gets urgency, sentiment, whether it expects a reply, whether it mentions money, and a priority score. The agent lane has to write all of that out. The System One lane answers it in the same call.

A few things to try:

- **Slide questions per item from 2 to 6.** The agent lane's per-item time and cost roughly triple, because every extra field is more output to generate. The System One lane's request time moves by hundredths of a second, and the batch is done before you can read the Tally.
- **Push the malformed rate to 15 percent.** Red retry labels appear in the agent lane and the malformed count climbs. Each retry is a full regenerate, so it costs time and output tokens twice.
- **Set the malformed rate to zero.** Even a perfect parser does not close the gap. The gap is generation, not parsing.
- **Read the Console while the System One lane runs.** The label answer carries the whole probability distribution and a confidence value. A message that is half lead and half billing shows up as a spread, not a guess.
- **Press Replay when both lanes finish.** The batch restarts and the Scoreboard resets, so you can compare a second run with a different setting.

## How the two lanes are modeled

Nothing in the demo calls a model. Both lanes are timing and cost models with their constants printed at the bottom of the Scoreboard, and the point is the shape of the difference, not the exact figures.

- **Agent lane.** 0.7 seconds to the first token, then 55 tokens per second. Output is modeled as 24 tokens plus 26 per question (a JSON object with one field per question). Input is 320 tokens plus 40 per question. Priced at $3 per million input tokens and $15 per million output, which is a typical frontier-model price sheet rather than a specific vendor. A malformed output (4 percent by default) pays for a full second generation.
- **System One lane.** 0.16 seconds per request plus 0.02 per extra question, inside the 0.16 to 0.31 second band TypeSafe reports for its cookbooks. Input is about 90 tokens plus 52 per question, since the criteria travel with the request. Priced at Jev 1.13's published $0.042 per million input tokens, with output tokens free.
- **Both lanes.** Three concurrent workers, 80 messages, and a simulated clock that runs 2.5 times faster than real time so the agent lane finishes in about twenty seconds instead of a minute.

Three workers keep the System One lane well inside the published rate limit of 1,200 requests per minute, which the docs note is adjusting dynamically under demand. At the defaults the batch costs a few cents in the agent lane and a few hundredths of a cent in the System One lane. The ratio is large, but the more useful number is per item: about a hundredth of a cent for typed answers to two questions, which is the scale at which you stop thinking about whether to ask.

## Why adding a question barely costs anything

A language model produces an answer by writing it. Every field you ask for is more tokens to generate, and generation is the slow, expensive part of inference. A System One model does not write. It reads the state once and returns a probability for each question, and the questions are evaluated in parallel inside one request. That is why the docs say adding questions barely changes the response time, and why the demo's request time moves by hundredths of a second between one question and six.

The malformed column is the other half of the story. When you ask a language model for JSON you get text that is usually JSON, and the usual fixes are a retry loop, a schema-repair pass, or [constrained decoding](https://protrailblazer.com/posts/constrained-decoding-json/) at the sampler. A typed answer skips the problem: the API defines the shape, so there is nothing to parse. What it does not skip is being wrong. TypeSafe's own docs put it plainly: typed output guarantees the interface, not the truth. The zero in that column means zero parse failures, not zero mistakes, and the probabilities exist so your code can route the uncertain ones instead of trusting them.

## When to reach for this instead of an agent

The word "instead" is about the job, not the tool. Claude Code is the right tool when the output is code or prose, or when the work needs a sequence of tool calls that only makes sense after reading the previous result. A System One model is the right tool when the output is a decision over state you already have: a label, a score, a yes or no, a pick from a list. Most triage, routing, ranking, flagging, and extraction work is that second kind, and a lot of it ends up in an agent session only because the agent was already open.

Reach for typed judgments when:

- You have the state in hand and the answer is one of a defined set. Sorting messages, ranking candidates, deciding which handler runs, checking whether a field is present.
- You would otherwise prompt-and-parse. If your code already has a regex around a model's output, that is the signal.
- You need probabilities to act on. A threshold on a probability is a policy; a paragraph is not.
- Volume matters. At a hundredth of a cent per item you can judge everything, including the things you used to sample.

TypeSafe's suggested way to find these spots is to hand the search to the agent: ask Claude Code, with the TypeSafe skill installed, to explore the codebase for parsing and branching logic that a small judgment could replace, then run a few experiments against the API before proposing changes. The skill install is two commands (see the [preflight post](https://protrailblazer.com/posts/typesafe-before-claude-code/)). Human review then focuses on the questions and the thresholds, which the docs describe as the part agents are least good at writing.

The [first post in this series](https://protrailblazer.com/posts/typesafe-before-claude-code/) puts a typed gate in front of Claude Code so only ready, low-risk work reaches it. The [third](https://protrailblazer.com/posts/typesafe-after-claude-code/) puts one after it to verify what came out. For how an LLM-as-judge differs from a typed check, see [LLM-as-judge explained](https://protrailblazer.com/posts/llm-as-judge-explained/).

> **info:** Both lanes are timing and cost models. The System One numbers come from the TypeSafe docs (latency band, Jev 1.13 pricing, rate limit); the agent-lane numbers are illustrative and no vendor is named. The typed answers in the Console are drawn from a ground truth I wrote for each of the 24 messages in the batch, so they show the shape of the answers, not a measured accuracy.

## Frequently asked questions

### Is Jev a language model?

TypeSafe calls it a System One model: it reads natural language state and returns typed answers with probabilities rather than generating text or reasoning. The docs describe it as trained for calibrated decisions, and the whole point of the design is that there is no output to parse.

### How accurate are typed answers?

That depends on your questions and your domain, and the docs are explicit that you should validate on your own data. One published example: adding two TypeSafe calls to pick a skill for an agent turn cut wrong skill loads from 16.8 percent to 7.3 percent across 488 requests, at 0.25 to 0.43 seconds added per turn. Treat numbers like that as a shape to check, not a promise.

### Why is the agent lane's malformed count nonzero even at a low rate?

Because over 80 items a 4 percent rate produces about three retries, and each retry regenerates the whole answer. The slider lets you set it to zero to see that parsing was never the main cost.

### Can I run the batch from Claude Code?

Yes. With the TypeSafe skill installed and TYPESAFE_API_KEY set, Claude Code can write the script that sends each item with its questions. The docs recommend keeping the questions and thresholds in one file so they are easy to review and edit collaboratively.

### Where do I get an API key?

Sign in at console.typesafe.ai and create a key under settings. Export it as TYPESAFE_API_KEY. The Python SDK installs with pip install typesafe-sdk, and the raw HTTP call is a single POST to /v1/systemone.

## Key takeaways

- Generation is the cost, not parsing. Set the malformed rate to zero and the gap barely moves, because a language model still has to write the answer out.
- Questions are nearly free on the System One side. They run in parallel inside one request, so the marginal question costs hundredths of a second and a few dozen input tokens.
- Zero malformed means zero parse failures, not zero mistakes. The probabilities are there so code can route the uncertain answers instead of trusting all of them.
- "Instead of Claude Code" is a statement about the job. If the output is a decision over state you already have, an agent session is the wrong shape for it.
- Find the spots by asking the agent to find them. The exploration prompt in the TypeSafe skill is designed for exactly that search.