TypeSafe before Claude Code: a typed preflight gate

The interactive below runs the gate on requests pulled from this site's own backlog: a Safari modal bug, a Polar webhook returning 500, a Slack message that just says "make the site faster". Each one goes out as a single TypeSafe request with four questions. Watch the four answers fill in, watch which policy line fires, then drag the two thresholds and see every judged request re-sort without a new API call.

What you're seeing

The left window is the Preflight. At the top is the request card with its source (an issue, Slack, email, or my own todo list). Below it, the state block is the JSON that actually goes to the model: the request text, where it came from, and the repo. Then four rows, one per question. Each row shows the question id, its type, the criteria in plain words, and, once the answer lands, the typed value with a probability bar. The header of that section reports the modeled latency, the input tokens counted, and the cost of that one call.

The right window is the Router. The policy block at the top is the entire decision logic, four lines of it, and the line that fired for the current request is highlighted in red. Below it, the route counts chart shows how the judged requests have split so far, and the recent list shows the newest ones with their route chip. The Tally across the top keeps the totals: requests judged, how many reached Claude Code, how many agent runs the gate avoided, and what all of that preflight cost.

The four routes map to four things you would do by hand. Dispatch means the request is clear, low-risk, and needs code, so it goes to Claude Code with a scoped brief. Clarify first means the model is not confident the request is specified well enough, so the cheapest move is one question back to whoever asked. Human scopes it means the request is clear but risky (payments, auth, secrets, deleting data), or the model could not tell what kind of work it is, so a person writes the brief before any agent touches it. No code means the answer is a reply or a content task, and spinning up a coding agent for it would be waste.

A few things to try:

• Drag the ready threshold up to 0.90. Requests that were dispatching at 0.70 slide into clarify first. The counts chart re-sorts instantly, the header above it flashes "re-sorted N judged · 0 new calls", and the cost in the Tally does not move.
• Drop the confidence floor to 0.30. Requests where the kind answer was torn between two options (bug or chore, question or unclear) stop bouncing to a human and start dispatching. Decide for yourself whether you would want that.
• Pause and use Step. Each click judges one request and stops, so you can read all four answers and the fired policy line before the next one arrives.
• Watch the risk row on the Polar webhook and the secret rotation. Both are clear and both need code, but the score lands near 2, and the third policy line catches them before they reach an agent.
• Read the state block on "Fix the thing we talked about yesterday". The model has nothing to work with, and the ready probability says so.

How the preflight actually works

Every request is one HTTP call. The body has three parts: the state (the JSON you saw in the demo), the model name, and a map of questions. Here is the shape the demo follows, trimmed to two of the four questions:

{
  "model": "jev-latest",
  "state": {
    "request": "Polar webhook returns 500 on refund events",
    "source": "issue",
    "repo": "protrailblazer.com"
  },
  "questions": {
    "kind": {
      "type": "choice",
      "instructions": "What kind of work is `request` asking for?",
      "criteria": {
        "bug": "Something that used to work is broken",
        "feature": "New behavior the site does not have",
        "chore": "Maintenance with no user-facing change",
        "question": "Wants an explanation, not a change",
        "unclear": "Not enough here to tell"
      }
    },
    "risk": {
      "type": "score",
      "instructions": "How much damage could a wrong change do?",
      "criteria": [
        "Cosmetic only",
        "Logic or build behavior",
        "Auth, payments, secrets, or data"
      ]
    }
  }
}

The answers come back typed. A choice answer names the option with the highest probability and includes the whole distribution plus a confidence value that describes how peaked that distribution is. A noul answer (TypeSafe's name for a yes/no question) is a single probability of yes. A score answer is the probability-weighted position on the levels you defined, so a risk of 1.94 means almost all of the weight sat on "auth, payments, secrets, or data". Trimmed to the fields the policy reads:

{
  "answers": {
    "kind": {
      "type": "choice",
      "choice": "bug",
      "confidence": 0.79,
      "probabilities": { "bug": 0.86, "feature": 0.07, "chore": 0.04, "question": 0.02, "unclear": 0.01 }
    },
    "ready": { "type": "noul", "noul": 0.91 },
    "risk": { "type": "score", "score": 1.94, "probabilities": { "0": 0.01, "1": 0.04, "2": 0.95 } },
    "needs_code": { "type": "noul", "noul": 0.97 }
  }
}

The response also reports token usage. Jev 1.13 is priced at $0.042 per million input tokens and output tokens are free, so a 290-token preflight call costs roughly a thousandth of a cent. Then the policy, which lives in your code and not in the model:

if (a.needs_code.noul < 0.50) return "no-code";
if (a.ready.noul < READY_THRESHOLD) return "clarify";
if (a.risk.score >= 1.5 || a.kind.confidence < CONFIDENCE_FLOOR) return "human";
return "dispatch";

Four questions go out in one request and run in parallel. They cannot see each other's answers, which is fine here because none of them depends on another. The docs put the per-request latency for a call like this in the 0.16 to 0.31 second band, and the demo draws its latency readout from that band. The order of the policy lines matters and is a choice: needs_code is checked first because a question should never be blocked for being risky.

Why moving a threshold never re-runs inference

The model returned probabilities, not decisions. The decision is a comparison your code makes against those probabilities, and the probabilities do not change when you change the comparison. So when you drag the ready threshold from 0.70 to 0.90, the demo loops over every stored answer and re-applies the four lines. No request goes out, no tokens are spent, and the Tally's cost stays put. The header above the counts chart says exactly that each time a slider moves.

Compare that with the same gate built by prompting a language model to "triage this and be strict". Getting stricter means rewording the prompt and re-running every request, and you still have to parse whatever comes back. Keeping the judgment typed and the policy in code is what makes the threshold a knob instead of a rewrite. TypeSafe's own guidance is to keep the questions and thresholds in one place so a reviewer can look at those two things instead of the whole implementation.

Where this fits in a Claude Code session

In practice this is a script, a hook, or a small worker that runs when a request lands: a new issue, a Slack message, a line added to a todo file. It needs an API key from the TypeSafe console at console.typesafe.ai, exported as TYPESAFE_API_KEY, and about thirty lines of code. If you want Claude Code to write those thirty lines, install the TypeSafe skill first:

claude plugin marketplace add typesafe-ai/skills
claude plugin install typesafe@typesafe-ai

The dispatch route is where the answers pay off a second time. The kind and risk values go into the brief that Claude Code receives, so a risk-2 bug can be routed to a stronger model tier with tests required, and a risk-0 chore can run on the cheaper one. The clarify route is worth more than it looks: a request that bounces back with one question costs a thousandth of a cent, while a Claude Code session that starts on an ambiguous brief costs real minutes and usually ends in a rewrite anyway. Keeping a human in the loop for the risky ten percent is the part you would have done anyway. The gate just makes it the default instead of a habit.

This is the first of three posts on where a System One model sits around an agentic workflow. The second, TypeSafe instead of Claude Code, covers jobs that are decisions rather than code. The third, TypeSafe after Claude Code, puts the same kind of gate on the other end of a run to verify what the agent produced. For the wider picture of how agents chain tasks, see agentic workflows.

Frequently asked questions

Key takeaways

• Preflight is cheap because the model never writes anything. A four-question call costs roughly a thousandth of a cent and returns a set of probabilities, which is a different economic unit from a session that reads files and writes code.
• The policy is four lines of code, and it is the whole product. Reviewers should read the questions and the thresholds, not the integration.
• Thresholds are knobs because answers are probabilities. Moving a threshold re-sorts stored answers; it never re-runs inference. A prompt that says "be stricter" cannot do that.
• The order of the policy lines is a decision. Checking needs_code before risk means a question is never blocked for being dangerous, which is exactly the behavior you want and exactly the kind of thing a prompt gets wrong on a bad day.
• The clarify route is the cheapest win. One question back costs nothing; a session started on an ambiguous brief costs the session.