PROTRAILBLAZER
Every System One answer carries a confidence value from 0 to 1. A companion that acts above a threshold and asks you below it becomes a different character at 0.3 than at 0.9: eager and occasionally wrong, or careful and chatty. This simulation lets you move that threshold while playing, and shows what calibrated means by charting how often the companion was right at each confidence level.
A command goes into hero.last_command_text on the next tick and stays there, with its age, for 6 seconds.
Overconfident mode is on. Every answer reports at least 0.92, so the threshold never triggers a question.
Same mock, same distributions. The reported confidence is raised to at least 0.92 and the question never appears, so every mistake happens silently.
Bars are the current mode. Gray outlines are the other mode. A bucket with fewer than 5 decisions is drawn hatched. Calibration gap is the average distance between bar and diagonal, weighted by count.
The bar spans 500 ms. The red mark is 300 ms, when the next tick is due. A tick is skipped while a request is still out.
Three questions go out every 300 ms. Enter the dungeon to watch them update.
{}{}Every 300 ms the game sends the room as JSON state (where the hero and hound are, what the hero last said, every visible object and enemy, the last five events) and asks three questions: which of eight actions the hound should take, which thing it is about, and whether the hero needs help in the next second. The hound does not just run the top answer. It reads the confidence attached to it first, and that one comparison decides whether it acts on its own or turns to you.
every 300 ms (skipped while a request is out):
send({ state: room_state(), model: "jev-latest", questions })
on answer(a): // lands 70 to 500 ms later
if hound.busy and not (hound.following and a.help_wanted_now.noul >= 0.7):
return // finish the current job first
if a.action.confidence >= threshold:
do(a.action.choice, a.target.choice) // act alone
else:
top2 = two_highest(a.action.probabilities)
ask_player(top2, timeout = 2 s) // escalate
on timeout: do("follow")
When the hound stands between an unlit torch and a closet door, two actions both look reasonable, and the probabilities say so: two bars of similar height with the rest spread thin. That spread is the useful part. A model that only returned its top pick would hide the coin flip, and the game would find out by opening the wrong door. Here the low confidence turns into a two-button question that already contains the right answer most of the time, which is cheaper for the player than fixing a mistake.
A threshold of 0.7 is a promise: the hound acts alone only when it expects to be right about 70 percent of the time or better. That promise holds only if the reported confidence matches the observed hit rate, which is what calibrated means. The chart checks it live. Each bar is one confidence bucket and its height is how often the room rules agreed with the hound in that bucket. Calibrated bars sit on the dashed diagonal. Switch to overconfident mode and the same answers report 0.92 or more: the bars pile into the top bucket at the same accuracy as before, far under the line, and the mistake count climbs because nothing ever asks. A model that is often right but never says when it is unsure cannot be given autonomy, because no threshold separates its good calls from its bad ones.
The same number that trades speed for accuracy also sets how much a player has to manage. At 0.3 the hound is eager, rarely interrupts, and sometimes pulls the wrong thing. At 0.9 it asks about almost everything, which some players want: a player who wants company but not surprises, or who needs time to read and decide, can move it up and get a companion that never acts without asking. That only works because the confidence is honest. On an overconfident model the setting would do nothing at all.
About this demo: every answer here comes from a local mock, and nothing on this page calls Jev, which is in early access. The mock is built to be calibrated, and it can be because it cheats: it reads the room rules table, so it knows the correct action. For each call it draws a target confidence from a hardcoded curve set by the situation (an explicit command raises it, an enemy next to the hero raises attack or guard, two valid objects lower it, a health level near the attack or guard cutoff lowers it). With that probability the peak of the distribution goes on the correct action, otherwise on a plausible wrong one, and the peak height is solved so that the reported confidence comes out at the drawn value. Over many decisions, accuracy per bucket matches the reported number. Confidence is shown as 1 minus normalized entropy, an approximation of TypeSafe's unpublished formula; the chart is about the relationship between the reported number and observed accuracy, not the formula. Commands are read with a small keyword table. Latency is drawn uniformly from 70 to 500 ms, and token counts are the JSON length divided by 4.