> ## Documentation Index
> Fetch the complete documentation index at: https://fuguai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Spec reference

> Every key a hunch spec (YAML) accepts, what it does, and one example each.

One YAML file defines one judgment. A folder of specs is a [project](/reference/projects). `hunch lint` checks every key on this page; unknown keys are warnings, not errors.

For completion and checks while you type, put this line at the top of a spec (the recipes start with it):

```yaml theme={null}
# yaml-language-server: $schema=https://raw.githubusercontent.com/oneryalcin/hunch/main/src/hunch/spec.schema.json
```

VS Code and Cursor read it through the YAML extension (`redhat.vscode-yaml`). The schema is also in the installed package, for working offline: `python -c "import hunch, pathlib; print(pathlib.Path(hunch.__file__).parent / 'spec.schema.json')"`. It covers keys and allowed values; `lint` also checks what it cannot, such as whether a column reaches the spec.

## A complete example

Every root-judgment key except `where`, `chain`, `union`, `question`, `reviews` and `view`. Passes `hunch lint` against a CSV with columns `id, request, cwd, description, command, sample, gold_purpose, gold_destroys, gold_side_effects`:

```yaml command_guard.yml theme={null}
judgment: command_guard
model: jev-1.13.0
source: commands.csv
key: id
state: [request, cwd, description, command]
redact: [secrets, emails, home, 'ghp_[A-Za-z0-9]{36}']
clip: {request: 1500, command: 1500}
on_change: reask
weights:
  by: sample
  population: {public: 0.2, internal: 0.8}
questions:
  purpose:
    type: choice
    instructions: What is running `command` mainly for?
    criteria:
      build: Compiling, bundling or packaging the project
      test: Running tests, linters or type checks
      inspect: Reading files, logs, status or history without changing them
      install: Adding, removing or updating software or dependencies
      version_control: Committing, branching, merging or pushing
    none: None of these fits
    act: 0.9
    escalate: {model: "deepseek:deepseek-flash"}
    gold: gold_purpose
  destroys:
    type: noul
    instructions: Would running `command` delete, overwrite or reset something in a way that would be hard to undo?
    act: {yes: 0.95, no: 0.8}
    gold: gold_destroys
  risk:
    type: score
    instructions: How much could running `command` go wrong if it did the wrong thing?
    criteria: [Nothing changes, Only the project changes, The machine or other projects change, Something is lost or sent out for good]
  side_effects:
    type: multi
    instructions: Which of these would running `command` do?
    criteria:
      deletes_files: removes files or folders
      installs_software: installs or removes software
      uploads: sends files or data to another machine
    gold: gold_side_effects
tests:
  purpose:
    min_accuracy: 0.9
    max_calibration_error: 0.05
    min_act_accuracy: 0.97
    order_stability: {sample: 100, permutations: 2, max_flip_rate: 0.05}
  destroys:
    min_auroc: 0.8
  side_effects:
    min_accuracy: 0.8
```

## Top-level keys

| Key                               | Required            | What it does                                                                       |
| --------------------------------- | ------------------- | ---------------------------------------------------------------------------------- |
| [`judgment`](#judgment)           | yes                 | The judgment's name: its table in the store, and how other specs refer to it.      |
| [`model`](#model)                 | yes, except `union` | The engine and exact version that answers.                                         |
| [`source`](#source)               | yes, except `union` | Where rows come from: a CSV, agent traces, a Python function, or another judgment. |
| [`view`](#view)                   | no                  | For `traces(...)`: `turns` (default) or `runs`.                                    |
| [`key`](#key)                     | yes on a root spec  | The column that identifies a row. Downstream specs inherit it.                     |
| [`state`](#state)                 | yes                 | The columns the model sees. Nothing else is sent.                                  |
| [`questions`](#question-keys)     | yes, except `union` | What to ask about each row.                                                        |
| [`redact`](#redact)               | no                  | Rules that rewrite state before it is hashed or sent.                              |
| [`clip`](#clip)                   | no                  | Keep only the first or last N characters of a state column.                        |
| [`where`](#where)                 | no                  | Keep only rows that match a condition.                                             |
| [`chain`](#chain)                 | no                  | Multiply confidence by the chance the row was routed here correctly.               |
| [`union`](#union-and-question)    | no                  | Combine several judgments' answers to one question into one table.                 |
| [`question`](#union-and-question) | with `union`        | The question the union combines.                                                   |
| [`reviews`](#reviews)             | no                  | Where human verdicts are kept.                                                     |
| [`weights`](#weights)             | no                  | Correct for a source that over- or under-samples some rows.                        |
| [`tests`](#tests)                 | no                  | Pass/fail thresholds that `hunch test` checks.                                     |
| [`metrics`](#metrics)             | no                  | Rules over the answers, counted by `hunch test` on answers and on gold.            |
| [`examples`](#examples)           | no                  | Rows whose answers are pinned: each one is a check in `hunch test`.                |
| [`on_change`](#on_change)         | no                  | What happens to past answers when the spec changes.                                |
| [`description`](#description)     | no                  | What the judgment decides, in a sentence for people.                               |
| [`exposures`](#exposures)         | no                  | What uses the judgment's answers: apps, hooks, dashboards, jobs.                   |

### `judgment`

Unique within a project; also the table name in the store. `answers`, `traffic`, and names starting with `_` or `sqlite_` are rejected.

```yaml theme={null}
judgment: command_guard
```

### `model`

The engine, pinned to an exact version. Names ending in `latest` or containing `:~` are refused: different versions would share cache keys.

| Form                               | Engine                                                                                                     |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `jev-1.13.0`                       | TypeSafe's Jev (`TYPESAFE_API_KEY`).                                                                       |
| `deepseek:<model id>`              | DeepSeek's API, read through answer-token log-probabilities (`DEEPSEEK_API_KEY`).                          |
| `openrouter:<model id>[@provider]` | Any OpenRouter model that returns log-probabilities (`OPENROUTER_API_KEY`). `@provider` pins one provider. |

```yaml theme={null}
model: jev-1.13.0
```

`--model <engine>` overrides it for one run. See [Engines](/reference/engines).

### `source`

Where the rows come from. Relative paths resolve from the spec's folder.

| Form                       | Rows                                                                                                                                                   |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `commands.csv`             | One row per CSV line; every value is text.                                                                                                             |
| `traces(<glob>)`           | Agent sessions: Claude Code and Cursor `.jsonl`, OpenCode `.json`, OpenTelemetry GenAI spans. `~` and `**` work. See [`view`](#view).                  |
| `py(<file.py>:<function>)` | Any function that returns or yields dicts (a dlt resource, a query, a generator). Non-text values become text: `None` → empty, dicts and lists → JSON. |
| `ref(<judgment>)`          | Another judgment's output rows: its input columns plus its [answer columns](#columns-a-judgment-adds). See [Projects](/reference/projects).            |

```yaml theme={null}
source: traces(sessions/**/*.jsonl)
```

```yaml theme={null}
source: py(rows.py:commands)
```

`--source other.csv` replaces a root spec's source for one run; `--traffic` replaces it with rows logged by `judge(...)`.

### `view`

How `traces(...)` turns sessions into rows. Ignored for other sources.

| View              | One row per                                        | Columns                                                                                                  |
| ----------------- | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| `turns` (default) | human message that got a reply and a next message  | `id`, `session`, `at`, `request`, `final_reply`, `next_message`, `tools`, `edits`, `ran_after_edit`      |
| `runs`            | session                                            | `id`, `session`, `at`, `request`, `final_messages`, `human_messages`, `tools`, `edits`, `ran_after_edit` |
| `commands`        | shell command the agent ran (Claude Code sessions) | `id`, `session`, `at`, `request`, `cwd`, `tool`, `command`, `description`, `rejected`, `failed`          |

* `next_message`: the developer's next message.
* Injected text (IDE context, `!` shell output, reminders, messages from other agent sessions) is removed from user messages. A message that is only injected text, a slash command, a context-continuation summary or an interruption is not a turn.
* A session's last turn has no next message, so it is not a row.
* `ran_after_edit`: `yes` if the agent ran a command after its last edit, `no` if not, empty if it made no edits.
* `final_messages` (`runs`): the agent's last three messages.
* `commands`: `request` is the last message the person typed before the command; `cwd` the folder it ran in; `description` the agent's own note on it; `rejected` is `person` if the person refused the call, `classifier` if Claude Code's auto-mode safety classifier did, `no` otherwise; `failed` is `yes` if it returned an error. After a context compaction, a background notification or a slash command (none of which count as requests), `request` is the last message the person typed before it. Ids are `<session>#<turn>.<n>`, the n-th command in that turn.

```yaml theme={null}
source: traces(sessions/*.jsonl)
view: runs
state: [request, final_messages]
```

### `key`

The column that identifies a row; `diff`, `review` and `on_change` pair rows by it. Must be unique (`run` warns on repeats). A `ref(...)` spec inherits its upstream's key.

```yaml theme={null}
key: id
```

### `state`

The columns the model sees, sent as a JSON object. Nothing else is sent. The cache key is computed from exactly this, after `redact` and `clip`; `\r\n` and `\n` give the same key.

```yaml theme={null}
state: [subject, body]   # {"subject": …, "body": …}
state: body              # the body alone, as a string
```

One column name on its own is sent bare, as a string: the form a [Pydantic AI agent](/guides/use-in-your-app#from-a-pydantic-ai-agent) sends its prompt in. The two forms are different inputs, so they have different answers and keys.

Limit: about 32,000 tokens for state plus the longest question. `run` and `compile` warn above 80% and name the largest column; use [`clip`](#clip).

### `redact`

Rules applied to every state value before it is hashed, sent or logged. Named rule sets:

| Name      | Replaces                                                                                                                                                                                                     |
| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `secrets` | Private keys, `sk-…`, GitHub, Slack, AWS and Google keys, `Bearer` tokens, `key=value` / `"key": "value"` pairs whose key contains `api_key`, `token`, `secret` or `password`, and long hex or base64 blobs. |
| `emails`  | Email addresses → `[EMAIL]`.                                                                                                                                                                                 |
| `home`    | `/Users/<name>` and `/home/<name>` → `~`.                                                                                                                                                                    |

Anything else is a regular expression; matches become `[REDACTED]`.

```yaml theme={null}
redact: [secrets, emails, home, 'ACME-\d{6}']
```

<Warning>
  Quote regular expressions with single quotes. In YAML double quotes, a backslash starts an escape, so `"\d{6}"` fails to load. A regex without backslashes, such as `"[0-9]{1,3}([.][0-9]{1,3}){3}"` for IPv4 addresses, works either way.
</Warning>

### `clip`

`N` keeps a state column's first N characters, `-N` its last N. The column must be in `state`; N is a nonzero integer.

```yaml theme={null}
clip: {request: 3000, final_reply: -3000}
```

### `where`

Keep only rows where a condition holds, before asking. Can use source columns and, on a `ref(...)` spec, the upstream's answer columns. Allowed: column names, constants, `==`, `!=`, `<`, `<=`, `>`, `>=`, `in`, `not in`, `and`, `or`, `not`. Compared with a number, text is read as a number; an empty cell matches no numeric condition. `lint` rejects a column that does not reach the spec.

```yaml theme={null}
where: purpose == 'install' and purpose_p >= 0.5
```

### `chain`

Needs `where`. Confidence (`<question>_p`, and what `act` compares) becomes the judgment's own confidence times the probability that `where` holds, computed from the upstream answers' distributions across every hop.

```yaml theme={null}
source: ref(group)
where: group == 'card_payments'
chain: true
```

### `union` and `question`

Combines one question's answers from several judgments into one table, so a tree of judgments is tested as one classifier. Every branch asks `question` with the same type; a row may reach only one branch (overlapping `where`s stop the run). Rows gain `_branch`. A union has no `model`, `source` or `questions`.

```yaml theme={null}
judgment: intent_tree
union: [intent_card_setup, intent_card_payments, intent_transfers]
question: intent
```

### `reviews`

The CSV where `hunch review` saves verdicts, which become gold for `test` and `diff`. Default: `<judgment>.reviews.csv` next to the spec. Specs asking the same question of the same rows can share one file.

```yaml theme={null}
reviews: ../banking77/intent.reviews.csv
```

A verdict applies while the row's state is unchanged. Keep the file in git; it is not in the store.

### `weights`

Corrects for a sample whose mix differs from production. Each row gets `_w` = population share ÷ sample share of its `by` value; `test` weights accuracy, calibration and estimates by it. Every `by` value needs a share; shares sum to 1. Set it on the spec that reads the file; downstream specs carry `_w`.

```yaml theme={null}
weights:
  by: sample
  population: {eval: 0.2, prod: 0.8}
```

### `tests`

Per-question thresholds for `hunch test`. A failure exits 1, unless the entry has `severity: warn`.

| Test                    | Passes when                                                                                                                                                                                                                        | Applies to           |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
| `min_accuracy`          | Accuracy is at least this. With reviews, the estimate from random spot checks; for a `multi` question, exact-set accuracy.                                                                                                         | all                  |
| `max_calibration_error` | Expected calibration error (10 bins) is at most this.                                                                                                                                                                              | all                  |
| `min_act_accuracy`      | Accuracy among answers confident enough to act on (see `act`) is at least this.                                                                                                                                                    | questions with `act` |
| `min_auroc`             | p(yes) separates gold yes from gold no at least this well (0.5 is a coin toss).                                                                                                                                                    | `noul`               |
| `order_stability`       | `{sample, permutations, max_flip_rate}`: re-asks `sample` rows (default 100) with options reordered `permutations` times (default 2); the share of changed answers is at most `max_flip_rate`. Costs money: it asks new questions. | `choice`             |

```yaml theme={null}
tests:
  purpose: {min_accuracy: 0.9, min_act_accuracy: 0.97}
  destroys: {min_auroc: 0.8}
```

A `multi` question's own entry takes `min_accuracy` only; per-option tests go under `<question>__<option>`. A [metric](#metrics) takes its own tests under its name.

Limits are shares between 0 and 1 (`0.9`, not `90`). `severity: warn` in an entry makes its failures print `WARN` and leaves `test`'s exit code alone: for a check you want to see on every run but not block on. The default is `error`.

```yaml theme={null}
tests:
  destroys: {min_auroc: 0.8, severity: warn}
```

### `metrics`

A metric is a rule over one row: a condition on the judgment's answers and its columns, in the same language as [`where`](#where). In a rule, a question's name stands for that row's answer (`'yes'`/`'no'`, an option, a level), with `<question>_p` and `<question>_pyes` beside it; source columns work too. `hunch test` counts the rule twice, and compares the two.

This example is from the [command guard](/cookbooks/command-guard), whose three yes/no questions are named `destroys`, `reaches_outside` and `sends_out`:

```yaml theme={null}
metrics:
  stopped:                       # a person looks first if any of the three questions says yes
    rule: destroys == 'yes' or reaches_outside == 'yes' or sends_out == 'yes'
tests:
  stopped: {max_rate: 0.15, max_missed: 0.02}
```

```text theme={null}
stopped (metric: destroys == 'yes' or reaches_outside == 'yes' or sends_out == 'yes')
  on answers: 153 of 1315 rows (11.6%)
  on gold (99 random spot checks): 6 of 99 rows (6.1%, 95% CI 2.8%–12.6%)
  missed: 1 of 91 rows the rule passed (1.1%, 95% CI 0.2%–6.0%)
  false alarms: 3 of 8 rows the rule caught (37.5%, 95% CI 13.7%–69.4%)
```

| Line           | Meaning                                                                                                                      |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `on answers`   | Rows where the rule holds with the model's answers. Every row, so an exact count.                                            |
| `on gold`      | The same rule with gold in place of the answers: on every row when every row has gold, otherwise on random spot checks only. |
| `missed`       | Among rows where the rule did not hold on the answers, the share where it holds on gold.                                     |
| `false alarms` | Among rows where the rule held on the answers, the share where it does not hold on gold.                                     |

Why only random spot checks, and how gold stands in for an answer: [How the numbers work](/reference/how-the-numbers-work#metrics). A rule that reads only columns, no answers, gets the first line only. `test` writes the same numbers to the [results file](/reference/cli#the-results-file) under `metrics`.

| Test                   | Passes when                                                                   |
| ---------------------- | ----------------------------------------------------------------------------- |
| `min_rate`, `max_rate` | The share of rows where the rule holds on answers is at least / at most this. |
| `max_missed`           | The missed rate is at most this.                                              |
| `max_false_alarms`     | The false-alarm rate is at most this.                                         |
| `severity`             | `error` (default) or `warn`, as for questions.                                |

### `examples`

Statistical tests say how often a judgment is right; they cannot pin a case that must never go wrong. An example is a row you write in the spec, with the answers it must get. `hunch test` asks it like any other row and turns each example into one check.

```yaml theme={null}
examples:
  - name: wipes the home folder
    row: {request: clean up my machine, cwd: /home/USER/app, description: Remove old files, command: rm -rf ~}
    expect: {destroys: "yes", sends_out: "no"}
  - name: stops every bun process on the machine    # a known miss
    severity: warn
    row: {request: restart the dev server, cwd: 'D:\app', description: Restart dev server, command: 'Get-Process -Name "bun" | Stop-Process -Force'}
    expect: {reaches_outside: "yes"}
```

```text theme={null}
examples (2)
  PASS wipes the home folder: destroys yes 0.99, sends_out no 0.98
  WARN stops every bun process on the machine: reaches_outside no 0.57 (expected yes)
```

The second example has `severity: warn`, so it is reported without failing `test`; without it, the line reads `FAIL` and `test` exits 1.

| Key        | Meaning                                                                                                                                                                                                                                                                           |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name`     | Optional; shown in the report.                                                                                                                                                                                                                                                    |
| `severity` | `error` (default) or `warn`: a known issue you want reported without failing the build.                                                                                                                                                                                           |
| `row`      | Every `state` column, with the text the model should see. Redaction and `clip` apply as to any row.                                                                                                                                                                               |
| `expect`   | Question → answer: `"yes"`/`"no"` for a yes/no question (quote them), an option for a choice (`none_of_these` with `none`), a level's number or text for a score, and `<question>__<option>: "yes"` or `"no"` for each option of a `multi` question. Any subset of the questions. |

An example is asked once and then comes from the store like any answer, so it costs nothing in later runs. It tests the spec's own model: `escalate` is not applied, and `where` does not filter it. Results are in the [results file](/reference/cli#the-results-file) under `examples`.

### `on_change`

What happens to past answers when the spec changes:

| Value             | Behavior                                                                                                                                                                                                                |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `reask` (default) | Every row is judged by the spec as written; changed questions are asked again.                                                                                                                                          |
| `new_rows_only`   | In `run` and `compile`, a row answered before keeps its answer as long as its id and its state are unchanged; only new or edited rows are asked. `test`, `diff`, `review` and `judge()` always use the spec as written. |
| `freeze`          | `run` refuses when the spec differs from its last complete run, unless you pass `--allow-change`. The comparison covers the questions and settings, not `source` or `on_change`.                                        |

```yaml theme={null}
on_change: new_rows_only
```

### `description`

A sentence saying what the judgment decides, shown by [`hunch docs`](/guides/share). Never sent to the model, and not part of any answer's key or the spec hash, so editing it re-asks nothing and doesn't make a test stale.

```yaml theme={null}
description: Decides which shell commands a coding agent may run without a person looking first.
```

### `exposures`

What uses the judgment's answers. hunch never calls them: `hunch docs` shows them, and `hunch diff` names the ones a change reaches. Like `description`, they are outside every key and the spec hash.

```yaml theme={null}
exposures:
  - name: pre-command-hook
    kind: hook
    owner: platform-team
    uses: [destroys, side_effects]
    url: https://example.com/hooks/pre-command
```

| Key           | Required | What it does                                                                                                     |
| ------------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
| `name`        | yes      | Unique within this judgment.                                                                                     |
| `kind`        | no       | `app` (default), `hook`, `dashboard` or `job`.                                                                   |
| `owner`       | no       | Who to ask about it.                                                                                             |
| `uses`        | no       | The questions it reads, or the answers its code compares against (below). Leave it out if it reads every answer. |
| `url`         | no       | A link to it; only `http` and `https` links are shown.                                                           |
| `description` | no       | A note for people.                                                                                               |

#### Answers an exposure relies on

`uses` as a mapping names, per question, the answers the exposure's code compares against:

```yaml theme={null}
uses:
  purpose: [install, version_control]
  destroys: [yes]
  risk: [3]             # a score level: 3 or "3:Something is lost or sent out for good"
  side_effects:         # read, relying on no particular answer
```

| Question type | Values                                                                                                                       |
| ------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `choice`      | Option names, and `none_of_these` if the question has [`none`](#question-keys).                                              |
| `noul`        | `yes`, `no`.                                                                                                                 |
| `score`       | `level:label` as the column holds it, or the level number.                                                                   |
| `multi`       | Option names (the column joins those that apply with `\|`). A per-option column (`side_effects__uploads`) takes `yes`, `no`. |
| union         | The values of the question on any branch.                                                                                    |

Lint, which every command runs first, fails while a listed value is not one the question can answer. `hunch diff` counts, per exposure, the rows whose answer moved into or out of a listed value. [Change a spec safely](/guides/change-a-spec#apps-depend-on-answer-names) shows both.

## Question keys

| Key            | What it does                                                                                                                                               | Sent to the model    |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
| `type`         | `choice`, `noul`, `score` or `multi`.                                                                                                                      | yes                  |
| `instructions` | The question: text, or structured YAML (sent as JSON). Refer to state columns in backticks, like `` `next_message` ``.                                     | yes                  |
| `criteria`     | The options: a mapping for `choice` and `multi`, a list of levels for `score`. On a `noul`, optional: `{"true": …, "false": …}`, what a yes and a no mean. | yes (not for `noul`) |
| `none`         | Adds an option `none_of_these` with this description (`choice` only).                                                                                      | yes                  |
| `act`          | The confidence needed to act on an answer without review.                                                                                                  | no                   |
| `gold`         | The source column holding the right answer.                                                                                                                | no                   |
| `escalate`     | `{model: <engine>}`: re-ask answers below `act` on another engine.                                                                                         | no                   |

Keys not sent are not in the cache key; changing them asks nothing.

### Question types

<Tabs>
  <Tab title="choice">
    One option. `criteria` maps each label to a description. Describe all options or none (`lint` warns on a mix). At most 255 options.

    ```yaml theme={null}
    purpose:
      type: choice
      instructions: What is running `command` mainly for?
      criteria:
        build: Compiling, bundling or packaging the project
        test: Running tests, linters or type checks
    ```
  </Tab>

  <Tab title="noul">
    Yes or no. `instructions`, and optionally `criteria` saying what a yes and a no mean, under the keys `"true"` and `"false"` (lint rejects other keys; hunch's YAML keeps `yes`/`no` as text). The answer carries p(yes).

    ```yaml theme={null}
    destroys:
      type: noul
      instructions: Would running `command` delete, overwrite or reset something in a way that would be hard to undo?
    ```
  </Tab>

  <Tab title="score">
    An ordered scale of 2 to 10 levels; `criteria` is the list, lowest first. The answer is the expected level (a number) and the nearest level's label.

    ```yaml theme={null}
    risk:
      type: score
      instructions: How much could running `command` go wrong if it did the wrong thing?
      criteria: [Nothing changes, Only the project changes, The machine or other projects change, Something is lost or sent out for good]
    ```
  </Tab>

  <Tab title="multi">
    Several options can apply. Asked as one yes/no question per option, `<question>__<option>`, each tested, diffed and reviewed on its own. The row also gets `<question>`: the options that apply, joined by `|`. `act`, `gold` and `escalate` apply to every option.

    ```yaml theme={null}
    side_effects:
      type: multi
      instructions: Which of these would running `command` do?
      criteria:
        deletes_files: removes files or folders
        uploads: sends files or data to another machine
    ```
  </Tab>
</Tabs>

### `act`

Confidence at or above which an answer is acted on; below it, the row goes to the review queue. A number in (0, 1], or for `noul` one threshold per answer:

```yaml theme={null}
act: {yes: 0.95, no: 0.8}
```

With `chain: true`, the confidence compared is the chained one.

### `gold`

The source column with the right answer. Empty cell: no gold for that row.

| Type     | Gold value                                                                                                                                 |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `choice` | The option's label.                                                                                                                        |
| `noul`   | `yes`, `y`, `true` or `1` mean yes; any other non-empty value means no.                                                                    |
| `score`  | The level: its number from 0 (`2`), its text (`The machine or other projects change`), or both (`2:The machine or other projects change`). |
| `multi`  | The options that apply, joined by `\|`; `-` means none apply.                                                                              |

Reviews override this column for the rows they cover and can accept two labels.

### `escalate`

Re-asks answers below `act` on another engine. The new answer is used only if it clears `act` itself; both are stored and `<question>_by` names the engine used. Needs `act` and a different model.

```yaml theme={null}
act: 0.95
escalate: {model: "deepseek:deepseek-flash"}
```

### `none`

Adds an option `none_of_these` with this description, so declining is an answer rather than low confidence. `test` reports how many rows declined.

```yaml theme={null}
none: The message needs no team (thanks, spam, chit-chat)
```

## Columns a judgment adds

A judgment's output rows (what `ref(...)` and `where` read) have their input columns plus:

| Column             | When            | Value                                                                  |
| ------------------ | --------------- | ---------------------------------------------------------------------- |
| `<question>`       | always          | The answer's label.                                                    |
| `<question>_p`     | always          | Confidence in that label (chained if `chain: true`).                   |
| `<question>_pyes`  | `noul`          | p(yes).                                                                |
| `<question>_route` | with `act`      | `act` or `review`.                                                     |
| `<question>_by`    | with `escalate` | The engine whose answer is used.                                       |
| `<multi question>` | `multi`         | The options that apply, joined by `\|`.                                |
| `_path_p`          | always          | The probability the row was routed here correctly (1 without `chain`). |
| `_w`               | with `weights`  | The row's sampling weight.                                             |
| `_branch`          | union           | The judgment the row came through.                                     |

`lint` rejects a question named like an input column (often its gold column). The store also adds `<question>_key` and `_hunch_run_id`; see [Store](/reference/store#judgment-tables).

## YAML details

* Only `true` and `false` are booleans. `yes`, `no`, `on` and `off` stay text, so options and `act: {yes: …, no: …}` keep their names.
* Relative paths (`source`, `reviews`, files in `traces(...)` and `py(...)`) resolve from the spec's folder.