Skip to content

Detention and office hours

When a run cannot make progress, it stops and waits for a person instead of spending more. That stop is detention. The person fixes the blocker and explains it in office hours; the explanation becomes a lesson that later loops are given, in any repository on your machine unless it holds only in its own.

What sends a run to detention

Every detention prints the reason, then one next: line that says what to do about it.

Reason in the log What happened What next: says
agent reported a blocker The agent answered blocked, for example for a missing credential or a contradiction in the spec. Retrying cannot fix that, so there is no retry. Provide what the agent asked for (printed above it): a credential, access or a decision
rubric failed (exit N) The rubric failed on CHALK_MAX_RETRIES + 1 loops in a row. Fix the failure in io/rubric.log, or make the checkpoint smaller
agent stopped early (CLASS: message) The loop's Claude Code call failed. CLASS says how, and the message is the CLI's own. A failure that would repeat (tls, auth, cli_missing, usage_limit) detains at once; the others (network, rate_limit, overloaded, server, timeout, budget, max_turns, error, crash) are retried first. One line per class, for example: check ANTHROPIC_API_KEY or CLAUDE_CODE_OAUTH_TOKEN (auth); set CHALK_EXTRA_CA_CERTS (tls); raise CHALK_AGENT_TIMEOUT (timeout) or CHALK_BUDGET_USD (budget)
rubric timed out after Ns (CHALK_RUBRIC_TIMEOUT) The rubric ran longer than CHALK_RUBRIC_TIMEOUT on every retry. A rubric killed with exit 137, usually for running out of memory, and one whose command was not found (126, 127) are named the same way. Fix the failure in io/rubric.log, or make the checkpoint smaller
the final review did not run (CLASS: message) The final review's call failed, and CHALK_REVIEW_REQUIRED is true (the default), so nothing is pushed without a review. Fix what the line for CLASS says, then chalk office-hours; or set CHALK_REVIEW_REQUIRED=false to open requests that say the review did not run
rubric passed but no checkpoint was ticked in the spec Retries ran out, and the last loop left the tests passing without finishing a checkpoint. Tick it if it is done, or make it testable
loop limit of N reached with M checkpoints open The run used CHALK_MAX_LOOPS loops. Split the open checkpoints, or raise CHALK_MAX_LOOPS
final review still finds problems after N fix round(s) The final review failed again after CHALK_REVIEW_ROUNDS fix rounds. The findings are printed. Fix the findings, or raise CHALK_REVIEW_ROUNDS
deja_vu: … Only with CHALK_FP_RULES=on: the loop fails like an open detention of another ticket. Fix that ticket, which it names, first
repeat: … Only with CHALK_FP_RULES=on: the same failing change as the loop before. Fix the failure yourself; a retry would repeat it
no_change: … Only with CHALK_FP_RULES=on: the agent changed nothing. Make the checkpoint clearer, or do the step yourself
the loop's commits could not be brought onto chalk/<TICKET> The branch changed in its worktree during the run (a commit or an edit there), so the loop's commits could not be fast-forwarded onto it. Git's reason is printed. Merge the detention branch into chalk/<TICKET> yourself, and commit nothing to a branch while Chalk runs on it

When the spec check's call fails in a way that would repeat, such as a refused key, the run stops before the first loop with the same advice.

The verdicts are explained in Verdicts. Whatever the reason, when the last loop had three or more actions refused by permission checks, next: says so and points to chalk doctor instead: that is what a loop looks like when auto mode is unavailable for the model (see Permissions).

When a run stops without a detention

A run can also end because it was stopped (Ctrl-C, kill, a closed terminal) or because Chalk itself failed. Either way, once a loop has started, the agent's work since the last checkpoint is still committed and parked on a detention branch, and the log says run stopped by a signal (exit 143) or Chalk stopped unexpectedly (exit N) with the same three lines to continue. When the sandbox had nothing beyond your branch, the log says so instead.

If the work cannot be parked, for example because the container died, the log says why. A container that still exists is kept as chalk-sandbox-…-kept-<timestamp>, with the docker cp command that copies the work out; a container that is gone is reported as gone.

The telemetry database is not needed to finish a run. If it goes away mid-run (chalk db down refuses while runs are alive, unless you add --force), Chalk warns once, keeps the error in io/db.err, and the run goes on; the report card then misses what was not recorded.

What detention does

  • Nothing is pushed. The sandbox's work, including the failed attempt, is committed and parked on a local branch, detention/<TICKET>-<timestamp> (with -2, -3 and so on when one ticket is detained twice in the same second). Your chalk/<TICKET> branch keeps only the loops that passed.
  • The failure is logged in the lessons table as an open lesson: the reason, and the last 40 lines of the rubric's output or the blocker. When the rubric failed, the loop's fingerprint and first error are stored with it, so a later failure of the same kind can be matched.
  • The log says how to continue:
[PROJ-123] DETENTION: rubric failed (exit 1)
[PROJ-123] next: the rubric still fails after 2 retries; fix the failure in …/runs/PROJ-123/io/rubric.log, or make the checkpoint smaller
[PROJ-123] work parked on local branch detention/PROJ-123-1764000000. To unblock:
[PROJ-123]   cd '/home/sam/shop.worktrees/PROJ-123' && git switch 'detention/PROJ-123-1764000000'
[PROJ-123]   fix the blocker, commit, then: chalk office-hours -m "what was wrong"

Office hours

git switch detention/PROJ-123-1764000000
# fix the blocker (missing mock, wrong assumption, unclear spec), commit
chalk office-hours -m "Payments client needs the sandbox base URL in tests"

chalk office-hours must be run from a detention/… branch. It:

  1. Records your note as the resolution of the open lesson, with your git email. This is the record of human intervention that the pull or merge request reports.
  2. Distils a lesson. With CHALK_DISTILL=true (the default), a cheap model in a short-lived, read-only sandbox reads the failure, your note and your fix: the diff since the detention commit or, when you committed nothing, the fact that the fix was made outside the repository. Your note is the source of truth about what changed where. The model writes a rule and gives it a scope:

    • general: it holds in any repository, and is recalled anywhere.
    • repo: it depends on this repository, its sandbox image or its environment, and is recalled only here.
    • none: the evidence supports no rule, for example a one-off fix outside the repository. No lesson is recorded; your note is kept and recalled only here.

    The log says which: lesson: …, lesson, recalled in this repository only: … or no lesson distilled: …. If distillation fails, your note is still kept and used as the lesson. 3. Moves to a tutoring branch, tutoring/<TICKET>-<timestamp>, so the detention branch stays as it was. 4. Resumes the run if the spec still has open checkpoints, or opens the pull or merge request if your fix finished it. --detach resumes in the background.

The gates treat tutoring/* branches like chalk/* branches.

Writing a useful note

The note is what later loops are told, so describe the cause and the fix, not the symptom:

  • Good: "Payments client needs the sandbox base URL in tests; set PAYMENTS_URL in tests/setup.ts."
  • Less useful: "tests were failing".

When part or all of the fix is outside the repository, such as a rebuilt sandbox image, a credential or a CI variable, say so in the note. The diff shows only what you committed; without the note, a small commit that came with an image change reads as if it were the whole fix. For example: "PyYAML was missing from the sandbox image, which now installs python3-yaml; requirements.txt also declares PyYAML."

Rules that should apply to every ticket from now on belong in the textbook instead.

The failure lifecycle

flowchart LR
  run[Loops] -- cannot progress --> detention[Detention: work parked on detention/ branch, open lesson logged]
  detention --> fix[A person fixes the blocker and commits]
  fix --> office[chalk office-hours -m note]
  office --> lesson[Note resolves the lesson; a cheap model distils a rule and its scope]
  lesson --> tutoring[New tutoring/ branch]
  tutoring -- checkpoints open --> run
  tutoring -- all done --> request([Pull or merge request])
  lesson -. recalled on later loops .-> run
  1. Loops run until the run cannot progress, then it goes to detention: the work is parked on a detention/ branch and an open lesson is logged with the failure.
  2. A person switches to that branch, fixes the blocker and commits.
  3. chalk office-hours -m "…" records the note as the lesson's resolution, and a cheap model distils it, with the fix, into a rule for any repository, a rule for this one, or no rule.
  4. Chalk moves to a new tutoring/ branch. If checkpoints are still open, the loops resume there; if not, the pull or merge request opens.
  5. From then on, resolved lessons are recalled into the prompts of later loops whose failures look the same.

Giving up on a ticket

A detention branch stays until you delete it. chalk cleanup keeps branches that are neither merged nor pushed; chalk cleanup --all deletes them, detention work included.