Think when unsure

The questions decisio is unsure of are the ones a single read most often gets wrong. With thinking on, each of them is read again after the served model writes its reasoning, to get more of them right. The extra time goes to those questions only: a sure question is still answered from one read, in milliseconds.

Figure · Thinking when unsure

Only an unsure question is read again, after the model writes its reasoning.

A question with more than 16 options is never unsure: it stays at one read.

Source: docs/running.md ("Thinking when unsure") and docs/cli.md of the pinned release.

It follows the procedure of Chon (Apache-2.0). No measurement of its accuracy or its time is published yet.

1. Turn it on

Start the server with --think:

pip install "decisio[serve]==0.22.0"
python -m decisio.serve.vllm_engine --think

Shown, not run: it needs Linux with one NVIDIA card, which this check does not have.

• Thinking is off by default, on every base: a request is one read, as fast and as repeatable as before.
• --no-think turns it off, including when a tuned model’s repository turns it on in its defaults.think.
• A request decides for itself with "think": true or "think": false; without the field, the server’s setting applies.

2. Ask with it

curl -s http://127.0.0.1:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "Our invoice shows two seats, but we removed the second user last month.",
  "think": true,
  "questions": {"queue": {"type": "choice", "instructions": "Which team should handle this ticket?",
    "criteria": {"billing": "Invoices and charges", "access": "Logins and seats", "bug": "The product misbehaves"}}}
}' | python3 -c 'import json, sys; r = json.load(sys.stdin); print(json.dumps({"thinking": r.get("thinking"), "thinking_tokens": r["usage"].get("thinking_tokens")}, indent=1))'

Shown, not run: it needs a server that thinks; the CPU stand-in cannot write reasoning, only the vLLM engine on a card can.

• A question thinks when its top probability is under its type’s threshold: 0.75 for a choice, 0.92 for yes/no, 0.70 for a score.
• --think-factor scales the three thresholds, 1 by default; a lower factor thinks on fewer questions.
• --think-budget caps the reasoning at 512 tokens per question by default, from 16 to 4,096. It is set on the server only.
• All the unsure questions of a request go to the model in one call.

3. Read what it did

• thinking in the response says how many questions thought, the reasoning tokens and the time, and for each question whether it thought, or why not.
• usage.thinking_tokens counts the reasoning tokens, and usage.output_tokens includes them.
• The headers x-decisio-think-ms and x-decisio-think-tokens carry the time and the tokens.

The server’s setting, and how many questions have thought, are in /health:

curl -s http://127.0.0.1:8000/health \
  | python3 -c 'import json, sys; print(json.dumps(json.load(sys.stdin)["systemone"]["think"], indent=1))'

Run against the CPU stand-in at v0.22.0 on 2026-10-21. It printed:

{
 "default": false,
 "factor": 1.0,
 "budget": 512,
 "requests": 0,
 "questions": 0,
 "tokens": 0,
 "ms": 0.0
}

4. Know what stays at one read

• A question with more than 16 options.
• A question a registered task answers: its correction and certificate were fitted on one read.
• Every question of a request with images, an abstention option or two orders.

The response says why a question did not think. Where it runs, how it shares the server and what it does to certificates are in Requirements and limits.

What next

• Requirements and limits: thinking in production.
• Server flags: --think, --think-factor and --think-budget in full.
• The HTTP API: the think field and the thinking answer.