Skip to content
Decisio

DocsUse cases

3 min read · updated

Coding-agent gatekeeper

Decide allow, ask or block for each shell command a coding agent proposes, by your team's own rules, and ask a person only when decisio is not sure.

decisio names the rule a command falls under; your policy maps the rule to an action, and only a certified answer acts on its own:

import json
import urllib.request


POLICY = {
    "inspect": "allow",
    "git_read": "allow",
    "git_local_write": "allow",
    "git_discard": "ask",
    "git_push": "ask",
    "git_push_force_or_protected": "block",
    "test_build_lint": "allow",
    "run_scripts": "ask",
    "install_packages": "ask",
    "edit_workspace": "allow",
    "delete_in_workspace": "ask",
    "outside_workspace": "block",
    "secrets": "block",
    "network_and_cloud": "ask",
    "remote_code_or_privilege": "block",
    "other": "ask",
}


QUESTION = {
    "type": "choice",
    "instructions": "Which of the team's rules covers this shell command?",
    "criteria": {
        "inspect": None,
        "git_read": None,
        "git_local_write": None,
        "git_discard": None,
        "git_push": None,
        "git_push_force_or_protected": None,
        "test_build_lint": None,
        "run_scripts": None,
        "install_packages": None,
        "edit_workspace": None,
        "delete_in_workspace": None,
        "outside_workspace": None,
        "secrets": None,
        "network_and_cloud": None,
        "remote_code_or_privilege": None,
        "other": None
    }
}




def route(text):
    request = urllib.request.Request(
        "http://127.0.0.1:8000/v1/systemone",
        data=json.dumps({"state": text, "questions": {"rule": QUESTION}}).encode(),
        headers={"Content-Type": "application/json"},
    )
    with urllib.request.urlopen(request) as response:
        answer = json.load(response)["answers"]["rule"]
    # A certified answer acts on its own; anything else goes to a person.
    if answer.get("certified"):
        return POLICY[answer["choice"]]
    return "ask"




print(route("rm -rf build/ && npm run build"))

Run against the CPU stand-in at v0.18.1 on 2026-10-19. It printed:

ask

Until the task is certified (step 2), every answer goes to a person, as on the stand-in.

Good fit

  • Written rules for what an agent may do, each with an action.
  • Labelled commands for every rule, to teach and certify from.
  • A catch-all rule that asks, for the commands no rule names.

Not a good fit

  • A policy that is not written down: decisio learns rules from examples, it does not invent them.
  • Commands whose risk depends on things the text does not show, such as what a script does inside.

1. Write the question

Section titled “1. Write the question”

One choice question: which rule covers this command. Keep the policy beside it, so the action is yours, not the model’s, and changing an action needs no teaching.

2. Teach it and certify it

Section titled “2. Teach it and certify it”

Teach the question from labelled commands and certify it at 5%.

Send at least 320 labelled commands. The server holds back half to certify, and a 5% certificate needs 59 of those; the other half teaches, at 10 per option for this question's 16 options. More examples certify a larger share.

POST /v1/tasks
{"id": "gatekeeper",
 "examples": [{"request": {"state": "<a past item of yours>", "questions": {"rule": <the question above>}},
               "answer": "delete_in_workspace"},
              ...],
 "certify": {"alpha": 0.05}}

A shape to fill in, not a command to run.

Certify answers runs this call on a server, with its answer.

3. Route on the certificate

Section titled “3. Route on the certificate”
  • A certified answer takes the rule’s action: allow, ask or block.
  • Anything else asks a person.
  • In the measured run, a kind of command that no rule named was never certified, so it went to a person (Watch for drift).

What it did

Section titled “What it did”
  • 34% to 40% of a coding agent's shell commands decided on their own at a 5% error target, with the wrong action on 1.6% to 2.3% of them: 16 rules, Gemma 4 12B on a Mac (the 6-bit MLX conversion), taught with 20 labelled commands per rule, 3 draws, 1,481 test commands.
  • When a kind of command no rule names rose to 30% of the traffic, none of its 184 commands was certified.

Measured on decisio 0.14.0, Gemma 4 12B on a Mac, 2026-10-17 and 2026-10-18.

What next

Section titled “What next”