# Watch for drift

> Know when live traffic stops looking like the examples a task was certified on. The server watches without labels, and suspends certification when it moves.

Source: https://opendecisio.com/docs/tasks/drift/

A certified [task](https://opendecisio.com/docs/glossary/#task) watches its own traffic with three checks that need no labels. Certification is suspended while the confidence alarm is on, so a shift does not quietly fill your queue with wrong [certified answers](https://opendecisio.com/docs/glossary/#certified-answer):

 

[Figure: The alarm suspends certification soon after the traffic shifts. A certified CLINC150 task on Gemma 4 31B, sent 5,363 messages, with messages that fit none of its options rising from message 1,000 to 30% at message 4,000. Certification was first suspended at message 1,570, after 40 wrong certified answers.]

 

## 1 Know what it watches

  

| Check | What it sees | What it does |
| --- | --- | --- |
| The [serving setup](https://opendecisio.com/docs/glossary/#serving-setup) | any change to the setup the certificate was made on | suspends certification at once: `serving_stack_changed` |
| The confidences | how often an answer’s [top probability](https://opendecisio.com/docs/glossary/#top-probability) falls below the certification examples’ 5th percentile | suspends certification while it alerts: `traffic_changed`, with `confidence` among its signals |
| The answer mix | how often each [option](https://opendecisio.com/docs/glossary/#option) was chosen in the last 500 answers, checked every 100 | reports `traffic_changed` with `answer_mix`; changes nothing |

 

- The confidence alert clears once the traffic is back to normal for about 300 answers.
- While certification is suspended, the answers themselves do not change: they come back marked `certified: false`, with the reason.
- The answer mix is weak on a task with many options, where a shift spread over them barely moves the mix. That is why only the confidence check suspends.

 

## 2 Read the status

 

`GET /v1/tasks/{id}` gives a task’s `status`: `ok`, `suspended` with what changed, or `traffic_changed` with the signals that alerted. For the task [Certify answers](https://opendecisio.com/docs/tasks/certify/#2-teach-the-task-with-a-certificate) registers:

 

```
curl -s http://127.0.0.1:8000/v1/tasks/ticket-routing \
  | uv run python -c 'import json, sys; s = json.load(sys.stdin)["status"]; print(json.dumps({k: s.get(k) for k in ("state", "text")}, indent=1))'
```

 

Run against the CPU stand-in at v0.19.0 on 2026-10-20. It printed:

```
{
 "state": "ok",
 "text": "ok"
}
```

 

`GET /v1/tasks` lists every task’s status.

 

## 3 Alert on it

 

`GET /metrics` serves Prometheus text for certified tasks:

 

```
curl -s http://127.0.0.1:8000/metrics | grep '^decisio_task_'
```

 

Run against the CPU stand-in at v0.19.0 on 2026-10-20. It printed, trimmed to the fields this page uses:

```
decisio_task_state{task="ticket-routing",state="ok"} 1
decisio_task_state{task="ticket-routing",state="traffic_changed"} 0
decisio_task_state{task="ticket-routing",state="suspended"} 0
decisio_task_answers_total{task="ticket-routing"} 0
decisio_task_certified_answers_total{task="ticket-routing"} 0
decisio_task_mix_alerts_total{task="ticket-routing"} 0
decisio_task_window_answers{task="ticket-routing"} 0
decisio_task_mix_line{task="ticket-routing"} 0.24466866666666665
decisio_task_confidence_statistic{task="ticket-routing"} 0.0
decisio_task_confidence_limit{task="ticket-routing"} 8.663302868327452
decisio_task_confidence_alerts_total{task="ticket-routing"} 0
```

  

| Metric | What it counts |
| --- | --- |
| `decisio_task_state{task,state}` | each task’s state |
| `decisio_task_confidence_statistic`, `_limit`, `_alerts_total` | the confidence check: where it stands, where it alerts, and how often it has |
| `decisio_task_mix_distance`, `decisio_task_mix_line` | the answer mix, and the line it alerts at |

 

It also counts each task’s answers and certified answers. The counters live in memory and start again when the server does.

 

## 4 Check it with labels

 

The checks cannot see a wrong answer, because they have no labels. The only direct check that the promise holds on live traffic is to label a sample of it and certify again.

 

- **Give the task an “other” option.** A message that fits none of the options gets a wrong answer, and the checks only notice once enough of them arrive.
- **To report without suspending**, certify with `"suspend_on_alert": false` in the `certify` block.

 

## What next

 

- [Certify answers](https://opendecisio.com/docs/tasks/certify/): the promise these checks protect, and how to certify again.
- [Operate](https://opendecisio.com/docs/operating/): health, the exit code and what else to watch in production.
- [Certified answers on real tasks](https://opendecisio.com/demos/teach-and-certify/): a [drift](https://opendecisio.com/docs/glossary/#drift) alarm on real traffic, run on the release it was measured on.