Specify a review and escalation workflow

Specify the workflow that handles a learner's dispute of published AI feedback on the fictional platform, from filing to closure. Define the case states and transitions, who may take each action and how that is enforced, how a corrected result is bound to the exact reviewed artifact, what happens when a deadline passes, and how concurrent actions and repeated publication requests are handled. Add a risk-register entry for the risk that a wrong score stays in final contest results because its dispute is not decided in time, the audit events and the remedy for affected records, a reviewer-capacity estimate for the contest window, and acceptance fixtures that would prove the dispute path.

Keep the scenario concrete: "bias exists" tells nobody what to test. Separate observed incidents from hypothetical risks and record your confidence in the cause. A control needs an owner, a deadline, and evidence that it operates; a planned control is not an implemented one. The NIST AI RMF Playbook suggests actions for this work; adapt them rather than treating them as a mandatory checklist.

The review contract for the fictional platform in the fixtures below:

Rejected and Expired are final for that artifact revision; new output starts again as a Draft. Some workflows need two independent reviewers or more states; derive them from the use case rather than adding ceremonial approvals.

Suppose a coding-practice platform shows learners AI-drafted feedback on their solutions: an explanation and a practice score from 0 to 5 that counts toward their progress record. A model drafts each item, a reviewer approves it, a publisher service publishes it, and learners can dispute a published score.

Item F-31 is the draft feedback for learner Lee's solution S-88:

Actors. Identity always comes from the authenticated session, never from a request field.

Check order. Each command goes through these steps, and the first failing step decides the outcome.

1. Authorize the actor for the event. Failure: refused as unauthorized; nothing changes; a denied-attempt log entry is written.
2. Publish only: if the idempotency key was already applied, return the stored outcome and change nothing.
3. Compare the expected revision with the stored one. Mismatch: conflict; nothing changes; the caller is shown the current state.
4. Check that the table allows the event from the current state. Otherwise: refused as an invalid transition.
5. Check preconditions in table order. Failure: refused with the specific reason, which is returned to the actor and logged; no audit event; the state does not change. If a reviewer cannot open required evidence, the queue lead is also alerted to reassign the item or restore access.
6. Apply atomically: new state, revision plus 1, one audit event.

Disputes. Only the learner a result belongs to may dispute it. A dispute is not an item transition: after step 1 it skips steps 2 to 6 and follows this rule instead. It may be filed against a Published, Restricted, or Withdrawn item. It leaves the item's state and revision unchanged, creates an Open case record with an owner from the dispute rota, writes one audit event linking the case to the item revision, and gives the learner the case ID and due time. A decision is due within 5 days, or before the contest results become final for a dispute filed during a contest window.

Contest load. During the weekly 8-hour contest window, disputes arrive at about 12 per hour and each re-review takes about 15 minutes. Two qualified reviewers are on shift, and contest results become final 2 hours after the window closes.