AI for Team and Tech Leads · 90 minutes
Choose which team workflows to hand to AI
Judge six workflows at a fictional home-services company by volume, error cost, checking effort, data, and the people affected, set an automation level and a human checkpoint for each, and choose the first pilot.
Scenario
1.5 h
Used in interview rounds:
Leadership Scenarios (Easy)1
Step 1: The question
You are the operations lead at the home-services company. Using the workflow tables and the facts below them, recommend an automation level for each of the six workflows and name its human checkpoint: who checks, what they look at, and when. Show the weekly hours each workflow takes today and what a person would still spend checking, and say which criteria decided each level. Then choose the first pilot and say how you would judge it, when you would stop it, and when you would decide.
Five questions for every workflow
Ask the same five questions of every workflow. They are the criteria for every decision in this guide.
| Question | Why it matters |
|---|---|
| How many items a week, and how many minutes each? | Sets how much time is at stake |
| What does a wrong output cost, and can it be undone before anyone is harmed? | Sets how close a person must stay to the work |
| How long does a person need to check one output, and can they tell a right output from a wrong one? | Decides whether checking saves time and whether the check is real |
| Which data does the work put into the tool, and is the tool approved for it? | Decides whether the workflow can start now |
| Who bears a wrong output: staff, customers, or a person a decision is about? | Raises the bar when customers or decisions about people are involved |
Volume tells you how much is at stake, not how far to automate. The level comes from the other four questions.
NIST's AI Risk Management Framework (voluntary guidance, not law) asks teams to document the expected benefits of an AI system and the costs of its errors (Map 3.1 and Map 3.2), and to define how people oversee it (Map 3.5).
Five levels of automation
An automation level says how much of the work the AI does and where a person stays involved. This guide uses five levels. The examples come from other settings.
| Level | What the AI does | Where the person is | Example |
|---|---|---|---|
| 1. No AI | Nothing | A person does all the work | Negotiating a contract renewal with a key client |
| 2. AI suggests, a person does the work | Gathers information or suggests options; produces nothing final | A person does the work and decides | Lists three possible meeting times; the organizer picks one and sends the invite |
| 3. AI drafts, a person approves each item | Produces a draft | A person checks, edits, and approves every item before it is used | Drafts meeting minutes; the chair corrects and sends them |
| 4. AI acts, a person audits a sample and handles exceptions | Produces final outputs | Rules send unusual items (exceptions) to a person, and a person checks a regular random sample | Routes incoming email to the right shared inbox; mail it cannot place goes to a person, and a lead checks 20 a week |
| 5. AI acts alone, with monitoring | Produces final outputs | A person watches overall measures, such as error reports and complaints, and can stop it | Removes duplicate contacts from a mailing list, with a weekly count someone reviews |
- One workflow can have steps at different levels. For example, a recruiting team might let the AI check that each job advert has the required sections at Level 4 while every hiring decision stays with people at Level 1.
- Levels are earned. Start lower, raise the level with evidence, and drop a level when errors rise or the work changes.
- The idea comes from automation research: Parasuraman, Sheridan, and Wickens (2000) show that each function of a system (gathering information, analyzing it, deciding, acting) can sit at its own level, from fully manual to fully automatic. These five levels are a simplified version for team decisions, not their scale.
Scenario: six workflows at a home-services company
Suppose a regional home-services company does plumbing, heating, and electrical work. It has about 120 technicians and a 14-person office: 8 on the booking desk, 2 accounts clerks, 1 customer care lead, and 3 estimators. You are the operations lead. The company is trialing one business AI assistant, Option A. All figures are synthetic.
Table 1. Work and time
| ID | Workflow | Weekly volume | Minutes per item today | Done today by | Minutes to check one AI output |
|---|---|---|---|---|---|
| W1 | Turn technician visit notes into customer job reports | 1,500 | 8 | The technician, on a tablet after each visit | 2, by the technician who made the visit |
| W2 | Draft replies to booking and rescheduling emails | 720 | 5 | Booking desk (8 staff) | 1.5, with the booking calendar open |
| W3 | Code supplier invoices to a job and a cost category | 600 | 3 | 2 accounts clerks | 1, against the purchase order |
| W4 | Approve or deny warranty claims | 30 | 25 | The customer care lead | 20: checking a decision needs the same evidence as making it |
| W5 | Translate incoming customer messages into English | 60 | 12 | The one bilingual booking-desk member, or a phone interpreting service | Most staff cannot check. The bilingual member needs 2, but only for the 2 languages they read |
| W6 | Quote prices for new jobs from customer photos | 180 | 20 | 3 estimators | 20: the estimator must do the full assessment |
Table 2. Risk, data, and people
| ID | What a wrong output costs | Can it be undone? | Data involved | Who is affected |
|---|---|---|---|---|
| W1 | The customer gets a wrong record of work, parts, or safety advice. The report also backs invoices and warranty claims | Partly: a corrected report can be sent, but the customer may already have acted on it | Customer name and address, photos inside the home, appliance details | Customers; 120 technicians whose routine changes |
| W2 | A wrong date, time, or address means a missed visit and a wasted technician trip | Yes, if caught before the visit | Customer name, address, phone number, availability | Customers; booking desk |
| W3 | A wrong job or category misstates job profit in the monthly report | Yes: a correcting entry before the month-end close | Supplier names, amounts, job numbers | Accounts clerks; managers who read job profit reports |
| W4 | A wrong denial refuses a customer what they may be entitled to; a wrong approval costs a free return visit | A denial can be overturned on appeal, but trust suffers and complaints escalate | Customer details, job and payment history, photos of the home | The customer: it is a decision about them |
| W5 | Staff misread a request and answer the wrong question | Usually: staff can ask the customer to clarify | Customer name, address, message content | Customers who write in other languages |
| W6 | An accepted quote is a commitment: too low loses money, too high loses the job. Photos miss hidden conditions such as pipework behind walls or old wiring | No: changing an accepted price is hard and often loses the customer | Photos inside customers' homes (they can show people and documents), contact details | Customers; estimators; the company's margin |
Facts
- Option A: the IT lead has approved it for business documents such as supplier invoices and purchase orders (a purchase order is the company's record of what it ordered). Customer personal data (names, addresses, messages, photos of homes) may go into it only after a data-use review, expected to finish in about 6 weeks.
- About 1 in 10 supplier invoices (60 a week) match no purchase order and need a clerk to investigate. Payment approval is a separate step and is not changing.
- Today's month-end check finds about 3 miscoded invoices per 100. The month-end close is when the month's accounts are finalized; a correction is simple before it and harder after it.
- The 60 messages a week in other languages arrive in 5 languages. About half (30 a week) are in the 2 languages the bilingual booking-desk member reads.
- Emergencies (smell of gas, sparks, flooding) are handled by phone. The website and the email auto-reply tell customers to call the emergency line.
- The operations lead can run one pilot at a time this quarter.
Constraints
- Use the five automation levels from the guide. A workflow may be split into steps at different levels.
- Work from the tables and label any figure you estimate.
- Staffing and payment approval do not change this quarter, and only one pilot can run at a time.
- This is an operating decision, not legal advice: flag anything that needs counsel.
Deliverables
- Weekly hours for each workflow today and with a person checking each AI output
- For each workflow, the error cost, whether it can be undone, and the checking effort that set its level
- A level and a checkpoint (who, what, when) for each workflow
- The data each workflow puts into Option A, and who a wrong output affects
- The first pilot: why it, its starting level, success and stop measures against a baseline, an owner, and a decision date
2
Step 2: Hints
Optional. Each hint gives a little more away.
3
Step 3: Your answer
Kept in this browser. Saving your score below keeps a copy with it.
0 / 5000
4
Step 4: Reference answer
5
Step 5: Score yourself
Score each dimension 0, 1, or 2, then save the score to your history. Readiness thresholds are editorial estimates.
Grades your written answer against this rubric with your own AI provider key from Settings. It is a second opinion: the score you save is still yours.
What earns a 2: Computes weekly hours today and with a person checking each AI output for every workflow from the table's figures, showing the arithmetic.
What earns a 2: For each workflow, weighs what a wrong output costs, whether it can be undone, how long a check takes, and whether the checker can tell right from wrong.
What earns a 2: Assigns every workflow a level with reasons and a checkpoint that names who checks, what they look at, and when, with exception rules wherever a sample replaces checking every item.
What earns a 2: Matches each workflow's data against what the tool is approved for and names who bears a wrong output.
What earns a 2: Chooses one pilot with reasons drawn from the criteria, a starting level, quality and time measures against a baseline measured the same way, a stop rule, an owner, and a decision date.
0 of 10 · Scored 0 of 5 dimensions
Ready at 8 of 10, with no 0 in Automation level and human checkpoint or First-pilot decision
Kept in this browser until you sign up, then it moves to your account.
Mark this level complete when you can place a team's tasks on the automation scale from error cost, checking effort, data, and the people affected, name who checks each one, and pick a first pilot with its measures written in advance.
Your saved scores
Practice saves and mock interview answers on this question, newest first.
No saved scores yet. Score every dimension, then save.
6
Step 6: Follow-up questions
An interviewer may ask these next.
Why can handing a task to AI save no time at all?
What makes a human checkpoint meaningful?
When does a sampled audit fit better than approving every item?
Does high volume justify a higher level of automation?
Why start a pilot one level below the target?
7