AI Engineering
/
AI for Team and Tech Leads
/

Decide whether to build or buy an AI tool

AI for Team and Tech Leads · 2 to 3 hours

Decide whether to build or buy an AI tool

Choose between the AI feature in your finance suite, a specialist vendor, and an in-house build: apply hard gates first, read a pilot on your own invoices, compare three-year cost on one basis, and recommend one option with contract conditions and an exit plan.

Medium
Hands-on

Business case

3 h

Used in interview rounds:

Leadership Scenarios (Medium)

Step 1: The question

The CFO asks for one recommendation on automating invoice capture for the AP team: Option A, Option B, or Option C. Check each option against the five gates, read the pilot, compare three-year cost and cost per invoice on the same basis (correcting the internal tools team's estimate with the planning rules), and score the options that may be chosen. Recommend one option with the contract conditions you would negotiate, an exit plan, a fallback, and who decides by when.

Scenario: a fitness club chain's supplier invoices

The company, its figures, the options, their terms, and every price are fictional.

Suppose you are the finance lead at a chain of 40 fitness clubs in two EU countries. Your accounts payable (AP) team of 6 people processes about 5,000 supplier invoices a month, and volume is expected to stay about flat for three years. Invoices arrive by email as PDFs, scans, and phone photos: equipment servicing, cleaning, utilities, café stock, and self-employed personal trainers and class instructors, whose invoices carry their names, home addresses, bank details, and tax numbers. Staff key in 8 fields per invoice (supplier, invoice number, invoice date, net amount, VAT, total, club cost center, purchase order number) and check for duplicates before each payment run.

The CFO wants invoice capture automated and asks you for one recommendation. Three options are on the table:

  • Option A: the AI invoice-capture add-on in the finance suite the team already uses.
  • Option B: a specialist invoice-capture vendor.
  • Option C: an in-house build by the 3-person internal tools team on Provider M's model API (a service that runs Provider M's AI model on request and charges per use).

The company's gates. The CFO set them with IT security and counsel. An option that fails a gate cannot be chosen until a signed change fixes it. Every outside company that receives invoice data must meet G1, G2, and G3. Whichever tool the AP team uses, bought or built, must deliver G4 and G5. Anything built in-house must also pass the company's security review before go-live.

GateRequirement
G1 Data regionInvoice data is stored and processed in the EU
G2 No trainingThe contract states that the company's documents and data are not used to train or improve the provider's models
G3 Security reportA current SOC 2 Type II report or ISO/IEC 27001 certificate whose scope covers the service used
G4 Single sign-onStaff sign in with the company's single sign-on
G5 Export and deletionFull export of original files, extracted data, and the audit trail at any time, and deletion at exit

What each option offers.

GateOption A: suite add-onOption B: specialist vendor, standard contractOption C: in-house build, the team's plan
G1EU data centers, same as the suite; the add-on's AI runs in the same regionEU processing on all plans, on the vendor's own modelsProvider M's EU processing option, which the team plans to use
G2The suite's AI terms, part of the contract: customer data is not used for trainingStandard terms let the vendor use customer documents to improve its extraction models. A no-training data addendum is available only with the enterprise per-invoice priceProvider M's API business terms (the contract for API use): API data is not used for training
G3Current SOC 2 Type II report whose scope includes the AI add-onISO/IEC 27001 certificate and SOC 2 Type II report, both covering the extraction serviceProvider M: current SOC 2 Type II report covering its API. The app: company security review planned before go-live
G4Suite single sign-onSingle sign-on on all plansNot in the prototype; "can be added later if needed"
G5The suite's standard export of files, data, and audit trail; deletion at exitExport of originals, extracted data, and audit log at any time; deletion within 30 days of exitNot in the prototype (no audit trail, no export); "can be added later if needed"

Contract terms.

TermOption AOption BOption C
Length and noticeAnnual, with the suite; the add-on can be cancelled at renewal with 30 days' notice36-month minimum term; 180 days' notice before renewalNo contract beyond Provider M's pay-as-you-go terms
Price changesAt each renewal, no capAt each contract anniversary, no cap. The vendor raised list prices 15% at its last annual reviewProvider M may change prices and retire models; it gives notice before retiring a model

Pilot results, cost inputs, and planning rules

Own-data pilot. 400 invoices from last quarter, drawn across all 40 clubs. The sample includes 60 phone photos from trainers, 40 credit notes, and 12 known duplicates (invoices the AP team caught as resent last quarter; each original is also in the sample). Every option ran on the same 400 invoices. An AP specialist checked the 8 fields on each invoice against what was posted: 3,200 fields per option. An invoice needs a manual fix if any field is wrong or missing. An invoice with no result counts as needing a fix, with all 8 of its fields wrong.

OptionWrong or missing fields (of 3,200)Invoices needing a fix (of 400)Known duplicates flagged (of 12)Note
A112726
B482412Includes 3 password-protected PDFs that returned no result
C (prototype)64489The team built the prototype in the same two weeks, adjusting its prompts on these 400 invoices

Pilot bar, set before the pilot: at most 20% of invoices need a fix, and at least 6 of the 12 known duplicates are flagged.

Cost inputs. All prices are fictional. Three years = 36 months; 5,000 invoices a month = 180,000 invoices.

LineOption AOption BOption C, the team's estimate
Fee€37.80 per user per month, 6 usersPlatform fee €690 per monthHosting €85 per month
UsageIncluded€0.083 per invoice (standard); €0.107 per invoice with the no-training data addendum (enterprise price)Model usage €0.0137 per invoice at today's price
Implementation40 internal tools hoursOnboarding fee €4,750 plus 80 internal tools hoursBuild: 320 hours (two engineers at half time, 40 hours a week combined)
Internal running time2 hours a month4 hours a monthMaintenance: 4 hours a month
Time to go live3 weeks6 weeks8 weeks

Loaded rates (the full cost of an hour, including benefits and overhead): AP staff €46 an hour; internal tools engineers €95 an hour. All internal hours in the table are engineer hours.

Cost rules. Count the same lines for every option over 36 months: fees and usage, implementation, internal running time, and AP time to fix invoices. AP fix time per month = the option's pilot fix rate × 5,000 invoices × 5 minutes, at the AP rate. The one-minute check every invoice gets is the same for every option and is left out. Training time is similar across options and is left out.

Planning rules for in-house tools (company policy; the team's estimate did not apply them):

  • P1: Gate features are part of the build. Asked to size SSO, the audit trail, full export, and a duplicate check against 24 months of past invoices, the team estimated 160 more hours, at the same 40 hours a week.
  • P2: Maintenance is 25% of total build hours per year.
  • P3: Plan one model migration a year (Provider M retires models on its own schedule), 30 hours each, including a re-test on a fresh invoice sample.
  • P4: Plan model usage at twice today's per-invoice price, to cover a successor model priced differently and re-runs.

Scoring (weights fixed before the pilot): three-year total cost 50%, duplicate flag rate 30%, time to go live 20%. On each criterion the best option scores 10; the others score 10 × lowest cost ÷ their cost, 10 × their duplicate rate ÷ best rate, and 10 × fastest weeks ÷ their weeks. Score only options that may be chosen. Show criterion scores to two decimals and the weighted total to one decimal.

Constraints

  • Use only the figures in the scenario, the pilot, and the cost inputs. All prices are fictional.
  • An option that fails a gate as offered may be chosen only after a signed change; say which change and what it costs.
  • Three years at 5,000 invoices a month (180,000 invoices).
  • No legal advice: say what you would check with counsel.

Deliverables

  • Gate check for each option, as offered and after any change
  • What the pilot shows and what it cannot show
  • Three-year cost and cost per invoice on the same basis, with the build estimate corrected
  • Contract conditions and an exit plan
  • One recommendation with its conditions, a fallback, an owner, and dates

Step 2: Hints

Optional. Each hint gives a little more away.

Step 3: Work through it

Predict the outcome

Check the options against the gates

Use the scenario and the pilot results. For each row, decide which gates it fails as described, whether it meets the pilot bar, and whether it may be scored. Then reveal the answers.

OptionGates failedPilot barMay be scored?
Option A, suite add-on???
Option B, standard contract???
Option B with the no-training data addendum???
Option C as the team scoped it (320 hours)???
Option C with the P1 gate features???

Predict the outcome

Work the three-year cost and the scores

Use the cost inputs, cost rules, planning rules, and scoring rules. For each row, compute the three-year lines, the total, and the cost per invoice over 180,000 invoices (three decimals). The price-rise row is a risk check of the same option: raise all of its fees and usage by 15% at the start of year 2 and again at the start of year 3 (year 3 prices are 15% above year 2 prices). Then give the weighted score for each row that may be scored, or say why the row is not scored. Reveal the answers when you are done.

Option and basisFees and usageImplementationInternal timeAP fix timeThree-year totalPer invoiceWeighted score
Option A???????
Option B, standard contract???????
Option B with the no-training addendum???????
Option B with the addendum, fees and usage up 15% at each anniversary???????
Option C, the team's lines only???????
Option C, the team's lines plus AP fix time???????
Option C corrected with P1 to P4, plus AP fix time???????

Diagnose

Review the internal tools team's estimate

The internal tools team sent this note with its offer (summarized). Use the cost rules and planning rules in the cost inputs.

  1. "We built the prototype in two weeks, adjusting its prompts on the 400 pilot invoices until the results looked good."
  2. "Build: 320 hours, two of us at half time, so it is live in 8 weeks."
  3. "Model usage at today's price is €0.0137 per invoice, €68.50 a month."
  4. "Hosting is €85 a month, and maintenance about 4 hours a month."
  5. "SSO, the audit trail, full export, and checking duplicates against past invoices can be added later if needed."
  6. "Because we own it, there is no lock-in and no price increases, and over three years it costs far less than paying a vendor."
  1. List what the estimate leaves out or understates, and size each item in euros over three years.
  2. Say whether the prototype's pilot results can be compared with Options A and B, and what evidence you would ask for.
  3. Judge claim 6, then write two or three sentences to the internal tools team explaining how their offer was assessed.

Kept in this browser. Saving your score below keeps a copy with it.

0 / 5000

Step 4: Reference answer

Step 5: Score yourself

Score each dimension 0, 1, or 2, then save the score to your history. Readiness thresholds are editorial estimates.

Grades your written answer against this rubric with your own AI provider key from Settings. It is a second opinion: the score you save is still yours.

Hard requirements before any score

What earns a 2: Checks every option against each gate as offered, names the written evidence each gate needs, keeps an option that fails out of the score until a signed change fixes it, and applies the same gates to the outside provider behind a build.

Pilot on the team's own invoices

What earns a 2: Reads the pilot at the invoice level as well as the field level, counts invoices with no result as failures, checks each option against the bar set in advance, and says which results are not comparable or too small to settle and what fresh test would settle them.

Three-year cost on the same basis

What earns a 2: Puts every option on the same lines for three years, including internal hours at loaded rates and the time to fix errors, corrects the build estimate with the stated planning rules, and shows cost per invoice and the effect of a price rise.

Contract terms and exit plan

What earns a 2: Turns each risk found into a contract term or a test before signing (term, notice, price cap, data terms, export, and deletion) and describes a tested way back to manual work or another option.

Choice with conditions

What earns a 2: Makes one recommendation from the options that may be chosen, states the conditions it depends on and the fallback if they are refused, names the owner, the decision date, and the review date, and explains the decision to the people it affects, including the internal tools team.

0 of 10 · Scored 0 of 5 dimensions

Ready at 8 of 10, with no 0 in Hard requirements before any score or Three-year cost on the same basis or Choice with conditions

Kept in this browser until you sign up, then it moves to your account.

Complete this level when you can check options against hard gates before scoring, put a suite feature, a vendor, and a build on the same three-year cost basis from your own pilot, and recommend one option with the contract conditions, exit plan, and fallback it depends on.

Your saved scores

Practice saves and mock interview answers on this question, newest first.

No saved scores yet. Score every dimension, then save.

Step 6: Follow-up questions

An interviewer may ask these next.

What does a SOC 2 Type II report show that a Type I report does not?

Why is a hard requirement a gate rather than a weighted criterion?

Why count staff time to fix a tool's errors as part of its cost?

Does building in-house on a model API remove the vendor data questions?

Why fix the scoring weights before the pilot results arrive?

Step 7: Mark complete