Tool Calling & Agents · 3 to 4 hours
Extend a tool adapter with a search tool
Extend the validated ticket adapter with a paginated search_tickets tool behind a tool registry under a 3-hour cap, hand in tests and a README, then review a sample submission as the interviewer would.
Take-home brief
4 h
Used in interview rounds:
Take-home & Review (Medium)1
Step 1: The question
Complete the take-home in the brief within its 3-hour cap: add search_tickets to the validated adapter behind a tool registry, keep get_ticket unchanged, and hand in the code, the tool declaration, tests that run with one command and include every acceptance call, and a README. Then prepare for the review: write your three most important comments on the sample submission in priority order, and defend one trade-off you made under the cap.
The take-home brief
Suppose the assistant team at a fictional help-desk company sends you this take-home. Their support assistant already reads one ticket at a time through the validated get_ticket adapter (the starting point below). Support agents now ask questions such as "which of my open tickets mention login?", which needs a search.
Task. Add a second read-only tool, search_tickets, that follows the search contract below. Move dispatch into a tool registry so that each tool has its own argument checks and handler, and keep get_ticket exactly as it is. Start from your solution to the validated adapter exercise, or write get_ticket from the table below first (about 15 minutes).
Time. Cap the build at 3 hours; the review is a separate 45-minute call. If time runs out, stop: hand in what works with every test passing (mark an unfinished test as skipped, with the reason), and list in the README what you skipped and how you would finish it.
In scope: executeTool with both tools, the in-memory fixture store, the search_tickets declaration the model receives (name, description, and argument schema), tests, and a README.
Out of scope: write tools, the model call and the orchestration loop, asynchronous code, a database, network calls, relevance ranking, and dependencies beyond a test runner.
Language. TypeScript, JavaScript, or Python. Tool arguments and results keep the same names in every language (ticket_id, next_cursor) because the model reads them; the trusted context follows each language's convention, as in the starting adapter.
Hand in:
- The adapter source.
- The
search_ticketsdeclaration. - Tests that run with one command and include every acceptance call (listed in the acceptance exercise).
- A README: how to run it, the contract you implemented, assumptions, decisions and their trade-offs, known limits, what you skipped, and next steps.
Review questions to expect in the 45-minute call:
- Walk through one call from
executeToolto a handler. Where would a third tool plug in? - Why is the cursor a ticket ID rather than an offset or a token that carries more state?
- Which test gives you the most confidence that one tenant cannot read another tenant's tickets?
- What did you skip under the cap, and why that?
- With a database behind the store, what changes in search and paging, and what could fail?
- What would you add with another day?
The starting point: get_ticket
The starting adapter is the validated get_ticket tool, called as executeTool(name, args, context, records) in JavaScript or TypeScript and execute_tool(name, args, context, records) in Python.
nameandargscome from the model and are untrusted.argsis already parsed from JSON, but its shape is not guaranteed.contextis trusted and built from the authenticated session:{ tenant, canReadTickets }, or in Python{"tenant": ..., "can_read_tickets": ...}.recordsis aMap(Pythondict) from"tenant:ticket_id", such as"alpha:TKT-104", to the stored ticket. A stored ticket can carry internal fields besidesid,title, andstatus.
get_ticket runs these checks in order and returns at the first failure:
| Order | Code | When |
|---|---|---|
| 1 | unknown_tool | name is not exactly a registered tool name (before the refactor, "get_ticket") |
| 2 | invalid_arguments | args is not a plain object: null, undefined, an array, a string, or a number |
| 3 | invalid_arguments | args has any key other than ticket_id, or no ticket_id |
| 4 | invalid_arguments | ticket_id is not a string of TKT- followed by exactly three digits from 0 to 9 |
| 5 | forbidden | canReadTickets is false |
| 6 | not_found | records has no entry under context.tenant + ":" + ticket_id |
Success is { ok: true, ticket: { id, title, status } }, a fresh object with exactly those three fields copied from the stored ticket. Every failure is { ok: false, code }, returned, not thrown. A tenant argument is rejected even when it matches the trusted tenant, and the adapter never modifies records.
The search_tickets contract
search_tickets lists the caller's tickets one page at a time. It is read-only and needs the same read permission as get_ticket.
Arguments. args must be a plain object. Every argument is optional, and {} is valid. A key that is present must hold a valid value, so null is invalid. Any other key, tenant included, is invalid.
| Argument | Valid values | When absent |
|---|---|---|
status | "open" or "closed" | No status filter |
query | A string of 1 to 40 characters, used as given (no trimming) | No text filter |
limit | An integer from 1 to 10. Booleans, fractions, and numeric strings are not integers | 5 |
cursor | A ticket ID in the get_ticket format | Start from the lowest ID |
Checks, in order, returning at the first failure: unknown_tool when the name is not registered; invalid_arguments when args is not a plain object, has a key outside the table, or holds an invalid value; forbidden when canReadTickets is false. A search never returns not_found.
Search, in this order:
- Take only the tickets whose store key starts with the trusted tenant followed by
:. - Sort them by
id, ascending. Every ID has the sameTKT-plus three digits format, so string order is numeric order. - If
statusis given, keep the tickets with that status. Ifqueryis given, keep the tickets whose title, lowercased, contains the lowercasedquery. Only the title is searched. - If
cursoris given, keep the tickets whoseidis greater than it. The cursor is a position: it does not have to be an existing ticket. - Return the first
limittickets.
Result. { ok: true, tickets: [{ id, title, status }, ...], next_cursor }. Each ticket is a fresh copy with exactly those three fields. next_cursor is the id of the last returned ticket when more matching tickets remain after it, otherwise null. An empty tickets list is a successful answer.
The fixture store
The tests use one in-memory store with ten tickets in two tenants, listed in insertion order (not ID order). Each stored ticket reads as id, title, status, then any internal field. The first two rows are the starting adapter's fixtures.
| Store key | Stored ticket |
|---|---|
| alpha:TKT-104 | TKT-104, "Settings crash", open |
| beta:TKT-104 | TKT-104, "Private billing issue", closed |
| alpha:TKT-112 | TKT-112, "Settings page shows wrong time zone", open |
| alpha:TKT-101 | TKT-101, "Login fails after password reset", open, internal_note "Escalated by the on-call engineer" |
| beta:TKT-108 | TKT-108, "Login fails after SSO change", open, internal_note "Security review pending" |
| alpha:TKT-107 | TKT-107, "Login page slow on mobile", open |
| alpha:TKT-115 | TKT-115, "Password reset email delayed", closed |
| beta:TKT-103 | TKT-103, "Refund not received", open |
| alpha:TKT-102 | TKT-102, "CSV export times out", closed |
| alpha:TKT-110 | TKT-110, "Invoice PDF missing logo", closed, internal_note "Refund approved by finance" |
Alpha has seven tickets and beta three. Their ID ranges interleave, and both tenants have a TKT-104. The acceptance calls use three trusted contexts: alpha with read permission, alpha without it, and beta with it.
Constraints
- The build is capped at 3 hours; record what you skipped instead of going over
- Tenant and read permission come only from the trusted context; every tool argument is untrusted input
- get_ticket keeps its contract and behavior exactly
- No write tools, database, network calls, ranking, or dependencies beyond a test runner
- Results carry only id, title, and status, as fresh copies
Deliverables
- Contract notes for search_tickets and the declaration the model receives
- executeTool with both tools behind a registry, passing every acceptance call
- Tests that run with one command: acceptance calls, failure paths, boundaries, a page walk, and tenant isolation
- A README with how to run it, assumptions, decisions, limits, skipped work, and next steps
- Three prioritized review comments on the sample submission and one defended trade-off
2
Step 2: Hints
Optional. Each hint gives a little more away.
3
Step 3: Work through it
Design exercise
Plan before you build
Before you write code, plan the take-home as you would in its first 15 minutes, using the brief, the starting point, the search contract, and the fixture store.
Requirements
- Every acceptance call passes and get_ticket is unchanged
- A new tool needs one registry entry and no change to other tools
- No argument value can change whose tickets are read
Constraints
- 3-hour build cap, including tests and the README
- No dependencies beyond a test runner
Deliverables
- Contract notes for the README, including decisions where the brief is silent
- The registry shape and the order of checks
- The search_tickets declaration the model receives
- A test list grouped by what each group proves
- How you will split the 3 hours, and what you will drop first if time runs short
Code exercise
Build the extension
This is the take-home itself, so there is no practice workspace: build it in your own editor, in TypeScript, JavaScript, or Python, with your own tests. Implement executeTool with get_ticket and search_tickets behind a registry, load the fixture store, and make the tests run with one command. The reference implementation behind the reveal runs every acceptance call and prints its result; open it only after your own suite passes.
Expected
Every acceptance call returns what the search contract defines, get_ticket behaves exactly as in the starting point, following next_cursor from any first page returns each of the caller's matching tickets once and in ID order, and no result carries a field other than id, title, and status.
Limitations
The store is an in-memory snapshot keyed by a string composite key, which is safe only because the tenant comes from trusted context and IDs are validated; a database would use a tenant-qualified query with an index on tenant and ID. Lowercasing is reliable for ASCII titles, and string length is counted in each language's own units, so non-ASCII queries can differ between languages. There is no model call, orchestration, concurrency, or write path.
Predict the outcome
Predict the acceptance results
Each row is one call against the fixture store. Predict the result before you reveal it: the error code, or the ticket IDs in order and next_cursor. Every returned ticket carries only id, title, and status. Your tests must include all thirteen calls.
| Call | Trusted context | Tool and arguments | Expected result |
|---|---|---|---|
| A1 | alpha, can read | get_ticket {"ticket_id": "TKT-104"} | ? |
| A2 | alpha, can read | search_tickets {} | ? |
| A3 | alpha, can read | search_tickets {"cursor": "TKT-110"} | ? |
| A4 | alpha, can read | search_tickets {"status": "open", "query": "LOGIN"} | ? |
| A5 | alpha, can read | search_tickets {"cursor": "TKT-108", "limit": 2} | ? |
| A6 | alpha, can read | search_tickets {"cursor": "TKT-107", "limit": 3} | ? |
| A7 | alpha, can read | search_tickets {"status": "closed", "limit": 10} | ? |
| A8 | alpha, can read | search_tickets {"query": "refund"} | ? |
| A9 | alpha, can read | search_tickets {"limit": 11} | ? |
| A10 | alpha, can read | search_tickets {"status": "open", "tenant": "alpha"} | ? |
| A11 | alpha, cannot read | search_tickets {} | ? |
| A12 | beta, can read | search_tickets {"cursor": "TKT-102", "limit": 2} | ? |
| A13 | alpha, can read | list_tickets {} | ? |
Diagnose
Review a sample submission
A candidate handed in the submission below, and its tests pass with npm test. get_ticket is unchanged from the starting adapter and not shown. The excerpts are the registry and search_tickets, the test file, and the README.
// tools.ts (excerpt)
const TOOLS = new Map<string, (args: any, context: Context, records: Records) => object>([
["get_ticket", getTicket], // unchanged from the starting adapter
["search_tickets", searchTickets],
]);
export function executeTool(name: string, args: any, context: Context, records: Records) {
const tool = TOOLS.get(name);
if (!tool) return { ok: false, code: "unknown_tool" };
return tool(args, context, records);
}
function searchTickets(args: any, context: Context, records: Records) {
if (typeof args !== "object" || args === null || Array.isArray(args)) {
return { ok: false, code: "invalid_arguments" };
}
if (args.status !== undefined && args.status !== "open" && args.status !== "closed") {
return { ok: false, code: "invalid_arguments" };
}
if (!context.canReadTickets) return { ok: false, code: "forbidden" };
// The cursor is opaque, so the model cannot tamper with it.
let tenant = context.tenant;
let after = "";
if (args.cursor) {
const decoded = JSON.parse(Buffer.from(args.cursor, "base64").toString("utf8"));
tenant = decoded.tenant;
after = decoded.after;
}
const limit = args.limit ?? 5;
const query = (args.query ?? "").toLowerCase();
const matches = [...records.entries()]
.filter(([key]) => key.startsWith(tenant + ":"))
.map(([, ticket]) => ticket)
.filter((t) => args.status === undefined || t.status === args.status)
.filter((t) => t.title.toLowerCase().includes(query))
.filter((t) => t.id > after)
.sort((a, b) => (a.id < b.id ? -1 : 1));
const page = matches.slice(0, limit);
const more = matches.length > limit;
const last = page[page.length - 1];
return {
ok: true,
tickets: page,
next_cursor: more ? Buffer.from(JSON.stringify({ tenant, after: last.id })).toString("base64") : null,
};
}// search.test.ts
import { executeTool } from "./tools";
import { records } from "./fixtures";
const alpha = { tenant: "alpha", canReadTickets: true };
test("returns the first page", () => {
const result: any = executeTool("search_tickets", {}, alpha, records);
expect(result.tickets).toHaveLength(5);
});
test("returns the next page", () => {
const first: any = executeTool("search_tickets", {}, alpha, records);
const second: any = executeTool("search_tickets", { cursor: first.next_cursor }, alpha, records);
expect(second.tickets).toHaveLength(2);
});
test("filters by status", () => {
const result: any = executeTool("search_tickets", { status: "closed" }, alpha, records);
expect(result.tickets.every((t: any) => t.status === "closed")).toBe(true);
});README.md
Ticket tools. Run: npm install, then npm test.
- Added search_tickets with a status filter, a text search, and pagination.
- The cursor is opaque (base64), so the model cannot tamper with it.
- Dispatch now goes through a tool map.
Review the submission as the take-home reviewer. Write your three most important comments in priority order. For each, name the line or behavior, give an input from the fixture store that shows the problem, say why it matters, and suggest a fix with the test that would have caught it. Then list anything else you noticed in one line each, and name one thing the submission does well.
Kept in this browser. Saving your score below keeps a copy with it.
0 / 5000
4
Step 4: Reference answer
5
Step 5: Score yourself
Score each dimension 0, 1, or 2, then save the score to your history. Readiness thresholds are editorial estimates.
Grades your written answer against this rubric with your own AI provider key from Settings. It is a second opinion: the score you save is still yours.
What earns a 2: Documents every argument rule, default, and bound and the order of checks, writes a declaration whose schema matches the code, and keeps the build inside the brief's scope with skipped work recorded.
What earns a 2: Takes scope only from the trusted context before applying the cursor, validates the cursor as a position, sets next_cursor only when matches remain, and shows with a second tenant that shares IDs that no other tenant's ticket can appear.
What earns a 2: Dispatches through an exact-name registry with per-tool parsers and one shared order of checks, and proves get_ticket unchanged with its original fixtures.
What earns a 2: Runs with one command and tests a failing value per rule, both sides of each bound, a full page walk, and isolation, with a README whose assumptions, decisions, limits, skipped work, and next steps match the code.
What earns a 2: Orders review comments by impact, backs each with a fixture input, the reason it matters, and a fix with its test, and defends one trade-off made under the cap with what would change it.
0 of 10 · Scored 0 of 5 dimensions
Ready at 8 of 10, with no 0 in Tenant-safe pagination or Tests and README or Review and defense
Kept in this browser until you sign up, then it moves to your account.
Complete this level when your build passes every acceptance call within the cap, a reviewer could run and judge it from your README alone, and you can walk through your highest-priority comment on the sample submission and one trade-off you made.
Your saved scores
Practice saves and mock interview answers on this question, newest first.
No saved scores yet. Score every dimension, then save.
6
Step 6: Follow-up questions
An interviewer may ask these next.
Why is a cursor untrusted input even though your server created it?
Why can offset pagination repeat or skip items between pages while keyset pagination does not?
In JavaScript, why can tools[name] dispatch something that is not a tool?
When should a search return a null continuation value?
What belongs in a take-home README besides how to run it?
7
