AI Engineering
/
Tool Calling & Agents
/

Extend a tool adapter with a search tool

Tool Calling & Agents · 3 to 4 hours

Extend a tool adapter with a search tool

Extend the validated ticket adapter with a paginated search_tickets tool behind a tool registry under a 3-hour cap, hand in tests and a README, then review a sample submission as the interviewer would.

Medium
Hands-on

Take-home brief

4 h

Used in interview rounds:

Take-home & Review (Medium)

Step 1: The question

Complete the take-home in the brief within its 3-hour cap: add search_tickets to the validated adapter behind a tool registry, keep get_ticket unchanged, and hand in the code, the tool declaration, tests that run with one command and include every acceptance call, and a README. Then prepare for the review: write your three most important comments on the sample submission in priority order, and defend one trade-off you made under the cap.

The take-home brief

Suppose the assistant team at a fictional help-desk company sends you this take-home. Their support assistant already reads one ticket at a time through the validated get_ticket adapter (the starting point below). Support agents now ask questions such as "which of my open tickets mention login?", which needs a search.

Task. Add a second read-only tool, search_tickets, that follows the search contract below. Move dispatch into a tool registry so that each tool has its own argument checks and handler, and keep get_ticket exactly as it is. Start from your solution to the validated adapter exercise, or write get_ticket from the table below first (about 15 minutes).

Time. Cap the build at 3 hours; the review is a separate 45-minute call. If time runs out, stop: hand in what works with every test passing (mark an unfinished test as skipped, with the reason), and list in the README what you skipped and how you would finish it.

In scope: executeTool with both tools, the in-memory fixture store, the search_tickets declaration the model receives (name, description, and argument schema), tests, and a README.

Out of scope: write tools, the model call and the orchestration loop, asynchronous code, a database, network calls, relevance ranking, and dependencies beyond a test runner.

Language. TypeScript, JavaScript, or Python. Tool arguments and results keep the same names in every language (ticket_id, next_cursor) because the model reads them; the trusted context follows each language's convention, as in the starting adapter.

Hand in:

  1. The adapter source.
  2. The search_tickets declaration.
  3. Tests that run with one command and include every acceptance call (listed in the acceptance exercise).
  4. A README: how to run it, the contract you implemented, assumptions, decisions and their trade-offs, known limits, what you skipped, and next steps.

Review questions to expect in the 45-minute call:

  1. Walk through one call from executeTool to a handler. Where would a third tool plug in?
  2. Why is the cursor a ticket ID rather than an offset or a token that carries more state?
  3. Which test gives you the most confidence that one tenant cannot read another tenant's tickets?
  4. What did you skip under the cap, and why that?
  5. With a database behind the store, what changes in search and paging, and what could fail?
  6. What would you add with another day?

The starting point: get_ticket

The starting adapter is the validated get_ticket tool, called as executeTool(name, args, context, records) in JavaScript or TypeScript and execute_tool(name, args, context, records) in Python.

  • name and args come from the model and are untrusted. args is already parsed from JSON, but its shape is not guaranteed.
  • context is trusted and built from the authenticated session: { tenant, canReadTickets }, or in Python {"tenant": ..., "can_read_tickets": ...}.
  • records is a Map (Python dict) from "tenant:ticket_id", such as "alpha:TKT-104", to the stored ticket. A stored ticket can carry internal fields besides id, title, and status.

get_ticket runs these checks in order and returns at the first failure:

OrderCodeWhen
1unknown_toolname is not exactly a registered tool name (before the refactor, "get_ticket")
2invalid_argumentsargs is not a plain object: null, undefined, an array, a string, or a number
3invalid_argumentsargs has any key other than ticket_id, or no ticket_id
4invalid_argumentsticket_id is not a string of TKT- followed by exactly three digits from 0 to 9
5forbiddencanReadTickets is false
6not_foundrecords has no entry under context.tenant + ":" + ticket_id

Success is { ok: true, ticket: { id, title, status } }, a fresh object with exactly those three fields copied from the stored ticket. Every failure is { ok: false, code }, returned, not thrown. A tenant argument is rejected even when it matches the trusted tenant, and the adapter never modifies records.

The search_tickets contract

search_tickets lists the caller's tickets one page at a time. It is read-only and needs the same read permission as get_ticket.

Arguments. args must be a plain object. Every argument is optional, and {} is valid. A key that is present must hold a valid value, so null is invalid. Any other key, tenant included, is invalid.

ArgumentValid valuesWhen absent
status"open" or "closed"No status filter
queryA string of 1 to 40 characters, used as given (no trimming)No text filter
limitAn integer from 1 to 10. Booleans, fractions, and numeric strings are not integers5
cursorA ticket ID in the get_ticket formatStart from the lowest ID

Checks, in order, returning at the first failure: unknown_tool when the name is not registered; invalid_arguments when args is not a plain object, has a key outside the table, or holds an invalid value; forbidden when canReadTickets is false. A search never returns not_found.

Search, in this order:

  1. Take only the tickets whose store key starts with the trusted tenant followed by :.
  2. Sort them by id, ascending. Every ID has the same TKT- plus three digits format, so string order is numeric order.
  3. If status is given, keep the tickets with that status. If query is given, keep the tickets whose title, lowercased, contains the lowercased query. Only the title is searched.
  4. If cursor is given, keep the tickets whose id is greater than it. The cursor is a position: it does not have to be an existing ticket.
  5. Return the first limit tickets.

Result. { ok: true, tickets: [{ id, title, status }, ...], next_cursor }. Each ticket is a fresh copy with exactly those three fields. next_cursor is the id of the last returned ticket when more matching tickets remain after it, otherwise null. An empty tickets list is a successful answer.

The fixture store

The tests use one in-memory store with ten tickets in two tenants, listed in insertion order (not ID order). Each stored ticket reads as id, title, status, then any internal field. The first two rows are the starting adapter's fixtures.

Store keyStored ticket
alpha:TKT-104TKT-104, "Settings crash", open
beta:TKT-104TKT-104, "Private billing issue", closed
alpha:TKT-112TKT-112, "Settings page shows wrong time zone", open
alpha:TKT-101TKT-101, "Login fails after password reset", open, internal_note "Escalated by the on-call engineer"
beta:TKT-108TKT-108, "Login fails after SSO change", open, internal_note "Security review pending"
alpha:TKT-107TKT-107, "Login page slow on mobile", open
alpha:TKT-115TKT-115, "Password reset email delayed", closed
beta:TKT-103TKT-103, "Refund not received", open
alpha:TKT-102TKT-102, "CSV export times out", closed
alpha:TKT-110TKT-110, "Invoice PDF missing logo", closed, internal_note "Refund approved by finance"

Alpha has seven tickets and beta three. Their ID ranges interleave, and both tenants have a TKT-104. The acceptance calls use three trusted contexts: alpha with read permission, alpha without it, and beta with it.

Constraints

  • The build is capped at 3 hours; record what you skipped instead of going over
  • Tenant and read permission come only from the trusted context; every tool argument is untrusted input
  • get_ticket keeps its contract and behavior exactly
  • No write tools, database, network calls, ranking, or dependencies beyond a test runner
  • Results carry only id, title, and status, as fresh copies

Deliverables

  • Contract notes for search_tickets and the declaration the model receives
  • executeTool with both tools behind a registry, passing every acceptance call
  • Tests that run with one command: acceptance calls, failure paths, boundaries, a page walk, and tenant isolation
  • A README with how to run it, assumptions, decisions, limits, skipped work, and next steps
  • Three prioritized review comments on the sample submission and one defended trade-off

Step 2: Hints

Optional. Each hint gives a little more away.

Step 3: Work through it

Design exercise

Plan before you build

Before you write code, plan the take-home as you would in its first 15 minutes, using the brief, the starting point, the search contract, and the fixture store.

Requirements

  • Every acceptance call passes and get_ticket is unchanged
  • A new tool needs one registry entry and no change to other tools
  • No argument value can change whose tickets are read

Constraints

  • 3-hour build cap, including tests and the README
  • No dependencies beyond a test runner

Deliverables

  • Contract notes for the README, including decisions where the brief is silent
  • The registry shape and the order of checks
  • The search_tickets declaration the model receives
  • A test list grouped by what each group proves
  • How you will split the 3 hours, and what you will drop first if time runs short

Code exercise

Build the extension

This is the take-home itself, so there is no practice workspace: build it in your own editor, in TypeScript, JavaScript, or Python, with your own tests. Implement executeTool with get_ticket and search_tickets behind a registry, load the fixture store, and make the tests run with one command. The reference implementation behind the reveal runs every acceptance call and prints its result; open it only after your own suite passes.

Expected

Every acceptance call returns what the search contract defines, get_ticket behaves exactly as in the starting point, following next_cursor from any first page returns each of the caller's matching tickets once and in ID order, and no result carries a field other than id, title, and status.

Limitations

The store is an in-memory snapshot keyed by a string composite key, which is safe only because the tenant comes from trusted context and IDs are validated; a database would use a tenant-qualified query with an index on tenant and ID. Lowercasing is reliable for ASCII titles, and string length is counted in each language's own units, so non-ASCII queries can differ between languages. There is no model call, orchestration, concurrency, or write path.

Predict the outcome

Predict the acceptance results

Each row is one call against the fixture store. Predict the result before you reveal it: the error code, or the ticket IDs in order and next_cursor. Every returned ticket carries only id, title, and status. Your tests must include all thirteen calls.

CallTrusted contextTool and argumentsExpected result
A1alpha, can readget_ticket {"ticket_id": "TKT-104"}?
A2alpha, can readsearch_tickets {}?
A3alpha, can readsearch_tickets {"cursor": "TKT-110"}?
A4alpha, can readsearch_tickets {"status": "open", "query": "LOGIN"}?
A5alpha, can readsearch_tickets {"cursor": "TKT-108", "limit": 2}?
A6alpha, can readsearch_tickets {"cursor": "TKT-107", "limit": 3}?
A7alpha, can readsearch_tickets {"status": "closed", "limit": 10}?
A8alpha, can readsearch_tickets {"query": "refund"}?
A9alpha, can readsearch_tickets {"limit": 11}?
A10alpha, can readsearch_tickets {"status": "open", "tenant": "alpha"}?
A11alpha, cannot readsearch_tickets {}?
A12beta, can readsearch_tickets {"cursor": "TKT-102", "limit": 2}?
A13alpha, can readlist_tickets {}?

Diagnose

Review a sample submission

A candidate handed in the submission below, and its tests pass with npm test. get_ticket is unchanged from the starting adapter and not shown. The excerpts are the registry and search_tickets, the test file, and the README.

// tools.ts (excerpt)
const TOOLS = new Map<string, (args: any, context: Context, records: Records) => object>([
  ["get_ticket", getTicket], // unchanged from the starting adapter
  ["search_tickets", searchTickets],
]);

export function executeTool(name: string, args: any, context: Context, records: Records) {
  const tool = TOOLS.get(name);
  if (!tool) return { ok: false, code: "unknown_tool" };
  return tool(args, context, records);
}

function searchTickets(args: any, context: Context, records: Records) {
  if (typeof args !== "object" || args === null || Array.isArray(args)) {
    return { ok: false, code: "invalid_arguments" };
  }
  if (args.status !== undefined && args.status !== "open" && args.status !== "closed") {
    return { ok: false, code: "invalid_arguments" };
  }
  if (!context.canReadTickets) return { ok: false, code: "forbidden" };

  // The cursor is opaque, so the model cannot tamper with it.
  let tenant = context.tenant;
  let after = "";
  if (args.cursor) {
    const decoded = JSON.parse(Buffer.from(args.cursor, "base64").toString("utf8"));
    tenant = decoded.tenant;
    after = decoded.after;
  }
  const limit = args.limit ?? 5;
  const query = (args.query ?? "").toLowerCase();

  const matches = [...records.entries()]
    .filter(([key]) => key.startsWith(tenant + ":"))
    .map(([, ticket]) => ticket)
    .filter((t) => args.status === undefined || t.status === args.status)
    .filter((t) => t.title.toLowerCase().includes(query))
    .filter((t) => t.id > after)
    .sort((a, b) => (a.id < b.id ? -1 : 1));

  const page = matches.slice(0, limit);
  const more = matches.length > limit;
  const last = page[page.length - 1];
  return {
    ok: true,
    tickets: page,
    next_cursor: more ? Buffer.from(JSON.stringify({ tenant, after: last.id })).toString("base64") : null,
  };
}
// search.test.ts
import { executeTool } from "./tools";
import { records } from "./fixtures";

const alpha = { tenant: "alpha", canReadTickets: true };

test("returns the first page", () => {
  const result: any = executeTool("search_tickets", {}, alpha, records);
  expect(result.tickets).toHaveLength(5);
});

test("returns the next page", () => {
  const first: any = executeTool("search_tickets", {}, alpha, records);
  const second: any = executeTool("search_tickets", { cursor: first.next_cursor }, alpha, records);
  expect(second.tickets).toHaveLength(2);
});

test("filters by status", () => {
  const result: any = executeTool("search_tickets", { status: "closed" }, alpha, records);
  expect(result.tickets.every((t: any) => t.status === "closed")).toBe(true);
});

README.md
Ticket tools. Run: npm install, then npm test.

  • Added search_tickets with a status filter, a text search, and pagination.
  • The cursor is opaque (base64), so the model cannot tamper with it.
  • Dispatch now goes through a tool map.

Review the submission as the take-home reviewer. Write your three most important comments in priority order. For each, name the line or behavior, give an input from the fixture store that shows the problem, say why it matters, and suggest a fix with the test that would have caught it. Then list anything else you noticed in one line each, and name one thing the submission does well.

Kept in this browser. Saving your score below keeps a copy with it.

0 / 5000

Step 4: Reference answer

Step 5: Score yourself

Score each dimension 0, 1, or 2, then save the score to your history. Readiness thresholds are editorial estimates.

Grades your written answer against this rubric with your own AI provider key from Settings. It is a second opinion: the score you save is still yours.

Contract and scope

What earns a 2: Documents every argument rule, default, and bound and the order of checks, writes a declaration whose schema matches the code, and keeps the build inside the brief's scope with skipped work recorded.

Tenant-safe pagination

What earns a 2: Takes scope only from the trusted context before applying the cursor, validates the cursor as a position, sets next_cursor only when matches remain, and shows with a second tenant that shares IDs that no other tenant's ticket can appear.

Registry and regression

What earns a 2: Dispatches through an exact-name registry with per-tool parsers and one shared order of checks, and proves get_ticket unchanged with its original fixtures.

Tests and README

What earns a 2: Runs with one command and tests a failing value per rule, both sides of each bound, a full page walk, and isolation, with a README whose assumptions, decisions, limits, skipped work, and next steps match the code.

Review and defense

What earns a 2: Orders review comments by impact, backs each with a fixture input, the reason it matters, and a fix with its test, and defends one trade-off made under the cap with what would change it.

0 of 10 · Scored 0 of 5 dimensions

Ready at 8 of 10, with no 0 in Tenant-safe pagination or Tests and README or Review and defense

Kept in this browser until you sign up, then it moves to your account.

Complete this level when your build passes every acceptance call within the cap, a reviewer could run and judge it from your README alone, and you can walk through your highest-priority comment on the sample submission and one trade-off you made.

Your saved scores

Practice saves and mock interview answers on this question, newest first.

No saved scores yet. Score every dimension, then save.

Step 6: Follow-up questions

An interviewer may ask these next.

Why is a cursor untrusted input even though your server created it?

Why can offset pagination repeat or skip items between pages while keyset pagination does not?

In JavaScript, why can tools[name] dispatch something that is not a tool?

When should a search return a null continuation value?

What belongs in a take-home README besides how to run it?

Step 7: Mark complete