RAG & Data Pipelines · 3 to 4 hours
Debug a partially failed re-embedding backfill
Find why a partly failed re-embedding backfill left a help-center index missing, duplicating, and mixing vectors, then plan a repair and a cutover gate you could defend in a release review.
Hands-on debugging
4 h
Used in interview rounds:
Debugging & Evaluation (Hard)1
Step 1: The question
Suppose the help-center backfill from embedding configuration A to configuration B failed partway and was resumed. Using the incident setup, the query traces, and the job log and code in the diagnosis exercise, attribute every symptom to its cause and reconcile the live index against the manifest. Then write a containment and repair plan that fits the repair constraints, a cutover gate with rollback triggers that you declare before reading any results, the tests that would have caught each cause, and the quick fixes you would reject.
Constraints
- Use only the evidence in the incident setup, query traces, repair constraints, and the diagnosis exercise
- From the moment containment starts, every served result must come from one embedding configuration
- Access filters on the serving path stay as they are
- Budget 30 units; configuration A's query encoder retires at hour 48; keep the previous configuration ready for 24 hours after a cutover
Deliverables
- Cause table: each symptom, its cause, and the log line, trace rank, or record field that shows it
- Reconciliation against the manifest with the arithmetic
- Containment and repair plan, with targeted re-embedding and a full rebuild costed against the constraints, and one test per cause
- Cutover gate written before any results, with rollback triggers
- The quick fixes you would reject, and why
Hard questions are part of Premium: the study guide, reference answer, rubric, and exercises unlock with Premium.