Lexical and Dense Ranking

Lexical and Dense Ranking

A help center compares two ways of ranking its articles for a query: lexical matching on shared words, and dense ranking on supplied embedding vectors. It then scores each ranking against labeled relevant articles. Implement the five helpers below.

Each item is { id, text, vector }. ranked and relevant are arrays of IDs.

Rules

• Lexical score. Lowercase the query and the item text, then split each into runs of letters and digits ([a-z0-9]+). The score is the number of unique query words that also appear in the item text. For example, the query "Billing billing INVOICE" scores 2 against "Download a billing invoice.".
• Ranking. Higher score first. Break equal scores by ascending ID, using plain string comparison. Return at most k IDs, or every ID when there are fewer than k items. Items that score zero are still returned.
• Cosine. Throw an error when the vectors are empty or have different lengths, when any value is not finite (NaN or infinity), or when either vector has zero length: cosine is undefined for a zero vector.
• Metrics. A relevant ID counts once, even if it appears twice in the ranking. Precision always divides by k, even when the ranking holds fewer than k IDs. Recall divides by the number of relevant IDs; when there are none, recall is undefined, so throw an error. You may assume k is a positive integer and relevant has no duplicates.

Fixtures

Every document is eligible and labels are document-level.

With k = 2:

In Python, implement cosine, rank_lexical, rank_dense, precision_at_k, and recall_at_k. Items are dicts with "id", "text", and "vector" keys, and every error is a ValueError.