Community JavaScript Snippet

Dedupe by Key With Last-Write-Wins (JS)

dedupeBy(rows, 'id') is the function I copy into every ETL script. Last-write-wins is the right policy 90% of the time; this version also exposes a configurable resolver for the other 10%.

Dedupe by Key With Last-Write-Wins (JS)

dedupeBy(rows, 'id') is the function I copy into every ETL script. Last-write-wins is the right policy 90% of the time; this version also exposes a configurable resolver for the other 10%.

JavaScript
Frontend
3 snippets
hash-map
code-template
data-pipeline
camilarao

By @camilarao

April 15, 2026

·

Updated May 20, 2026

561 views

5

4.7 (8)

Six lines, but the trick is the choice of Map over a plain object. Map.set overwrites in place yet preserves the original insertion key in iteration order, which is exactly the last-write-wins-but-keep-arrival-shape semantics that ETL scripts want. The keyOf argument accepts either a string ('id') or a function ((r) => r.user.id) so it handles nested keys without forcing the caller to pre-flatten. I have shipped this verbatim in three different services; the named-function version is enough for almost every dedupe case.