How to Remove Duplicate CSV Rows by Key Columns
Remove duplicate CSV or TSV records by comparing complete rows or selected columns. Retain the first or last matching record.
Decide which fields identify a duplicate
Full-row comparisons remove records only when every parsed field is equal. Key-column comparisons group records using selected fields. Use a stable ID or a combination such as customer ID and region when rows can describe different versions of the same record.
Choose which copy to retain
With an export sorted oldest first, Keep last occurrence retains the most recent matching record if that order is accurate. Keep first when the initial record is authoritative. The surviving records stay in the order of their original positions.
Work through a small example
Paste id,name,status followed by 001,Ana,draft and 001,Ana,active on separate rows. Select column 1 and keep last. The output preserves the header and keeps 001,Ana,active. Column positions start at one.
Review the result before import
Comparison is case-sensitive and keeps spaces and leading zeros. Quoted separators and multiline cells are read as part of their fields. The tool validates consistent row widths, reports how many records it removed, and exports the surviving fields with valid CSV quoting. Keep your original export for reconciliation.
Open the CSV Deduplicator to try the workflow in your browser.
Ready to Format Data Faster?
Experience instant, 100% private in-browser delimiter tools, SQL conversions, and file converters with zero server uploads.