Remove Duplicate Lines
Paste a list and strip out repeats, with counts and the order kept. 100% free, no signup. Everything runs in your browser.
Removing duplicates is one of those jobs that sounds trivial until you try it on real data. Excel's own remove duplicates only matches exactly, so Anna@Example.com and anna@example.com both survive, and so does any value with a trailing space you cannot see. This tool lets you decide what counts as the same thing: exact, ignoring case, ignoring spaces at the ends, or both. It keeps the original order by default, can sort instead, and can show how many times each value appeared, which is often the more useful answer. It can also flip the question and show you only the duplicates, or only the values that appear exactly once. Everything runs in your browser.
How to use
- Paste your list into the box, one item per line. Copying a column straight out of Excel gives you exactly that.
- Choose how strictly to match. Ignore case and spaces is the right choice for email addresses, names and reference codes, because that is where invisible differences hide.
- Choose what to keep. The first time each value appears is the normal answer. Only the repeated lines is how you audit a list rather than clean it.
- Tick Show how many times each appeared if you want counts. The result comes out tab separated, so it pastes into two spreadsheet columns.
- Copy the result or download it. The three figures tell you how many lines went in, how many came out and how many were removed.
Why use our remove duplicate lines?
The matching options are the whole point. Real lists are messy in specific and predictable ways: case differences from different sign up forms, trailing spaces from copy and paste, and both at once. A tool that only does exact matching finds a fraction of the actual duplicates and gives you false confidence that the rest are clean. Being able to loosen the match and watch the removed count jump is usually the moment people realise how dirty their data was.
Showing only the duplicates is the other half. Cleaning a list and auditing a list are different jobs, and most tools only do the first. If you need to know which customers were entered twice, or which order references collided, the useful output is the repeats and their counts, not the tidied version. Because the counts come out tab separated they land in two columns when pasted into a spreadsheet, ready to sort by frequency.
Who is this tool for?
Anyone preparing an email list before importing it into a mailing platform starts here, since most platforms charge by contact and bill you for duplicates. The same applies to import files for any system that will happily create the same record twice.
It is equally common in data checking: finding repeated invoice numbers, duplicate SKUs in a product feed, or the same reference appearing in two exports. Developers use it on log output to see which errors actually repeat rather than scrolling. If your data is still in a spreadsheet file, our CSV viewer will open it and let you copy a single clean column out.
Frequently asked questions
Yes, unless you tick sort. Each value stays in the position where it first appeared, which matters when the order carries meaning, such as a chronological export.
Exact treats any difference as a different value. Ignore case treats Anna and anna as the same. Ignore spaces removes whitespace at the start and end before comparing. Both does the two together, which catches the largest number of real duplicates.
Yes. Set Keep to Only the repeated lines and you get an audit of exactly what appeared more than once, which you can pair with the counts option to see how often.
No. Even when matching ignores case and spaces, the line that is kept is the original text of the first occurrence, unmodified. The looser matching only decides what counts as a repeat.
Comfortably into the hundreds of thousands of lines. Matching is done with a hash lookup rather than by comparing every line against every other, so it stays fast as the list grows.
No. The text never leaves your browser, which is the point when the list is customer emails or anything else you should not be pasting into a random website.