Remove duplicate lines from text online
Cleaning up a list — email addresses, log lines, spreadsheet exports — often means removing repeated entries by hand. Paste your list here and duplicate lines are removed instantly, keeping the first occurrence of each line and preserving the original order. Everything runs in your browser; nothing you paste is sent anywhere.
First occurrence wins, order is preserved
When a line repeats, the first instance stays where it is and the later ones are removed. That preserves the original sequence rather than reshuffling everything, which matters when order carries meaning — a chronological log, a ranked list, or a set of steps. If you also want the result alphabetised, run it through the line sorter afterwards; deduplicating first and sorting second is the usual order.
Matching is exact and case-sensitive
Two lines must be character-for-character identical to count as duplicates. That means 'Apple' and 'apple' both survive, and so do two otherwise-identical lines where one has a trailing space. This is deliberate — silently merging lines that differ would be worse than leaving them — but it does mean messy data needs cleaning first. Running the whitespace remover before deduplicating catches the trailing-space case, and the case converter handles the capitalisation one.
Common uses
Email lists exported from two overlapping sources, log files where the same error repeats hundreds of times, keyword lists compiled from several research tools, and CSV columns pasted together from multiple spreadsheets. In each case the duplicates are noise rather than data, and removing them before further processing saves work downstream.
Frequently asked questions
- Does this keep the first or last occurrence of a duplicate?
- The first occurrence is kept; later duplicates of the same line are removed.
- Is the comparison case-sensitive?
- Yes, "Apple" and "apple" are treated as different lines. Convert case first if you need case-insensitive deduplication.
- Is my list sent anywhere?
- No, deduplication happens locally in your browser using JavaScript. Nothing is transmitted.
- Can I make it case-insensitive?
- Not directly. Convert everything to lowercase with the case converter first, then deduplicate.
- Does it remove blank lines?
- Repeated blank lines collapse to one, since they are identical to each other. A single blank line is kept.