remove duplicate lines
The first of each line stays exactly where it was.
The short answer
Repeated lines are removed and the first of each is left exactly where it was, so a list stays in the order you built it. Blank lines are left alone, because they are usually doing something: separating groups, or keeping a paste readable.
Strips repeated lines from a list without reordering what is left, so a list you built in a particular order stays in it.
The formula
keep a line the first time it is seen, drop it every time after
Worked examples
- one, two, one, three
- one, two, three
- Trailing spaces on a repeat
- still counted as the same line
- APPLE and apple
- kept both, case matters here
Reference table
| Case | Treated as duplicate | Why |
|---|---|---|
| one / one | Yes | Identical |
| one / one␠ | Yes | Compared after trimming the ends |
| one / One | No | Case is meaningful in most lists |
| one / two one | No | Whole lines, not words within them |
How to use it
- 01Paste your linesOne item per line. Blank lines are left alone.
- 02Copy the resultEvery repeat after the first is gone, the order is untouched.
Also searched for
Searches that land here include remove duplicate lines, delete duplicate lines, dedupe list, unique lines and remove repeats.
Deduplicating a list
Keeping the first occurrence matters
Which copy you keep changes the result whenever the order carries information: a priority list, a sequence of steps, a changelog. Keeping the first and dropping later repeats leaves the earliest position of every item intact, which is what people mean by "remove duplicates" when they have not thought about it.
Blank lines are left alone
A blank line is usually doing a job: separating groups, keeping a paste readable, marking a section. Deduplicating them collapses that structure, so they pass straight through untouched however many there are.
What this cannot see
Duplicates that are not identical. "St James's Road" and "St James Road" are the same street and two different lines, and no rule based on exact text will catch that. If your list has near-duplicates, sort it first and read down the result: they end up next to each other, which makes them easy to spot by eye.
Questions people ask
Does it sort the list?
No, and that is deliberate. Sorting and deduplicating are separate jobs, and doing both at once means you cannot tell which change did what. If you want both, run this and then sort the result, in that order.
Are Apple and apple duplicates?
No, both are kept. Case is meaningful in most lists people paste here: usernames, product codes, tags, filenames on a case-sensitive system. Treating them as the same would quietly delete real entries.
What about lines that differ only by a trailing space?
They count as duplicates. Lines are compared after the ends are trimmed, because an invisible trailing space is almost never a meaningful difference, and a list pasted out of a spreadsheet is full of them.