Text Runs locally Ready to use Built-in examples No tracking

List Dedupe & Sort

Paste a list to remove duplicates, sort alphabetically or numerically, and see original vs. kept counts.

Deduplicating is the workhorse of list tidying: a column of addresses, phone numbers or keywords with repeats in it, where one copy of each has to be kept and the number removed is worth knowing. Paste the list and the separator is detected automatically — newlines, commas or spaces — duplicates are dropped, optionally ignoring case, and sorting by alphabet, number or length can be applied on the way through.

Two things to watch. Deduplication keeps the first occurrence by default, so the output order matches the input; tick sorting as well and the result is reordered. And automatic detection works on the priority newline, then comma, then space, which means a list holding both commas and newlines is split on newlines first, so a single record spanning lines is not torn apart.

—
—
—
—

How to use

  1. Paste the list; the separator is detected automatically, or pick one yourself.
  2. Choose whether deduplication ignores case.
  3. Choose a sort — alphabetical, numeric, by length, or none at all.
  4. Read the result and the original-versus-kept counts.

How it works

Separators and splitting rules

With "auto-detect" separators, splitting follows the priority newlines > commas > spaces; one-per-line or comma-separated lists both paste in directly.

Sorting rules

Numeric ascending sort orders by value (10 comes after 2); alphabetical sort puts 100 before 2.

Deduplication and case-insensitivity

Checking "ignore case when deduplicating" treats Vue and vue as the same entry, keeping the casing of its first occurrence.

Code example

Shell Deduplicating and sorting on the command line 

# Deduplicate, keeping the first occurrence
awk '!seen[$0]++' list.txt > unique.txt

# Deduplicate + sort
sort -u list.txt > sorted.txt

# Case-insensitive deduplication
sort -fu list.txt > case-uniq.txt

# Compare the counts
printf "%s rows before → %s rows after\n" "$(wc -l < list.txt)" "$(wc -l < unique.txt)"

JavaScript Case-insensitive deduplication (order preserved) 

const dedupe = (arr, ignoreCase = false) => {
  const seen = new Set();
  return arr.filter((x) => {
    const k = ignoreCase ? x.trim().toLowerCase() : x.trim();
    if (seen.has(k)) return false;
    seen.add(k);
    return true;
  });
};

dedupe(['Apple', 'apple', 'Banana'], true); // ['Apple', 'Banana']
dedupe(['b', 'a', 'b']).sort();             // ['a', 'b']

FAQ

Does the order change after deduplication?

With the default "keep original order", deduplication only removes repeats at their first occurrence — order is untouched. Only choosing a sort option reorders.

How much data can it handle?

Up to 200,000 characters per run — name lists and keyword lists of a few thousand lines are fine. Everything is processed locally, nothing uploaded.

Does it work with mixed Chinese and English lists?

Yes. Dedup matches characters exactly; sorting interleaves Chinese and English by Unicode order; case-insensitivity affects only English letters.

How do I dedupe phone numbers or emails?

Paste the list directly (one per line or comma-separated); the tool compares whole entries and reports original and removed counts. For emails, enable "ignore case" — addresses are case-insensitive, so ABC@x.com and abc@x.com are the same mailbox.

Is dedup by whole row or by a column?

By whole-row text. Rows like "name, phone" survive unless the entire line matches. To dedupe by one column, isolate that column first, or extract it one value per line before processing.

Does sorting affect the dedup result?

No. Deduplication produces the "first occurrence kept, repeats removed" set; sorting afterwards changes display order only, not membership. Check the reported removal count to confirm before deciding whether to keep the original order.

Can it handle tens of thousands of Excel rows?

Yes. Pure text processing is local and fast for tens of thousands of lines; rendering many rows costs memory, though, so copy the result right away. For tables in the hundreds of thousands, prefer Excel's built-in Remove Duplicates or a command-line tool.