Why deduplication deserves a dedicated tool
Lists often collect duplicates while moving between spreadsheets, logs, APIs, documents, and command-line output. Duplicate entries can inflate reports, create repeated imports, and make allowlists or configuration files harder to review. Text Deduplicator focuses on one job: paste data, choose how equality should be interpreted, and produce a clean unique list without writing a temporary script.
The key is structure awareness. Plain text lines, numbers, and JSON arrays need different equality rules. Text may be compared with or without case sensitivity. Number mode treats 1 and 01 as the same numeric value. JSON arrays are parsed first so invalid input is reported before a result is shown.
Working with multiline text
Multiline mode treats each line as one item. This is useful for email fragments, names, slugs, domains, environment variable keys, tags, and notes. Trimming removes accidental spaces at the beginning or end of each line. Ignoring empty lines keeps copied spreadsheet ranges from producing blank output items.
Working with numbers
Numeric deduplication should use numeric identity, not string spelling. In number mode, 1, 01, and 1.0 represent the same value. The tool validates every non-empty line as a finite number and reports an error when mixed content appears, which prevents misleading output.
Working with JSON arrays
JSON mode expects an array because array membership is an explicit JSON concept. The tool does not infer business keys from arbitrary objects. When entries are objects, they are compared with a stable key-sorted representation while the output preserves the original JSON values.
Privacy and browser-side processing
The transformation runs in the browser. Pasted content is not uploaded to Tool3M for processing. This is suitable for everyday snippets, although users should still follow their own security policy before pasting sensitive data into any web page.
Use the tool
Open the Text Deduplicator, paste your data, load the demo if you want a quick example, then copy the generated output. Keep the options visible while reviewing the result so future runs can be repeated with the same assumptions.