What Are Invisible Characters and How Do They Corrupt Text?
Invisible characters are non-printing Unicode formatting code points (such as Zero-Width Space U+200B and Byte Order Mark U+FEFF) that occupy 0 pixels on screen but add hidden bytes that break string comparisons, compilers, and JSON/SQL parsers.
Invisible characters are hidden Unicode code points that render with zero width on your screen but remain present inside your text data. Common invisible characters include:
- Zero-Width Space (
\u200B): Hidden spaces that break string comparisons, database lookups, and programming syntax. - Zero-Width Non-Joiner (
\u200C) and Joiner (\u200D): Formatting control characters used in typography that often leak into copied text. - Byte Order Mark (
\uFEFF/ Zero-Width No-Break Space): A hidden marker at the start of files that breaks web parsers and JSON payloads. - Soft Hyphens (
\u00AD): Invisible discretionary hyphens inserted by publishing software that cause random hyphenation bugs. - Exotic Unicode Spaces (
\u00A0,\u2000–\u200A,\u202F,\u3000): Non-breaking spaces and ideographic spaces that fail strict ASCII validations.
How Does Text Cleaner Detect and Strip Invisible Characters?
The engine runs two deterministic rules: s/[\u200B\u200C\u200D\u2060\uFEFF\u00AD]//g to delete zero-width marks, and s/[\u00A0\u2007\u202F\u2009\u2002\u2003\u3000]/ /g to normalize exotic spaces to ASCII U+0020.
Clicking Invisible detects all non-printing zero-width characters and deletes them outright, while safely converting non-breaking and exotic spaces into standard ASCII spaces (\u0020).
- Worked Before / After Example: Input
This text contains hidden zero-width spaces\u200B and non-breaking spaces\u00A0that break code and search queries.(104 code points, 16 words, 1U+200B+ 1U+00A0) → OutputThis text contains hidden zero-width spaces and non-breaking spaces that break code and search queries.(103 ASCII characters, 16 words; 1 zero-width space deleted, 1 NBSP converted toU+0020).
| Unicode Code Point & Character Name | Action / Regex Rule | Input Example (Before) | Output Result (After) |
|---|---|---|---|
U+200B Zero-Width Space (ZWSP) |
Deleted (s/\u200B//g) |
api\u200B_key (8 chars, 10 UTF-8 bytes) |
api_key (7 chars, 7 bytes) |
U+FEFF Byte Order Mark (BOM / ZWNBSP) |
Deleted (s/\uFEFF//g) |
\uFEFF{"ok":true} (12 chars, 14 bytes) |
{"ok":true} (11 chars, 11 bytes) |
U+200C / U+200D / U+2060 (ZWNJ, ZWJ, WJ) |
Deleted (s/[\u200C\u200D\u2060]//g) |
word\u2060join (9 chars) |
wordjoin (8 chars) |
U+00AD Soft Hyphen (SHY) |
Deleted (s/\u00AD//g) |
extra\u00ADordinary (14 chars) |
extraordinary (13 chars) |
U+00A0 / U+202F / U+2009 / U+3000 (Exotic Spaces) |
Normalized to U+0020 (ASCII Space) |
SELECT\u00A0*\u202FFROM (13 chars) |
SELECT * FROM (13 ASCII chars) |
