Weird (unicode?) characters

unicode special-characters zalgo

11,539

Solution 1

These are combining diacritical marks. For the character é e-acute you can represent it using either the code point U+00E9 (LATIN_SMALL_LETTER_E_WITH_ACUTE) or the sequence U+0065 U+0301 (LATIN_SMALL_LETTER_E COMBINING_ACUTE_ACENT) where the text renderer places the accent above the preceding code point.

The user is exploiting this with a sequence of combining marks:

codepoint   glyph   escaped    UTF-8           info
=======================================================================
U+2665      ♥       \u2665     e2,99,a5,       MISCELLANEOUS_SYMBOLS, OTHER_SYMBOL
U+034a      ͊       \u034a     cd,8a,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0360      ͠       \u0360     cd,a0,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0357      ͗       \u0357     cd,97,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0309      ̉       \u0309     cc,89,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033d      ̽       \u033d     cc,bd,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0344      ̈́       \u0344     cd,84,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0309      ̉       \u0309     cc,89,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0351      ͑       \u0351     cd,91,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0340      ̀       \u0340     cd,80,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+035d      ͝       \u035d     cd,9d,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0301      ́       \u0301     cc,81,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0303      ̃       \u0303     cc,83,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0352      ͒       \u0352     cd,92,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030f      ̏       \u030f     cc,8f,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+034b      ͋       \u034b     cd,8b,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0303      ̃       \u0303     cc,83,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0305      ̅       \u0305     cc,85,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0307      ̇       \u0307     cc,87,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030a      ̊       \u030a     cc,8a,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030f      ̏       \u030f     cc,8f,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030e      ̎       \u030e     cc,8e,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0344      ̈́       \u0344     cd,84,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+034a      ͊       \u034a     cd,8a,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0350      ͐       \u0350     cd,90,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0309      ̉       \u0309     cc,89,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0351      ͑       \u0351     cd,91,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0304      ̄       \u0304     cc,84,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030c      ̌       \u030c     cc,8c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0309      ̉       \u0309     cc,89,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0301      ́       \u0301     cc,81,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0344      ̈́       \u0344     cd,84,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0341      ́       \u0341     cd,81,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0301      ́       \u0301     cc,81,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0305      ̅       \u0305     cc,85,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0307      ̇       \u0307     cc,87,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+034c      ͌       \u034c     cd,8c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033d      ̽       \u033d     cc,bd,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033d      ̽       \u033d     cc,bd,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0357      ͗       \u0357     cd,97,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0301      ́       \u0301     cc,81,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0360      ͠       \u0360     cd,a0,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0304      ̄       \u0304     cc,84,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033e      ̾       \u033e     cc,be,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0343      ̓       \u0343     cd,83,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0344      ̈́       \u0344     cd,84,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0307      ̇       \u0307     cc,87,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0358      ͘       \u0358     cd,98,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0305      ̅       \u0305     cc,85,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+035d      ͝       \u035d     cd,9d,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+035b      ͛       \u035b     cd,9b,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0301      ́       \u0301     cc,81,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0344      ̈́       \u0344     cd,84,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0350      ͐       \u0350     cd,90,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033d      ̽       \u033d     cc,bd,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0314      ̔       \u0314     cc,94,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030c      ̌       \u030c     cc,8c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030b      ̋       \u030b     cc,8b,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030c      ̌       \u030c     cc,8c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033e      ̾       \u033e     cc,be,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0360      ͠       \u0360     cd,a0,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0301      ́       \u0301     cc,81,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033f      ̿       \u033f     cc,bf,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+034c      ͌       \u034c     cd,8c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0314      ̔       \u0314     cc,94,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0315      ̕       \u0315     cc,95,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+034a      ͊       \u034a     cd,8a,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0346      ͆       \u0346     cd,86,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0344      ̈́       \u0344     cd,84,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0309      ̉       \u0309     cc,89,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+035d      ͝       \u035d     cd,9d,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0341      ́       \u0341     cd,81,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0315      ̕       \u0315     cc,95,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030e      ̎       \u030e     cc,8e,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0314      ̔       \u0314     cc,94,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030a      ̊       \u030a     cc,8a,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0357      ͗       \u0357     cd,97,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0358      ͘       \u0358     cd,98,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030a      ̊       \u030a     cc,8a,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0315      ̕       \u0315     cc,95,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0302      ̂       \u0302     cc,82,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030e      ̎       \u030e     cc,8e,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030d      ̍       \u030d     cc,8d,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030f      ̏       \u030f     cc,8f,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0308      ̈       \u0308     cc,88,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0340      ̀       \u0340     cd,80,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030f      ̏       \u030f     cc,8f,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031a      ̚       \u031a     cc,9a,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+034b      ͋       \u034b     cd,8b,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031a      ̚       \u031a     cc,9a,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031a      ̚       \u031a     cc,9a,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+034c      ͌       \u034c     cd,8c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+030b      ̋       \u030b     cc,8b,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033d      ̽       \u033d     cc,bd,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0304      ̄       \u0304     cc,84,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0310      ̐       \u0310     cc,90,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033d      ̽       \u033d     cc,bd,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0350      ͐       \u0350     cd,90,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031b      ̛       \u031b     cc,9b,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0358      ͘       \u0358     cd,98,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0300      ̀       \u0300     cc,80,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0323      ̣       \u0323     cc,a3,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0318      ̘       \u0318     cc,98,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031f      ̟       \u031f     cc,9f,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+035c      ͜       \u035c     cd,9c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0318      ̘       \u0318     cc,98,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+035c      ͜       \u035c     cd,9c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0325      ̥       \u0325     cc,a5,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0353      ͓       \u0353     cd,93,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+032b      ̫       \u032b     cc,ab,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+032a      ̪       \u032a     cc,aa,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0339      ̹       \u0339     cc,b9,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+032a      ̪       \u032a     cc,aa,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+032a      ̪       \u032a     cc,aa,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+035c      ͜       \u035c     cd,9c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+032e      ̮       \u032e     cc,ae,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+032f      ̯       \u032f     cc,af,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0327      ̧       \u0327     cc,a7,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031e      ̞       \u031e     cc,9e,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0318      ̘       \u0318     cc,98,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0319      ̙       \u0319     cc,99,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0326      ̦       \u0326     cc,a6,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031d      ̝       \u031d     cc,9d,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+032d      ̭       \u032d     cc,ad,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+032d      ̭       \u032d     cc,ad,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0355      ͕       \u0355     cd,95,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031c      ̜       \u031c     cc,9c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0330      ̰       \u0330     cc,b0,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0329      ̩       \u0329     cc,a9,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0317      ̗       \u0317     cc,97,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031f      ̟       \u031f     cc,9f,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0339      ̹       \u0339     cc,b9,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0354      ͔       \u0354     cd,94,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031c      ̜       \u031c     cc,9c,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0325      ̥       \u0325     cc,a5,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031f      ̟       \u031f     cc,9f,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0317      ̗       \u0317     cc,97,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0317      ̗       \u0317     cc,97,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0325      ̥       \u0325     cc,a5,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0326      ̦       \u0326     cc,a6,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0320      ̠       \u0320     cc,a0,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0316      ̖       \u0316     cc,96,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+032b      ̫       \u032b     cc,ab,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0355      ͕       \u0355     cd,95,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033a      ̺       \u033a     cc,ba,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0327      ̧       \u0327     cc,a7,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033b      ̻       \u033b     cc,bb,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+031e      ̞       \u031e     cc,9e,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0325      ̥       \u0325     cc,a5,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0327      ̧       \u0327     cc,a7,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0339      ̹       \u0339     cc,b9,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0347      ͇       \u0347     cd,87,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0331      ̱       \u0331     cc,b1,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0325      ̥       \u0325     cc,a5,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0325      ̥       \u0325     cc,a5,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+033b      ̻       \u033b     cc,bb,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0347      ͇       \u0347     cd,87,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0326      ̦       \u0326     cc,a6,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0319      ̙       \u0319     cc,99,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK
U+0323      ̣       \u0323     cc,a3,          COMBINING_DIACRITICAL_MARKS, NON_SPACING_MARK

Some points I made in the comments:

The Unicode standard considers all sequences of code points to be valid if not meaningful (see chapter 2 of Unicode 6)
Unicode does not describe how code points should be displayed - that's up to the text rendering technology
Normalizing to NFC and matching on code point category are likely to be useful for detecting redundant diacritics
You can create sequences like the one above using browser consoles
- Just type in a UTF-16 JavaScript string literal like "\u2665\u034a\u0360\u0357"
- You can just use the code point value from the charts for anything in the basic multilingual plane
- For anything outside the BMP you'll have to translate the code points to UTF-16

Solution 2

Blocking these codepoints may be enough for you:

http://en.wikipedia.org/wiki/Combining_character#Unicode_ranges

11,539

user1960364

Updated on September 15, 2022

Comments

user1960364 over 1 year
A user has been posting some weird characters on my site and I'd like to block them from doing so but without blocking characters used in foreign languages... Therefore, using a regex such as [a-z0-9!@#$%^&*()...] isn't an option.

Could someone explain to me what is happening here, a break down of why it displays the way it does. How the characters are created and how can I prevent them from doing it?
```
♥̧̧̧̛̣̘̟̘̥͓̫̪̹̪̪̮̯̞̘̙̦̝̭̭͕̜̰̩̗̟̹͔̜̥̟̗̗̥̦̠̖̫͕̺̻̞̥̹͇̱̥̥̻͇̦̙̣͊͗̉̽̈́̉͑̀́̃͒̏͋̃̅̇̊̏̎̈́͊͐̉͑̄̌̉́̈́́́̅̇͌̽̽͗́̄̾̓̈́̇̅͛́̈́͐̽̔̌̋̌̾́̿͌̔͊͆̈́̉́̎̔̊͗̊̂̎̍̏̈̀̏͋͌̋̽̄̐̽͐̀͘̕̕͘̕̚̚̚͘͜͜͜͠͝͠͝͠͝
```
Thanks

EDIT: So they're used to accent characters? Is there a common practice or way to prevent users from exploiting them without blocking them completely? I don't know enough about foreign languages or their actual use/purpose so crafting something to limit the use of the combining characters is outside my scope of possibilities. :-/
- Remy Lebeau about 10 years
  
  How are you allowing users to post text to your site in the first place??
- kirilloid about 10 years
  
  They are called "combining [diacritic] characters". You could search for codepoints range.
user1960364 about 10 years

I've already taken the standard security measures to prevent XSS and SQL Injectons, is there something else I should be worried about?
user1960364 about 10 years

Would you happen to know what character requires the most of these combining characters and how many characters it consists of?
McDowell about 10 years

Can't help you there. The Unicode standard considers all sequences valid if not linguistically meaningful. Visual appearance of any code point is outside the scope of the standard - that is, it's the problem of whatever draws the text.
McDowell about 10 years

You can generate them easily in a browser console using UTF-16 escape sequences like "\u2665\u034a\u0360\u0357" assuming font support. Anything in the basic multilingual plane you can just use the code point value from the charts. You should look at normalization to NFC and the character categories which you can match on in many regex implementations.
kirilloid about 10 years

Information (links) in this comment is worth adding to the answer itself. It is much more useful, than characters table.
RemcoGerlich about 10 years

How on earth do you get from "users can post arbitrary Unicode strings" to "users can probably do a lot more damage"? It's just a Unicode string that looks odd.
vonbrand about 10 years

@RemcoGerlich, if "weird characters" get through, and OP is worried about cleaning by eliminating many characters like <>&....