Removing control characters from a UTF-8 string

c# string utf-8 control-characters

29,628

Solution 1

I think the following code will work for you:

public static string RemoveControlCharacters(string inString)
{
    if (inString == null) return null;
    StringBuilder newString = new StringBuilder();
    char ch;
    for (int i = 0; i < inString.Length; i++)
    {
        ch = inString[i];
        if (!char.IsControl(ch))
        {
            newString.Append(ch);
        }
    }
    return newString.ToString();
}

Solution 2

This is how I roll:

Regex.Replace(evilWeirdoText, @"[\u0000-\u001F]", string.Empty)

This strips out all the first 31 control characters. The next hex value up from \u001F is \u0020 AKA the space. Everything before space is all the line feed and null nonsense.

To believe me on the characters: http://donsnotes.com/tech/charsets/ascii.html

29,628

Author by

Xaqron

Physician

Updated on August 18, 2020

Comments

Xaqron over 3 years

I found this question but it removes all valid utf-8 characters also (returns me a blank string, while there are valid utf-8 characters plus control characters). As I read about utf-8, there's not a specific range for control characters and each character set has its own control characters.

How can I modify above solution to only remove control characters ?
- Henk Holterman almost 13 years
  
  You know, with a few lines of code (what exactly is a utf-8 string?) and a small sample of the text this would start to look like a real question.
- Xaqron almost 13 years
  
  real string is an Arabic utf-8 string with some semicolon and control characters. I have provided a link to the most similar question: stackoverflow.com/questions/20762/…
- CodesInChaos almost 13 years
  
  How do you define control characters? Those with codepoint <32?
- Xaqron almost 13 years
  
  @CodeInChaos: I found it is not about real control characters. Any special character like ", ; etc makes problem. I'm trying to set the string as a HttpHeader but get this exception: Specified value has invalid Control characters
Xaqron almost 13 years

Thanks. I still get Specified value has invalid Control characters. exception while trying to set the string as a HttpHeader.
Centro almost 13 years

@Xaqon It didn't work for all control characters. I have changed the condition to !char.IsControl(ch) and now it should work for you.
Xaqron almost 13 years

I found the problem, it is not about real control characters. Any special character like \", ; etc makes problem. I have no idea how to remove them from a utf-8 string.
Centro almost 13 years

@Xaqron What is a special character in your case? Any non alphanumeric? You should define these special characters and change the condition.
Xaqron almost 13 years

For example 1563 is a ; and makes problem. I want to exclude all not alphabetical characters in all languages.
Jalal Said almost 13 years

@Xaqron: use char.IsLetter() method.
InLaw almost 6 years

it would only filter for ASCII (UTF-8 C0 Controls but without "delete (rubout)") but the question is about UTF-8. There you have more control chars (UTF-8 C1 Controls).