How to remove special characters except space from a file in python?

Question

I have a huge corpus of text (line by line) and I want to remove special characters but sustain the space and structure of the string.

hello? there A-Z-R_T(,**), world, welcome to python.
this **should? the next line#followed- by@ an#other %million^ %%like $this.

should be

hello there A Z R T world welcome to python
this should be the next line followed by another million like this

Solution 1

You can use this pattern, too, with regex:

import re
a = '''hello? there A-Z-R_T(,**), world, welcome to python.
this **should? the next line#followed- by@ an#other %million^ %%like $this.'''

for k in a.split("\n"):
    print(re.sub(r"[^a-zA-Z0-9]+", ' ', k))
    # Or:
    # final = " ".join(re.findall(r"[a-zA-Z0-9]+", k))
    # print(final)

Output:

hello there A Z R T world welcome to python 
this should the next line followed by an other million like this 

Edit:

Otherwise, you can store the final lines into a list:

final = [re.sub(r"[^a-zA-Z0-9]+", ' ', k) for k in a.split("\n")]
print(final)

Output:

['hello there A Z R T world welcome to python ', 'this should the next line followed by an other million like this ']

Solution 2

I think nfn neil answer is great...but i would just add a simple regex to remove all no words character,however it will consider underscore as part of the word

print  re.sub(r'\W+', ' ', string)
>>> hello there A Z R_T world welcome to python

Solution 3

you can try this

import re
sentance = '''hello? there A-Z-R_T(,**), world, welcome to python. this **should? the next line#followed- by@ an#other %million^ %%like $this.'''
res = re.sub('[!,*)@#%(&$_?.^]', '', sentance)
print(res)

re.sub('["]') -> here you can add which symbol you want to remove

Solution 4

A more elegant solution would be

print(re.sub(r"\W+|_", " ", string))

>>> hello there A Z R T world welcome to python this should the next line followed by another million like this

Here, re is regex module in python

re.sub will substitute pattern with space i.e., " "

r'' will treat input string as raw (with \n)

\W for all non-words i.e. all special characters *&^%$ etc excluding underscore _

+ will match zero to unlimited matches, similar to * (one to more)

| is logical OR

_ stands for underscore

View more solutions