Removing a list of characters in string
Solution 1
If you're using python2 and your inputs are strings (not unicodes), the absolutely best method is str.translate
:
>>> chars_to_remove = ['.', '!', '?']
>>> subj = 'A.B!C?'
>>> subj.translate(None, ''.join(chars_to_remove))
'ABC'
Otherwise, there are following options to consider:
A. Iterate the subject char by char, omit unwanted characters and join
the resulting list:
>>> sc = set(chars_to_remove)
>>> ''.join([c for c in subj if c not in sc])
'ABC'
(Note that the generator version ''.join(c for c ...)
will be less efficient).
B. Create a regular expression on the fly and re.sub
with an empty string:
>>> import re
>>> rx = '[' + re.escape(''.join(chars_to_remove)) + ']'
>>> re.sub(rx, '', subj)
'ABC'
(re.escape
ensures that characters like ^
or ]
won't break the regular expression).
C. Use the mapping variant of translate
:
>>> chars_to_remove = [u'δ', u'Γ', u'ж']
>>> subj = u'AжBδCΓ'
>>> dd = {ord(c):None for c in chars_to_remove}
>>> subj.translate(dd)
u'ABC'
Full testing code and timings:
#coding=utf8
import re
def remove_chars_iter(subj, chars):
sc = set(chars)
return ''.join([c for c in subj if c not in sc])
def remove_chars_re(subj, chars):
return re.sub('[' + re.escape(''.join(chars)) + ']', '', subj)
def remove_chars_re_unicode(subj, chars):
return re.sub(u'(?u)[' + re.escape(''.join(chars)) + ']', '', subj)
def remove_chars_translate_bytes(subj, chars):
return subj.translate(None, ''.join(chars))
def remove_chars_translate_unicode(subj, chars):
d = {ord(c):None for c in chars}
return subj.translate(d)
import timeit, sys
def profile(f):
assert f(subj, chars_to_remove) == test
t = timeit.timeit(lambda: f(subj, chars_to_remove), number=1000)
print ('{0:.3f} {1}'.format(t, f.__name__))
print (sys.version)
PYTHON2 = sys.version_info[0] == 2
print ('\n"plain" string:\n')
chars_to_remove = ['.', '!', '?']
subj = 'A.B!C?' * 1000
test = 'ABC' * 1000
profile(remove_chars_iter)
profile(remove_chars_re)
if PYTHON2:
profile(remove_chars_translate_bytes)
else:
profile(remove_chars_translate_unicode)
print ('\nunicode string:\n')
if PYTHON2:
chars_to_remove = [u'δ', u'Γ', u'ж']
subj = u'AжBδCΓ'
else:
chars_to_remove = ['δ', 'Γ', 'ж']
subj = 'AжBδCΓ'
subj = subj * 1000
test = 'ABC' * 1000
profile(remove_chars_iter)
if PYTHON2:
profile(remove_chars_re_unicode)
else:
profile(remove_chars_re)
profile(remove_chars_translate_unicode)
Results:
2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]
"plain" string:
0.637 remove_chars_iter
0.649 remove_chars_re
0.010 remove_chars_translate_bytes
unicode string:
0.866 remove_chars_iter
0.680 remove_chars_re_unicode
1.373 remove_chars_translate_unicode
---
3.4.2 (v3.4.2:ab2c023a9432, Oct 5 2014, 20:42:22)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
"plain" string:
0.512 remove_chars_iter
0.574 remove_chars_re
0.765 remove_chars_translate_unicode
unicode string:
0.817 remove_chars_iter
0.686 remove_chars_re
0.876 remove_chars_translate_unicode
(As a side note, the figure for remove_chars_translate_bytes
might give us a clue why the industry was reluctant to adopt Unicode for such a long time).
Solution 2
You can use str.translate()
:
s.translate(None, ",!.;")
Example:
>>> s = "asjo,fdjk;djaso,oio!kod.kjods;dkps"
>>> s.translate(None, ",!.;")
'asjofdjkdjasooiokodkjodsdkps'
Solution 3
You can use the translate method.
s.translate(None, '!.;,')
Solution 4
If you are using python3 and looking for the translate
solution - the function was changed and now takes 1 parameter instead of 2.
That parameter is a table (can be dictionary) where each key is the Unicode ordinal (int) of the character to find and the value is the replacement (can be either a Unicode ordinal or a string to map the key to).
Here is a usage example:
>>> list = [',', '!', '.', ';']
>>> s = "This is, my! str,ing."
>>> s.translate({ord(x): '' for x in list})
'This is my string'
Solution 5
''.join(c for c in myString if not c in badTokens)
Related videos on Youtube
Laura
Updated on July 16, 2022Comments
-
Laura almost 2 years
I want to remove characters in a string in python:
string.replace(',', '').replace("!", '').replace(":", '').replace(";", '')...
But I have many characters I have to remove. I thought about a list
list = [',', '!', '.', ';'...]
But how can I use the
list
to replace the characters in thestring
?-
Martijn de Milliano about 12 yearsSee stackoverflow.com/questions/1919096/… for various solutions and a nice comparison.
-
JustAC0der over 7 yearsIt's a pity that Python (which is said to come with batteries included) does not handle this use case out of the box. PHP's function str_replace does it - you can pass an array as the first argument and a string as the second (php.net/manual/pl/function.str-replace.php ).
-
-
Sven Marnach about 12 years@thg435: Nobody asked for this, but anyway:
s.translate(dict.fromkeys(map(ord, u",!.;")))
-
hobs over 10 yearsThis (and @PraveenGollakota's) simultaneous answer is exactly what @Laura asked for and should be the preferred answer(s).
-
antonavy over 9 yearsThe second method raises an error
TypeError: translate() takes exactly one argument (2 given)
. Apparently it takes dict as an argument. -
Wolf over 9 yearsUseful in similar cases not based on chars and strings +1
-
FuzzyAmi over 9 years@antonavy - the 2nd solution does work - but only of the string is not unicode (for which a different translate() is needed)
-
Gank over 8 yearswhy python3:TypeError: translate() takes exactly one argument (2 given)
-
Sven Marnach over 8 years@Gank: The
unicode.translate()
method has different parameters than thestr.translate()
method. Use the variant in the above comment for Unicode objects. -
rioted over 7 yearsAlso note that this can just as easily be done with list comprehensions, which is way more pythonic in my opinion.
-
Jun711 over 5 years@SvenMarnach what is map(ord, u",!.;"))? and does u stand for unicode?
-
Sven Marnach over 5 years@Jun This maps the
ord()
function on all characters of the Unicode stringu",!.;"
, resulting in a list (Python 2) or an iterator (Python 3) of Unicode code points. Theu
denotes a Unicode string in Python 2. In Python 3 it's optional, since strings are Unicode strings by default (and early versions of Python 3 did not even support theu""
syntax). -
Aybid over 3 yearsExplanation: 1. maketrans(x,y,z): the third parameter is for replace. x,y are '' here, so makes no change. Only characters with z are removed 2. translate(): returns a string where each character is mapped to its corresponding character in the translation table (here from the maketrans fn)
-
confiq over 2 yearsstr.translate() is from python2 and there is no need to use maketrans() func.
-
2e0byo over 2 years@config on the contrary: python 3 has a
str.translate()
which takes char ords, and thus does require something likestr.maketrans()
(or ee.g. a dict comp callingord
as in the other python3 answer).. This answer won't work without themaketrans()
(try it).