Split string on whitespace in Python

Question

I'm looking for the Python equivalent of

String str = "many   fancy word \nhello    \thi";
String whiteSpaceRegex = "\\s";
String[] words = str.split(whiteSpaceRegex);

["many", "fancy", "word", "hello", "hi"]

Solution 1

The str.split() method without an argument splits on whitespace:

>>> "many   fancy word \nhello    \thi".split()
['many', 'fancy', 'word', 'hello', 'hi']

Solution 2

import re
s = "many   fancy word \nhello    \thi"
re.split('\s+', s)

Solution 3

Using split() will be the most Pythonic way of splitting on a string.

It's also useful to remember that if you use split() on a string that does not have a whitespace then that string will be returned to you in a list.

Example:

>>> "ark".split()
['ark']

Solution 4

Another method through re module. It does the reverse operation of matching all the words instead of spitting the whole sentence by space.

>>> import re
>>> s = "many   fancy word \nhello    \thi"
>>> re.findall(r'\S+', s)
['many', 'fancy', 'word', 'hello', 'hi']

Above regex would match one or more non-space characters.

View more solutions

Comments

[yak]

Also good to know is that if you want the first word only (which means passing 1 as second argument), you can use None as the first argument: s.split(None, 1)

[Raymond Hettinger]

If you only want the first word, use str.partition.

[Gulzar]

this gives me a whitespace token at the end of the line. No idea why, the original line doesn't even have that. Maybe this ignores newline?

[Óscar López]

@Gulzar do a strip() at the end

[user3527975]

@yak : Can you please edit your comment. The way it sounds right now is that s.split(None, 1) would return 1st word only. It rather gives a list of size 2. First item being the first word, second - rest of the string. s.split(None, 1)[0] would return the first word only

[Mark Jin]

Note that this is usually slower than str.split if performance is an issue.

[lee penkman]

Also the default split trims whitespace from either side so you don't have to call str.strip() e.g. " asdf asdf \t\n ".split() returns ['asdf', 'asdf']

[galois]

does str.split() do something like re.split('\s+', string) behind the scenes?

[Sven Marnach]

@galois No, it uses a custom implementation (which is faster). Also note that it handles leading and trailing whitespace differently.

[Kishor Pawar]

Sven, in my case line, could contain words like 'Kishor Pawar' 'Sven Marnach'. What would you suggest?

[Sven Marnach]

@KishorPawar It's rather unclear to me what you are trying to achieve. Do you want to split on whitespace, but disregard whitespace inside single-quoted substrings? If so, you can look into shlex.split(), which may be what you are looking for. Otherwise I suggest asking a new question – you will get a much quicker and more detailed answer.

[Kishor Pawar]

Thank you @SvenMarnach. You guessed the case correctly. I will take a look at shelx.split()