Redirection of stdout to a file not working

Question

I have a script that uses subprocesses to fetch HTML:

misha@misha-K42Jr:~/git/domain_classifier$ python webkit_retrieve.py error-cut.txt html/error -N 5
http://kurabo.co.jp HostNotFoundError
http://monarch.com HostNotFoundError
http://nssmgmt.com HostNotFoundError
http://sbcglobal.net HostNotFoundError
http://dynamixcorp.com SslHandshakeFailedError
http://groupe-synox.com RemoteHostClosedError
QFont::setPixelSize: Pixel size <= 0 (0)
http://www.cnn.com NoError
http://pacbell.net TimeoutError

If I run the same script, but redirect output to a file, I get nothing in the output:

misha@misha-K42Jr:~/git/domain_classifier$ python webkit_retrieve.py error-cut.txt html/error -N 5 > stdout.txt
QFont::setPixelSize: Pixel size <= 0 (0)
misha@misha-K42Jr:~/git/domain_classifier$ cat stdout.txt
misha@misha-K42Jr:~/git/domain_classifier$

Why is the output empty? Should it not contain the same things that were printed to stdout in the first case?

The question is not about merge stdout and stderr but why redirected stdout produce an empty file

Solution 1

use &> for redirection, this should redirect stdout and stderr to designated file

Solution 2

You have sent stdout to the file, but your program is reporting errors which go to stderr. To setup redirection of stderr, use 2> syntax.

This link might help: http://www.tldp.org/LDP/abs/html/io-redirection.html

Comments

[mcalex]

what do you get in the file?

[mpenkov]

The file is empty, hence cat stdout.txt prints absolutely nothing.

[Richard Chambers]

What is the python source for setting up the output file handles and doing the output itself?

[Diego Torres Milano]

If the output for both commands is different and stdout.txt is really empty then the python script acts differently if the stdout is redirected

[mgarciaisaia]

If it was echoing to stderr, wouldn't it still show at the terminal when the command is run with just stdout redirected?

[mcalex]

good point. It almost looks like stderr is actually being redirected, given the 'Pixel size' line's appearance

[mpenkov]

stderr is correctly being printed to standard output (the pixel size stuff) as per the default behavior on linux systems. I don't want to redirect stderr, I want to redirect stdout.

[mpenkov]

If the output was going to stderror, then I would see it in the console (as I do in the first case). In the second case, the output is being redirected to a file, but the file ends up empty.

[m0skit0]

Doesn't work, file is not created

[cristianoms]

I've been using > successfully forever and after 20 years it stopped (not sure if it has anything to do with the fact it's a python script that wasn't working). Now with the &> it worked. What's that all about?