Are there benefits to running X86-64 Python on a 64-bit CPU in a 64-bit OS?

Question

What kind of benefits are there to running the amd64 builds of Python and extensions? (Lots of extensions compiled for amd64 here.) I have an i5 processor and Win7 64-bit, so it seems like it would be appropriate. But it also sounds like it is buggy, beta, unsupported, etc.

Does it actually provide a performance benefit? In which areas? I'd be running SciPy, NumPy, etc. I occasionally get "out of memory" errors with 32-bit Python and my machine has 4 GiB of RAM.

Can win32 packages be installed on a 64-bit Python base if no 64-bit version of the package exists?

Solution 1

The primary rationale to use a 64-bit Python is that you can access more than 2GB of main memory, e.g. if you have large dicts, lists, or long strings. This requires that you actually have that much memory in your system to be practical.

A secondary effect is that in AMD64 mode, the CPU has more registers, so the resulting code may run slightly faster (for integer operations).

Python in 64-bit mode on Windows certainly is not beta or unsupported. It may be buggy, but only if you actually do have very large data structures. 64-bit Python has been around 15 years (though not on Windows).

Solution 2

Same advantage as every other 64-bit program: lots of process space, and access to more and larger registers (for the VM and C modules at least). But no, you need to find 64-bit versions of the C modules.

Solution 3

The normal gains one get by using 64bit code in general. I haven't found any recent benchmarks, but at the time of x64 introduction, code could run up to 30% faster on 64bit than on 32bit on x86 hardware -- this gap certainly has fallen somehow with the optimizatgions since the time, but is still probably faster.

Also, you benefit of being ablle to transparantly use more than 4GB of memory should you need too.

Note however that the speed gains in 64bit are due to the awfull 32bit legacy ABI: enve a core i7 running in 32 bit has the same 4 general purpose registers (and a couple others) one had in the 80386 chip from 1987. And those mapped 1:1 to the register on the 8086 from the 70's. The 64 bit abi introduces more (real) general purpose registers, and that makes it for the speed gain. Otherwise, 64 bit code simply spends double the cache memory, which makes it, in other architectures, like PPC, be actually slower than 32bit code.

Solution 4

I've been running 64-bit Python on Linux for years already. No problems, it's not buggy. That includes numpy. I wouldn't worry about that.

One small benefit:

Python> sys.maxint
9223372036854775807

However, I suppose one could argue if there is any benefit to that, but the same goes for all 64-bit apps in general.

You get the most benefit when dealing with very large amounts of data.

Solution 5

I'd imagine you'd get performance increases with SciPy and NumPy since number handling and calculations is what gets vastly improved when you add more bits to a processor. But I can't be sure.

You could always run 64 bit and 32 bit side by side and do some comparisons, I'd be interested to know the results of tests on the number crunching side. And if you're running the same version of both, the code will run on both fine provided you have the modules you need for both.

View more solutions