For a 32-bit kernel, what are the pros/cons of PAE vs. non-PAE?

Question

I am using Ubuntu 12.04 32-bit. My kernels are PAE kernels. I know these are the kernels we can use for 32-bit OS which have 4 GB RAM, to utilize all 4GB of RAM.

My system is a 32-bit system. So I thought I don't need them. What are the advantages and disadvantages I am going have if I keep running PAE kernels?

raja@badfox:~$ uname -a
Linux badfox 3.2.0-24-generic-pae #39-Ubuntu SMP Mon May 21 18:54:21 UTC 2012 i686 i686 i386 GNU/Linux

Solution 1

Short answer: Go 64-bit if you can, which is faster than both PAE/non-PAE 32-bit; if you cannot, non-PAE may be 1-2% faster than PAE.

---

In theory PAE has a slight overhead over non-PAE...

• The big advantage of PAE is allowing a 32-bit processor/system to use more than 4GB of RAM
• But this requires slightly more overhead over non-PAE, which can lead to slightly decreased performance.
  • Here's a very simple explanation: in non-PAE mode, a 32-bit CPU must lookup (access) two tables to access a physical memory address; in PAE-mode, it must lookup three tables to do so. The one additional lookup requires some (very small) extra time, thus imposing additional overhead.
  • At the end of this answer are two images from the Wikipedia PAE article, illustrating the above point.
  • NX/XD bit: The PAE kernel also supports the No-eXecute/eXecute-Disable bit on 64-bit processors; this can help prevent some kinds of virus/malicious attacks (buffer overflows), but IMO this doesn't matter much when choosing 32-bit kernels for Ubuntu.

...but in practice this overhead is negligible (almost nothing)...

• Phoronix has done a number of tests over the years which show that on systems with 4GB or less, the PAE kernel may be at most approximately 5% slower than the non-PAE kernel. This is only for a specific test application; the usual difference is less than 1%.
  • Tests for 12.04 LTS - 8 GB system!
  • Tests for 11.04 - 4 and 8GB systems

and 64-bit almost always beats BOTH 32-bit kernels - go for it!

• The one thing those Phoronix benchmarks above tell you is that 64-bit is king -- even if you have less than 4GB of RAM (although I'd recommend 1GB as the minimum)
• If you have a 64-bit processor -- generally anything after 2006 except Intel Atoms -- you are probably losing performance by using a 32-bit kernel!

---

Comparing non-PAE vs. PAE page table accesses:

1. Non-PAE

   

2. PAE

   

Solution 2

For some real-world data, here are some benchmarks I gathered for i386, i386-pae and amd64 installs on an Atom based netbook and Sandybridge based laptop:

http://kernel.ubuntu.com/~cking/power-benchmarking/blueprint-foundations-p-64bit-by-default/hpmini-and-x220-tests/results-3/results.txt

..may give some insight into pros and cons.

Comments

[Deepak Verma]

So, if you have less than 4GB of RAM, are you saying 64-bit is faster than 32-bit? Why is that? Also, you make no mention of compatibility problems with 32-bit software that either has not yet been converted, or has bugs in the conversion - do you think this is ever a problem? It seems like it was, but perhaps this is a thing of the past.

[mx7]

Of course above comment have a point to think .

[Eliah Kagan]

@MartyFried Sometimes 64-bit is faster than 32-bit because compilers do things better for 64-bit. I don't remember the details, so maybe someone who does can expand on that, but I think it pertains to the biggest chunk of contiguous memory (even well less than 2GiB/4Gib) a program can allocate.

[Eliah Kagan]

The other things to keep in mind are (1) 64-bit may be faster because of the use of instructions added in the AMD64/EM64T instruction set (whereas with 32-bit most programs are compiled to the lowest common denominator for, at best, i686), and (2) 64-bit may be faster on machines with considerably less than 4 GiB of RAM because memory in RAM and memory swapped to disk are both part of each program's virtual memory space, and that virtual memory space is addressed with 32-bit pointers on a 32-bit system and 64-bit pointers on a 64-bit system. Pehraps izx can speak to some of these issues.

[ish]

Thanks for the comments @MartyFried and Eliah. I will update the answer in detail tomorrow, but here's the gist (Eliah is pretty much on the money): 1. "unconverted" 32-bit binaries run as fast on 64-bit, but will need all 32-bit dependencies (the former ia32-libs, etc.) . 2. A program written for 32-bit but compiled for 64-bit will most probably be faster, in varying degrees, because the compiler can take advantage of certain features in the AMD64 architecture (regardless of memory requirements). and 3. Programs written/optimized for 64-bit will take full advantage of it and be faster.

[ish]

@EliahKagan: do you think this Q might be a good candidate for a canonical Q: "I have a 64-bit processor but less than 4 GB of ram. Which kernel of the three should I choose?"

[mx7]

Thanks @izx & Eliah Kagan & Marty Fried , really awesome explanation about what then do and what they can't do .

[Deepak Verma]

@izx: I think that would be a good topic. I have a 64-bit processor with 4GB of RAM, and in the past, avoided 64 bit versions due to isolated problems or 32-bit only programs/drivers, etc. Also, Canonical always recommended the 32-bit version on the download page. I'm a retired Windows programmer, and I know the problems with programs that assumed 32-bit integers, etc, and I know that it's sometimes hard to catch all the naive programmer mistakes, so I've been straddling the fence on whether to go 64 bit for possibly little to no gain. I have more questions than suitable here.

[rsjethani]

"Canonical always recommended the 32-bit version"(from Marty) why is that so?

[phuclv]

@MartyFried x86_64 are faster even on systems less than 4GB of RAM because of reasons mentioned in here. Generally because the double register number and implied SSE2. The same to ARM64 because of more registers and a refreshed instruction set