What is the difference between 32-bit PAE and 64-bit kernels?

Question

I ask this only in regards to memory addressing.

I know that a PAE kernel allows the OS to access more memory than a standard 32-bit kernel, however, what are the other implications? What specific differences are there between a 64-bit kernel and a 32-bit PAE kernel?

According to Wikipedia, a processes address space remains at 32-bits, meaning it can only access a maximum of 4GB of memory. The OS however can access a 64GB address space, allocating 4GB chunks to processes.

To me it seems like this is a big distinction that seems to be ignored by many people.

Solution 1

The kernel sees the physical memory and provides a view to the processes. If you ever wondered how a process can have a 4 GB memory space if your whole machine got only 512 MB of RAM, that's why. Each process has its own virtual memory space. The addresses in that address space are mapped either to physical pages or to swap space. If to swap space, they'll have to be swapped back into physical memory before your process can access a page to modify it.

The example from Torvalds in XQYZ's answer (DOS highmem) is not too far fetched, although I disagree about his conclusion that PAE is generally a bad thing. It solved specific problems and has its merits - but all of that is argumentative. For example the implementer of a library may not perceive the implementation as easy, while the user of that library may perceive this library as very useful and easy to use. Torvalds is an implementer, so he's bound to say what the statement says. For an end user this solves a problem and that's what the end user cares about.

For one PAE helps solve another legacy problem on 32bit machines. It allows the kernel to map the full 4 GB of memory and work around the BIOS memory hole that exists on many machines and causes a pure 32bit kernel without PAE to "see" only 3.1 or 3.2 GB of memory, despite the physical 4 GB.

Anyway, for the 64bit kernel it's a symmetrical relation between the page physical and the virtual pages (leaving swap space and other details aside). However, the PAE kernel maps between a 32bit pointer within the process' address space and a 36bit address in physical memory. More book-keeping is needed here. Keyword: "Extended Page-Table". But this is somewhat more of a programming question. This is the main difference. More book-keeping compared to a full linear address space. For PAE it's chunks of 4 GB as you mentioned.

Aside from that both PAE and 64bit allow for large pages (instead of the standard 4 KB pages in 32bit).

Chapter 3 of Volume 1 of the Intel Processor Manual has some overview and Chapter 3 of Volume 3A ("Protected Mode Memory Management") has more details, if you want to read up on it.

To me it seems like this is a big distinction that seems to be ignored by many people.

You're right. However, the majority of people are users, not implementers. That's why they won't care. And as long as you don't require huge amounts of memory for your application, many people don't care (especially since there are compatibility layers).

Solution 2

You might want to look into what Linus Torwalds says about it here:

PAE turned that very simple fact on its head, and screwed things up royally. Whoever came up with the idea was totally incompetent, and had forgotten all the DOS HIGHMEM pains. There’s a damn good reason why we left the 286 behind, and started using 386′s, instead of having HIGHMEM crap with windows into a bigger physical space.

[...]

So repeat after me: PAE didn’t ever really fix anything. It was a mistake. It was just a total failure, and the result of hw engineers not understanding software.

Comments

[ChrisCornwall]

A 32-bit PAE kernel can access more physical memory than a 32-bit non-PAE kernel. But it can't run 64-bit code and has the same virtual memory limitations as any other 32-bit kernel.

[Mr. Shickadance]

Interesting...having been shown the DOS HIGHMEM assembly code by a professor, I can at least agree that it was a pain.

[Mr. Shickadance]

So, just to be pedantic, a process running on a 32-bit PAE kernel can only access a maximum of 4GB of memory? That was my main point, while the address space is extended, its not like some memory-hog of an application is going to be able to utilize all 64GB. Thanks for the detailed response, but I want to leave this open just for more comments.

[0xC0000022L]

@Mr. Shickadance: indeed. For all practical purposes and as a user this is all that is relevant. Just wanted to make sure it's understood that there is a logical separation between virtual address space and physical address space.

[ChrisCornwall]

@Mr.Shickadance No, that is totally incorrect. A process running on a 32-bit PAE kernel only has access to 4GB of virtual memory, but it can access all of physical memory. PAE removes the kernel limitations on physical memory, and user-space never access physical memory directly, only the kernel does.

[0xC0000022L]

@David : that's right. And even with this limitation of the virtual address space we can cheat as long as the system provides anonymous memory-mapped files. This way we could still use beyond 4 GB, just not at once.

[peterh]

PAE makes 2-4GB Linux laptops upgraded to 8GB working better than the 16GB 64bit windows 10 laptops. It is because the sad fact is that 64-bit software uses about 2x more RAM because nearly all of the data is pointers. And practically no one needs 4GB RAM/process today.