caches vs paging

Question

So I'm in a computer architecture class, and I guess I'm having a hard time differentiating between caching and pages.

The only explanation I can come up with is that pages are the OS's way of tricking a program that it's doing all it's work in a specified region of memory, vs a cache memory is the hardware's way of tricking the OS that it's reading from one specified region of memory, when it's really not.

Does the os direct the hardware that it needs a "new page" or is that taken care of by the os trying to read the address that is "out of range" of the current cache "page" (for lack of a better term).

Am I on the right track or am I completely crazy?

Solution

Caching and pages are orthogonal concepts.

A cache is a high-speed "memory" that acts to minimise the number of accesses to a large low-speed "memory". In the most general sense, the high-speed "memory" could be your hard disk acting to cache web pages fetched from the web (low-speed "memory"). Of course, in the context of computer architecture, the term "cache" is more likely to refer to physical RAM used to speed up access to slower RAM or disk.

Pages, OTOH, are simply a unit of management for the contents of RAM or disk.

These two concepts come together in implementing virtual memory systems. A process may allocate 500 MB of memory. This may be more that the physical RAM available to give to the process, so the operating system allocates blocks on disk called pages, which will hold the contents of certain logical pages in the process's address-space.

When the process accesses a location in its address-space, and the associated page isn't currently mapped into physical memory, the CPU signals a page fault, and the OS responds by fetching the page from disk while the process is in a suspended state. Once the page is mapped, the process resumes and is able to access that memory location as if it was there all along.

The common view that virtual memory is a way of tricking the process into thinking it has tons of RAM isn't the only way to think about this. You could also think of a process's address-space as being logically stored on disk pages, with the OS-assisted mapping into RAM being just a way to cache the contents of those pages such that the process isn't continually accessing the hard drive. In this sense, caching and paged virtual memory are logically the same thing. Just keep in mind that, while this viewpoint may help to understand the relationship between the two concepts, it isn't entirely accurate, since it is possible to run without virtual memory at all, just physical memory (in fact, most embedded systems run this way).

Comments

[onaclov2000]

OK, I think I had it in my head that basically a page resides in a cache "set" (assuming set associative cache, where there are multiple cached blocks). and thus when a address is called to the OS by the program and it was out of "range" then the OS would call the address to the cache which would cause a miss, and then force it to get it from main memory, but what I gather you're saying is that pages generally reside on RAM, but are program defined not hardware defined... I think that helps! Thanks a lot.

[onaclov2000]

So does that mean that if a page doesn't exist on the RAM, and a page fault occurs it would logically get it from cache or would there still be a disk access? (the cache doesn't "predict" which page will be needed next right?)

[Alex S]

In the context of virtual memory, there is no cache between the RAM page and the backing store (disk page). The physical RAM is essentially acting as a cache for pages that reside on disk. If a page fault occurs, it's off to the platter you go (or maybe SSD chips these days).

[Alex S]

One point I forgot to mention is that on many OSes, the filesystem cache (usually called the page cache) is implemented using the same mechanism as the virtual memory subsystem, such that paging RAM out onto the page file is practically indistiguishable from flushing a cached portion of a user-opened file to disk.

[Eric Aya]

Did you read the accepted answer before posting? :) Do you think that yours adds anything new or explains it better? Please improve your answer if you can. Thanks.