Is the file table in the filesystem or in memory?

Question

In the context of operating system control tables, does the term "file tables" refer to a data structure that is part of the filesystem, or that is in main memory (and in which case I assume it would only have references to open files)? My textbook¹ says,

The tables provide information about the existence of files, their location on secondary memory, their current status, and other attributes. Much, if not all, of this information may be maintained and used by a file management system, in which case the OS has little or no knowledge of files.

Also, what is a file management system? Does that mean the filesystem?

¹Stallings, Operating Systems, 7^th ed., p. 127

Solution

It is unclear without further context to determine if Stalling is talking about the in-memory inode table or the tables with in the filesystem. I lent a much earlier edition of book to someone, but never got it back; so I can't look up the context myself.

There are three "file tables", but the one being discussed here is more commonly called the "in-memory inode table"; the second is commonly called the "open file table", and exists per process. Both tables are in kernel memory and not accessible to a program. The third "table" is really two sets of tables within the filesystem (on disk), the first is the on-disk inode table and the second are the data blocks themselves (note: this discussion concerns traditional UNIX filesystem management, newer systems can have different organizations). Entries in the inode table have sequences of references into data blocks that contain either indirect reference blocks or actual data. The key to a file on the filesystem is the inode, not the data blocks themselves. When Stalling is talking about an on-disk "file table," it will generally be the "smaller" table on disk that denotes files, such as the inode table or the block definition table in FAT systems.

In terms of the in-memory inode table, the inode is loaded from the file system, its st_nlink value is incremented and then made accessible to the rest of the system, when the inode data is written to disk, the st_ctime is updated. If the inode is no longer needed in memory, the st_nlink value is decremented and the entry in the table is marked as free. Every process will start with references to about three or five entries into the in-memory inode table: the inodes of stdin, stdout, stderr - these are often a device file (tty) - and then references to the current directory and the root directory. An inode will only reside in the table once, so there may be multiple references to a single inode in the table.

The open file table is kept per process and contains references into the in-memory inode table as well as pointers to buffers, and state information (like fseek(2) value and flags from open(2). The file descriptor is literally an index into the open file table; but most people refer to the entry in the open file table when talking about the "file descriptor".

When a file is opened using open(2), an available entry in the open file table is found, the inode of the file reference by the pathname is determined, that inode is loaded into the in-memory inode table, if not already loaded, the st_nlink count is increased and the inode entry is referenced in the file descriptor, flags are set and buffers are allocated. When closed, the reverse occurs.

The routines within the kernel are called the "file management system" and the "filesystem" is the organization on disk. These days there are a number of 'plugable' modules that can be loaded (modprobe(8)) into the file management system for different organizations on disk. For example, there are ext2/ext3/ext4 filesystem types, and each of them have a different module in the kernel's file management system; the same with ntfs, sbfs, nfs, vfat, jfs, etc.

This is a bit more long-winded than I originally intended, so I'll stop here.

Comments

[Amir]

This is an incredibly good answer.

[Lizardx]

Yes, please don't stop yourself when you are providing such a good, comprehensive answer.

[Juraj Martinka]

I'm confused by the "open file table" which you describe as "per process". Isn't that a system-wide table (see stackoverflow.com/questions/14189944/unix-system-file-tables‌​) and the per process table is actually "File descriptor table"?

[Yves]

please get back your book and lend it to me

[Yossarian42]

in Unix, each process has its own file descriptor array for every connection it opens; also there is an open file table kept by OS, since the same file may have many open connections. see this notes (usna.edu/Users/cs/wcbrown/courses/IC221/classes/L09/Class.h‌​tml), might clearify these confusing concepts.