How to calculate proper amount of inode/block sizes for a linux filesystem

Question

I have an old reiser filesystem which I'm going to convert to Ext3. The problem I have is to determine the proper block- and inode-sizes for this partition.

The partition is 44 GB large and has to hold 3,000,000+ files of sizes between 1 kb and 10kb, how can I figure out the best ratio of inodes and blocksize?

The below is something I tried which seems OK but makes the copying files incredibly slow.

mkfs.ext3 \
 -t ext3 \
 -c \
 -c \
 -b 1024 \
 -i 4096 \
 -I 128 \
 -v \
 -j \
 -O sparse_super,filetype,has_journal\
  /dev/sdb1

Thanks.

Solution 1

There is no "right" answer to such a limited question. If your setting up a desktop or laptop, just do:

mkfs.ext3 /dev/sda1

The program can choose way better than an inexperienced human being. If your going to store a huge website, ext3 might require up to 25% inodes, because of the massive possible number of small individual files.

But you're really only telling the mkfs program "up to 25% inodes." Most file system creation settings are used in specialized applications, such as RAID striping, where the geometry of the file system has to be tuned or it will be incredibly slow.

I know people have their pet file system settings. I have yet to find anyone who can demonstrate the usefulness of departing from default values on a 40 GB desktop or laptop partition.

And ext3 is not the best file system for everything! I use three different partition format types on my lappy, because certain partitions hold many small, individual files. Others hold a much smaller number of larger files (good for ext3 system), like /home partition.

/usr can have 500,000 individual files, making ext3 klunky, but reiserfs flies. The other thing is acls. You must inform mkfs.ext3 that you'll be using acls, if you'll be using them. acls are fine tuning for permissions, usually not important on a single user system. But if you have a group of 20 normal users, and you want differing access controls for some of them, you must use acls.

I personally like xfs for a general use filesystem, although it doesn't work out of the box with SELinux. But it's the most sophisticated and efficient file system. It's used on several brands of HPCs. You can google it if you so please. ext3 uselessly wastes 8% of the partition. That's outrageous.

But ext4 is better. I'm not going to spit out a command line for you. You must determine the use of the partition, and design the file system accordingly. ext3 is reliable, but so is a tank. That doesn't mean you want to drive one around town.

I hope this helps a little.

Solution 2

You'll need an inode for each file, the size of the inode will define the number of direct blocks it can reach before requiring another inode and enough blocks to handle the files.

So, in your case, the minimum number of inodes will be the 3.000.000+. If you use the default value, you'll get 12 direct blocks. So, with a block size of 1kB you'd get what you need.

Obviously you can reduce the inode size and increase the block size to increase the number of inodes. This will get you less space available, but more files in the same filesystem.

Comments

[Admin]

try running with default options (no -b or -i or -I, etc) but use the -n flag. this will do a "dry-run" and won't create the filesystem, but will tell you how many inodes & what blocksize mkfs would create by default.

[hotei]

Probably not significant but there's an inode required for each directory as well. Probably nowhere near the 3.0e+6 used for files, but you never know.