How to format a 4k sector hard drive?

partitioning hard-drive gparted

64,800

Solution 1

It reports 512 bytes because that is the logical sector size for backward compatibility with older OSes ( Windows ). (g)parted aligns partitions to 1 MiB, which is more than enough to work properly with drives using 4k physical sectors, so it is fine the way it is.

Solution 2

First, most Advanced Format drives present a logical sector size of 512 bytes, even though the physical sector size is 4096 bytes (4KiB). This is done through juggling data in the drive's firmware, and it can result in performance problems if the partitions are not properly aligned. Chances are this isn't the source of your problem, though, and your focus on partition alignment is misplaced. (To be sure, partition alignment is important; it's just not the source of your problem.)

My hunch (and it is just a hunch) is that your problem results from switching between a USB enclosure and direct connection of the disk. Some enclosures translate 512-byte logical sectors on the disk into 4096-byte logical sectors presented to the computer -- that is, the opposite of what the firmware in an Advanced Format disk does. I'm not positive, but I suspect that some enclosures do this only on over-2TiB disks. Both MBR and GPT partitioning schemes refer to data by sector numbers, so changing the sector size invalidates the partitioning data. Thus, if you prepare the disk in a USB enclosure that translates in this way and then try to use the disk directly (or vice-versa), you'll see errors because the partitions (and even GPT backup data) won't be where the computer expects it to be. It'd be like telling a blindfolded person to "walk 2 forward," where you mean 2 feet, but the person thinks you mean 2 meters, and so walks into a wall.

The solution to this problem is to prepare and use the disk in one way -- either use the USB enclosure or use a direct connection, not both. If both are necessary for some reason, you'll need to find an enclosure that works without applying this type of translation.

To test my hypothesis, you can use a tool (like parted) that reports logical sector sizes. For instance:

sudo parted /dev/sda print
Model: ATA Hitachi HDP72505 (scsi)
Disk /dev/sda: 500GB
Sector size (logical/physical): 512B/512B

I've omitted a bunch of information from this output. The important detail is in the final line shown here: The logical and physical sector sizes are both 512 bytes. If you run this test, then reattach the disk in the other way and run it again, you can compare the results. If the logical sector size changes depending on how you attach the disk, then my hypothesis is correct and it's the source of your problem. Or at least, it's one problem; there may be others, too. In particular, the Error synchronizing after initial wipe error you mentioned could indicate another problem. OTOH, that could be a side effect of parted trying to read backup GPT data from beyond the end of the disk. Either way, if you see a change in logical sector size, you should address that issue before dealing with anything else. If you want to repartition the disk but have problems because of the Error synchronizing error, try using gdisk or cgdisk (in the gdisk package) rather than parted or GParted; the gdisk family is more robust against errors than is the parted family.

Solution 3

Changes have been made to both the Linux kernel and utilities to support Advanced Format drives. These changes ensure that all partitions on Advanced Format drives are properly aligned on 4K sector boundaries. Kernel support for Advanced Format drives is available in kernel versions 2.6.31 and above. Support for portioning and formatting Advanced Format drives is available in the following Linux utilities:

Fdisk: GNU Fdisk is a command line utility that partitions hard drives. Versions 1.2.3 and above support Advanced Format drives.

Parted: GNU Parted is a graphical utility for partitioning hard drives. Versions 2.1 and above support Advanced Format drives.

4K native (4Kn) disks have a 4-KB logical sector size. Modern operating systems can store data on these disks, but they generally cannot boot from these disks. These disks are commonly external drives with USB connection.

To create a partition used as a media storage device, just press Ctrl+Alt+T on your keyboard to open Terminal. When it opens, run the command(s) below:

sudo mkfs.ext4 -m 0 /dev/sdb1

Most of the 4k disks have a physical sector size of 4096 bytes so that 3TB can fit on the plates, but they still have 512B logical sector size for compatibility with OSs that expect one sector to be 512B. That's why when you use the print command in parted to print the details, is says sector size logical/physical 512B/4096B. That's normal.

This "conversion" is done by the intelligence on the disk itself, and it's normal. It's how it should work.

If you can't see the whole 3TB first check in BIOS that it's not limiting the disk size. Maybe you need bios update to support the disk.

Also using parted to create partition starting from 0 which is a mistake. You need to start the first parttion on sector 2048 which is equivalent to 1MiB so that the partitions on the disk are aligned for optimal performance.

So, change the unit to MiB, print the disk details which will also print the disk size in MiB, and create the partition (if you only want single partition) starting from 1MiB and ending at the last MiB (or one before the last).

For example, a 3TB disk should have in reality something like 2,861,022 MiB. So creating a single partition spanning the whole disk on a disk device called /dev/sdX would be like (replace the X with the correct letter):

sudo parted /dev/sdX
unit MiB
mklabel gpt
print (to see the total size in MiB)
mkpart primary 1 2861022
quit

If the disk size is few MiB more or less than 2861022 adjust the ending location.

That's it. After that create the filesystem you want on the partition.

^{Sources, and extra reading}

^{4K-sector drives and Linux}

^{Error check and install 4k drives in Linux}

^{Ubuntu Forums}

64,800

qwazix

Updated on September 18, 2022

Comments

qwazix over 1 year
I bought a Seagate ST3000DM001 3TB hard drive. It was sold inside an enclosure. I formatted to ext4, copied some data to it and then I removed it from the enclosure and connected it directly to the motherboard. My data wasn't visible to OS (it was seeing one unknown ~300GiB partition and the rest as free space) so I started reading about Advanced Format and >2TB drives.

I connected the disk with USB again, copied all the data, and plugged to sata once again. I used gparted to create a GPT partition table (the ubuntu disk utility wouldn't let me with Error synchronizing after initial wipe: Timed out waiting for object (udisks-error-quark, 0), created one big 3Tb Ext4 partition, left the default Align to value to MiB and formatted.

The drive is recognized correctly, and mounts properly. Files can be copied from and to the drive successfully.

However:

Gparted shows the disk having 512k sectors which I know is not true¹

while cat /sys/block/sdb/queue/physical_block_size reports 4096

I have found some reports that libparted has a problem with non-512k sectors² and others³ that say that aligning to MiB is enough.

Parted reports that the drive is aligned when I choose minimal:
```
(parted) align-check 
alignment type(min/opt)  [optimal]/minimal? min                           
Partition number? 1                                                       
1 aligned
(parted)    
```
Is my configuration correct? Do I have performance penalty because the disk is formatted with 512k sectors, is gParted reporting false values or I have understood the whole thing wrong?
qwazix over 10 years

Thanks for the response, but it doesn't really answer my question. I get that the kernel recognizes the sector size of the drive and that gparted is supposed to recognize that too, but it reports otherwise and I am curious if there is an explanation to that, if it's a bug, or if I have done something wrong.
qwazix over 10 years

I do believe that the problem is because of switching between usb and internal. Indeed that was my initial thought and that is why I re-created the partition table after putting the disk back into the pc and connected with sata. (I took it out in the first place just to take my data out). I don't want to remove it again to test the logical sector reported via USB but all facts suggest this.
qwazix over 10 years

I do not intend to keep switching interfaces, the disk works as it is now, prepared under SATA connection. I am just worried about the performance penalties you mentioned and I want to understand what really happens. parted /dev/sdb print returns 512/4096 so it seems that gParted reports the logical sector size which explains why @LiveWireBT sees 4096 for the same disk (connected via USB) and also confirms your hunch as above.
gaborous about 9 years

Had the same problem with a Verbatim Store'n'Go external drive, the logical sector size is 512 while the physical size is 4096.