How do I find the offset of an ext4 filesystem?

Question

I have a failing hard drive that is unable to write or read the first sectors of the disk. It just gives I/O errors and that is all there is. There are other areas on the disk that seem (mostly) fine. I am trying to mount a partition (ext4) and see if I can access some files I would like to recover. Since the mount command supports an offset option, I should be able to mount the filesystem even though the partition table is unreadable and unwriteable. The problem is how to find the offset. None of the ext4 tools seems to have this particular feature.

Solution 1

There isn't a standard offset per-se, as of course you can start the partition wherever you want. But let's assume for a moment that you're looking for the first partition, and it was created more or less accepting defaults. There are then two places you may find it, assuming you were using a traditional DOS partition table:

1. Starting at (512-byte) sector 63. This was the tradition for a very long time, and worked until someone came up with 4K disks...
2. Starting at (512-byte) sector 2048. This is the new tradition, to accommodate 4K disks.
3. A bonus option! Sarting at sector 56. This is what happens if someone moves the 63-start partition to make it align with a 4K sector.

Now, to proceed, you'll want to pick up your favorite hex-dump tool, and learn a little about the ext4 Disk Layout. In particular, it starts with 1024 bytes of padding, which ext4 ignores. Next comes the superblock. You can recognize the superblock by checking for the magic number 0xEF53 at offset 0x38 (from the superblock start, or 0x438 from the partition start, or 1080 in decimal.) The magic number is little-endian. So it's actually stored on disk as 0x53EF.

Here is what that looks like with xxd -a:

0000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................ * 0000400: 0040 5d00 0084 7401 33a0 1200 33db a600 .@]...t.3...3... 0000410: 4963 5300 0000 0000 0200 0000 0200 0000 IcS............. 0000420: 0080 0000 0080 0000 0020 0000 6637 0952 ......... ..f7.R 0000430: 6637 0952 0200 1600 53ef 0100 0100 0000 f7.R....S....... 0000440: 9938 f851 004e ed00 0000 0000 0100 0000 .8.Q.N..........

Note, that when you give the offset to mount (or losetup), you must give the offset to where the padding starts—not the superblock.

Now, if its not the first partition, or otherwise isn't in one of the two (three) expected spots, you basically get to search for the magic number 0xEF53. This is what testdisk (recommended in a comment) does for you.

Solution 2

Based on @derobert's answer, I wrote a program (gist) that will parse an input stream from dd and scan each sector for something that looks like the start of an ext partition.

It will work at least as fast as dd can read from your hard disk. An abridged version is below.

The simplest usage is just sudo dd if=/dev/xxx | ext2scan, although you will likely want to modify the dd command to improve the block size or choose a region to search.

#include <unistd.h>
#include <stdio.h>
#include <string.h>

int main() {
  unsigned char const MAGIC[2] = {0x53, 0xef};
  unsigned char const ZEROS[512] = {0};

  long long int sector = 0;

  char buf[4][512];
  int empty1, empty2;

  while (read(STDIN_FILENO, buf[sector&3], 512) > 0) {
    if (!memcmp(buf[sector&3] + 0x38, MAGIC, 2)) {
      printf("Found a possible ext2 partition at sector %lld", sector-2);

      empty1 = !memcmp(buf[(sector-2)&3], ZEROS, 512);
      empty2 = !memcmp(buf[(sector-1)&3], ZEROS, 512);

      if (empty1 && empty2) printf(" (first two sectors are empty :)\n");
    }
    sector++;
  }
}

Note: it will find not just the start of partitions, but also superblocks within them.

In either case, I would recommend using dumpe2fs to analyse the results. You can dump the start of the suspected superblock to a file (at least the first six sectors, according to my informal test), and if it is a superblock, then dumpe2fs will tell you (among other things) the relative locations of the other superblocks.

Solution 3

Make a guess as to where the partition starts and apply some brute force:

bsz=512 # or 1024, 2048, 4096 higher = faster

for i in {2..10000000}; do
    echo "--->$i<---"
    mount -o offset=$(($bsz*$i)) -t ext4 /dev/whatever /mnt/foo
    if [ $? == 0 ]; then # whahoo!
        echo Eureka
        break
    fi
done

I imagine this could take some time, but if you've already spent 6 hours with testdisk, maybe it is worth a try.

Solution 4

Try different option (eg, using debugfs, and fsck.ext4):

debugfs:

You have to mount debugfs first (not the failing harddisk itself):

http://johnsofteng.wordpress.com/2013/11/20/sysfs-procfs-sysctl-debugfs-and-other-similar-kernel-interfaces/

http://afzalkhanlinuxtalk.wordpress.com/2013/08/07/how-to-recover-deleted-file-in-linux/comment-page-1/#comment-8

http://blesseddlo.wordpress.com/2010/10/12/using-debugfs/

(essentially, is to use "debugfs -w" with write-enabled mode, and then followed by "lsdel" to list out all the deleted files). alternatively you can use

and here is fsck.ext4:

http://linuxexpresso.wordpress.com/2010/03/31/repair-a-broken-ext4-superblock-in-ubuntu/

Another is "sleuthkit" ("sudo apt-get install sleuthkit") which have command like "istat" for providing blocks info about the inodes - from which you can get the offset and thus block data content easily.

https://www.ibm.com/developerworks/cn/linux/l-cn-ext4resize/

(BTW, if the blocksize is 1024, from debugfs's "show_super_stats" command, then it follows that block 1 is 1024 bytes offset from start of the disk, and each block group can have several blocks too.)

Solution 5

I had an e-book firmware image which included ext3fs partition image, in order to mount and edit that I had to scan image using bgrep tool to find all positions of ext3fs magic number 0x53EF and try to mount using found offsets.

Here is a shortened script which performs mounting:

#!/bin/sh
FW_IMAGE=$1
MOUNT_POINT=$2

FS_TYPE=ext3
EXTFS_MAGIC_NUM=53ef
MAGIC_OFFSET=1080

OFFSETS=`bgrep $EXTFS_MAGIC_NUM $FW_IMAGE | awk '{print toupper($NF)}'`
for OFFSET in $OFFSETS; do
  OFFSET=`echo "ibase=16; $OFFSET" | bc`
  OFFSET=`expr $OFFSET - $MAGIC_OFFSET`
  sudo mount -t $FS_TYPE -o loop,offset=$OFFSET $FW_IMAGE $MOUNT_POINT 2>/dev/null
  if [ $? -eq 0 ]; then
    echo "Success!  Offset is: $OFFSET."
    break
  fi
done

Complete script located here.

View more solutions

Comments

[Admin]

Try testdisk and the accompanying photorec.

[Admin]

@jippie it took testdisk 6 hours to scan the whole disk and in the end it didn't find any partition. i think the best strategy is to try to find the location of the filesystem directly and mount it.

[Ernest A]

i don't see how this method could help me to figure out where the file system starts.

[derobert]

Hah, that is a lot of brute force!

[goldilocks]

It works but it is slow; I tried this on an multi-partition image whose offsets I knew, so I could start it fairly close. Threw in the echo "--->$i<---" line because of that since otherwise it's impossible to gauge the progress. I think you could increase bsz to 4096, which will speed things up.

[derobert]

You could speed it up a lot if you assume traditional layout where partitions start on a "track" (or is it cylinder?) boundary.

[Ernest A]

SUCCESS!!! I had to write my own script. testdisk wouldn't find it. Thank you all for the help.

[Ernest A]

my estimate was to poor for this solution to be practical but can work in other circumstances

[mwfearnley]

Based on this, you can use something like dd if=/dev/sda skip=$start_sector | xxd -a | grep '[02468]30: .... .... .... .... 53ef' to get some probable matches. Probably not very fast, but you can let it run while you find a better method.

[mwfearnley]

See my answer below for the "better method" now. Note: just scanning for this number in random data will find a false positive every 65536 sectors (32 MB).

[Jan-Stefan Janetzky]

thanks for this. due to the fact it took me a second read to notice testdisk, i just had to add an edit for a tl;dr: header

[stark]

Maybe use dd options iflag=direct bs=1M to speed up reads.