External HDD (3TB) corrupted - possibly partition table damaged

Question

I've got a LaCie HDD (3TB) which I cannot mount because there seem to be issues with the partition table, according to disk utility. I'm on a mac (Sierra, up to date). Disk utility can't repair it, but it lists the hdd. When trying to repair, disk utility says: "Repairing damaged partition table. Operation can't be completed (com.apple.DiskManagement error -69874.) Operation failed..."

In Disk Utility it's listed as:
ST3000DM 001-1CH166 Media, 3TB, not initialized
Location: external
Connection: USB
Partition-table: not supported
SMART-status: not supported
Capacity: 3TB
Number of sub-partitions: 0
Type: Disk. Device: disk2

The disk itself is formatted as Mac Journaled (GUID). My other HDD's have 2 partitions (according to testdisk the damaged disk has a Linux [L] & LBA [E] partition). Tried quick search and deep search on testdisk (it said something like filesystem was OK, partition table damaged. Don't know exactly though).
Used testdisk to write a partition table, but that didn't change anything. Am hesitant to just press some options on testdisk, since I don't want to damage my files further and don't know what all those options do.

The disk has data on it that I really need (when I found out it was corrupted, I just wanted to back the data up to another HDD.. ironic isn't it?), so it would be great if I could retrieve it. The files on it are mainly .PNG, .JPEG, .PSD and .CR2, some video formats and older/mobile image formats too. Think there's a time machine backup on it too.

What should I do? Is there another option with teskdisk, or photorec?

PS: Yes, I know, back-up back-up back-up.
PPS: I've tried contacting specialized businesses but those services are way to expensive for a student like me. Currently a demo of Data Rescue 4 is running to check what can be recovered, but heard that software like that can't give you back files like .psd, and since it's kind of expensive too I'm kind of hesitant to pay this much for something if it can't restore most of my files.

Testdisk wrote a new partition table after analyzing, as well as GParted, but both of those didn't work. Heard about photorec, but would it give me back all the files I have, or only files with specific extensions?

Solution 1

We fixed it. This is in no way a canonical answer, but it may contain helpful information for future visitors.

Everybody had been suggesting to analyse the disk with testdisk.

What analyse does:

Analyzes a drive's current partition structure and finds partitions, making it possible to recover lost partitions.

The problem in our case was not that we were missing partitions, but that the partitions that were there could not be accessed.

We both didn't know anything about data recovery, partition tables, etc. so we started researching and came to a conclusion that there was not something wrong with the partitions, but with the way that they are "indexed" on the disk. We thought this was managed by the partition table.

We have analysed the disk with testdisk multiple times, with different options for the partition table type (we didn't know the type originally, but at the end it turned out to be EFI GPT), hoping that testdisk would be able to find some issues with the partition table that it could restore so that we might be able to access the data again. We let it rewrite the partition table after analysing several times, but it never helped.

---

Prior to trying different potential solutions we didn't know of yet, we decided not to take any risks and bought a new 3TB hdd and cloned the old one onto this.

A strange thing that we noticed was that when we analysed the clone, testdisk only took about a second to display the results, whereas with the old one it would take many many hours. It would also say that it detected the partition table type to be EFI GPT. The old one had been detected as Linux and at least one other type, but not EFI GPT.

This made us think that it was very possible that there was something wrong with the original disk that was hardware related. We thought How can a clone behave differently from the original if there is not?

---

So everything appeared to be correct, but this new disk could not be mounted either. It would say

wrong fs type, bad option, bad superblock on /dev/sdb,  
missing codepage or helper program, or other error  
In some cases useful info is found in syslog - try  
dmesg | tail  or so

and dmesg would say

hfs: unable to find HFS+ superblock

This error has been reported left and right, even here on SU, but none of the proposed solutions (e.g. "specify the size and offset while mounting") worked for us.

While looking for clues we found an option in testdisk's advanced menu called "Superblock" or something similar.

In this menu, there are options to compare the superblock structure to the backup, and to overwrite the backup. First thing we did was to compare the two to see if they were the same.

There was no data in the backup. The hex dump showed only zeroes. We figured what could go wrong by overwriting a bunch of nothing, and let testdisk overwrite the backup structure. It did so and told us to reboot for changes to take effect.

After the reboot, ubuntu showed the disk in the launcher (first time it did that) and when clicking on it, it would mount successfully, allowing us to read the data. ☺

---

Sidenote, as pointed out in the comments:

What many people do not know is that you should NEVER try and restore files to the same drive! Never. Always clone the disk first. More correctly: NEVER WRITE TO the disk you're trying to recover data from.

Solution 2

It's apparently a USB drive, and USB connections are sometimes flakey & get worn out, as a starter have you tried new USB cables & a different port, even on a different computer with known good USB connections?

You should really make a master backup copy of the entire drive, and then do all your data recovery testing on the copy (or on a copy of the master copy if you have the space), if the drive is failing then all this reading could just make it fail faster, and any further failed writes could overwrite your data, and you haven't even recovered any data yet.

Even temporarily borrowing/buying another drive to store an image of the drive would be a safe idea, and be a lot cheaper than paying data recovery pros, and give some room to save the recovered files too.

I think osx is similar enough to linux to have the dd program, so a basic "copy the entire drive" command like this should work:

dd if=[original disk drive] of=/path/to/new/backup/file bs=10M

Where if= is the file to read from (the "in file"), and of= is where to write to (the "out file") - DO NOT MIX THEM UP! bs= means to read & write 10M bytes at a time, sometimes it defaults to 512 bytes which can cause extremely slow progress, 10M (megabytes) should go at a good speed.

If the drive is having hardware errors, then dd could fail, so using gddrescue / "GNU ddrescue" should be better, it can do things like skip "bad" or slow sectors, try reading "backwards", start again from where it left off, it has a nice bag of tricks for troublesome drives. I'm just not sure if it's available natively for OS X, but running it from a live Linux would work.

After that, you can safely unplug & save the original drive just in case you can't recover your files, while trying testdisk/photorec & all sorts of demo/trial "professional recovery" programs on a copy of the drive. Testdisk should work, and if not PhotoRec is very good at recovering files but no filenames or directory structure, they're free too so a great place to start & have good documentation & "how-to's" at their site http://www.cgsecurity.org/wiki/TestDisk

Comments

[Overmind]

In your case I would remove the drive from the LaCie, connect it directly to a SATA port, mount it and then run a file recovery software. After the recovery, I'd let it be scanned for errors and fixed.

[Lisa]

Already connected that way. Testdisk wrote a new partition table based on it's analysis but that didn't help. It said that the data was OK, but the partition table was damaged. So, basically, my data should be fine, but the way of reaching/reading it is damaged.

[Andrea Lazzarotto]

It would be a good idea to try Autopsy as well.

[Lisa]

Don't know the program. What does it do exactly?

[huch]

Don't know it either, but he probably meant: sleuthkit.org/autopsy/index.php It is available eg. via homebrew

[Lisa]

I've already screwed the external HDD apart: the tests are on the HDD connected through a SATA connector. The cables/ports weren't the issue. I already purchased a new 3TB HDD, but if I use the new one as a copy and run tests on, I can't use it to copy new data to through photorec or any other program. So I'd need a third HDD, which I simply cannot afford and no one I know has a empty/old 3TB+ HDD I can use. Which programs would you advice in Testdisk? The quick&advanced scan wrote a new partition table, but that didn't 'fix' the hdd. For making a raw copy, would HDDguru do the trick too?

[Lisa]

Maybe you can enlighten me which should be the right options for testdisk, then I can try again. It says the system is Linux MBR (but it should be Mac Journaled (extended) formatted, so GUID. I could be wrong but since I format all my HDD's the same way).

[Xen2050]

A direct SATA connection should help (as long as removing it is not when the problems started, some external USB cases do funny things & drives don't quite work the same out of them). The nice thing about many stores is they'll let you return a drive within a week or two if you don't like it. Testdisk should list & even copy files from partitions it finds, this looks like a good basic guide to using it, and it'll work if pointed to a disk image too cgsecurity.org/wiki/TestDisk_Step_By_Step

[Xen2050]

With testdisk, if it finds partitions you can list (& copy) it's files, that's a good check to see if it's found a good partition. If it didn't find any partitions/files when trying it with an Intel / MBR partition table, then try it with EFI GPT and see if it lists files then

[Tim]

I have been analysing the disk with Lisa. Testdisk has not been useful. Using it to analyse, both quick search and deeper search find 1 logical partition. When trying to list the files on it, testdisk says "can't list files, filesystem seems damaged". Only thing it can do is (re)write the partition table, which we tried - didn't seem to change anything. But testdisk always says "detected intel partition table type" so we analysed with that. Going to copy the disk and then try EFI GPT analyse, thanks

[Xen2050]

Are there any hardware errors? In Linux they'd show up in dmesg or /var/log/syslog, I think osx has dmesg? If the disk is having read errors that could be an unfortunate clue

[Tim]

This is the output for dmesg i.stack.imgur.com/xhE1m.png can you make sense of it? Running on linux

[Xen2050]

@TimCastelijns Just guessing, but it looks like the drive keeps getting reset, maybe the by the kernel because of read errors or some mysterious reason, the messages between could have more info, or red herrings... Or I remember older computers used to have problems with drives that were too big for them, but 3TB looks like the right size. Maybe just the partition table is saying partitions 5 & 6 (maybe MBR's extended partitions?) are too big for the device, could've been from writing an incorrect partition table earlier. Testdisk is usually the go-to program, not sure what's stopping it

[Tim]

@Xen2050 writing an incorrect partition table earlier. that it almost certainly the case, but we used testdisk to write that new partition table. Do you have any tips left?

[Xen2050]

Was Testdisk any better when looking for / set to GPT? Did Photorec find anything? I'm not sure if the problems stem from the drive itself, so did dd successfully make an image of the drive? Instead of dd, trying "gddrescue" / "gnu ddrescue" should work better on failing drives, it can skip "bad" areas, try reading "backwards", start again from where it left off, has a nice bag of tricks for troublesome drives. Did testdisk/photorec work better on the image? If the image can be read/recovered that would imply the drive itself has problems.

[Lisa]

We wanted to wait until we knew more about the error message, just in case it would impact the copy/testdisk process :). Will go on with the copy on friday (can take a day or two I think) and with the testdisk GPT. We tried copying to a 2TB disk before (didn't have 3TB then), but -since the data is all over the bytes, instead of the beginning- that one was even more corrupted than the current one.

[Lisa]

update: We successfully copied the drive with HDD guru, copy doesn't work (which is not surprising since it's got the wrong partition table too with the byte-to-byte copy). Are going to start analyzing with testdisk on GPT now and if that doesn't work, we're trying Autopsy (As suggested by Andrea Lazzarotto in his comment above) and after that we'll try photorec. We'll try to make screenshots of all information testdisk gives out.

[Tim]

If you are interested, I posted the outcome of our adventure in a separate answer below. By no means do I know what I'm talking about, but I'm thinking that because of a hardware issue with the HDD, the superblock got screwed up. Got a new disk, cloned the old one onto it, fixed the superblock and voila it worked

[Lisa]

A note on this answer: My macbook couldn't read the (new) hdd (it wouldn't mount and disk utility kept loading, the same as when the (old) hdd was in it's casing). So the copying was done with Linux, but I could read the copied (not byte-to-byte but a simple copy paste copy) files on a different hdd on my mac. So I've got my files back :D

[Pryftan]

+1 particularly for this: we decided not to take any risks and bought a new 3TB hdd and cloned the old one onto this. The fact you hadn't any experience with data recovery is all the more a good sign of your abilities. Seriously. What many people do not know is that you should NEVER try and restore files to the same drive! Never. Always clone the disk first. More correctly: NEVER WRITE TO the disk you're trying to recover data from. Actually I think it would be better if you were to edit this in (I don't like editing other people's answers).

[Pryftan]

@Lisa I’m glad you got your file backs. Huge relief I"m sure. Just a note for future (though it's now much further in the future): as the answer says never ever write to the disk you're trying to recover from. Also make sure to test your backups regularly and have a disaster recovery plan. Redundant backups are a good idea and remember that having them connected all the time is a bad idea (rather accessible); yes the idea macOS has is good in theory but risky. I have seen a lot of issues with the handling of USB drives in macOS and it's rather disconcerting but just some tips for you/others!

[Tim]

@Pryftan sure edited it in, thanks for the comments

[Pryftan]

Ta. It's something that is easily overlooked and probably all the more when you're desperate to get your data back; emotions are high, there's panic and then here you have this wonderful program that might be able to recover the files! And then you write to the same disk and you can make the situation even worse. Not good but I imagine all too common too (but even one instance of it is too common in my book).

[Pryftan]

@TimCastelijns Btw depending on what you're trying to do when you do a bit by bit copy you can duplicate the problem. So the real point of having a duplicate is so that you can make absolute certain that just in case there is a problem you have a backup of the corrupted data. You can see how useful that is: if the corrupted volume is destroyed further then what happens? But if you try and recover the duplicate you're safe (as long as you are diligent with consistency). Of course you still have to make sure to not write on the disk!

[Pryftan]

And yes macOS does have dd. But something that is absolutely vital is that you make absolute certainty you do not have the devices mixed up! That's one of the reasons it's safest to have a duplicate first but that's a general rule whereas for dd you still have to be careful even when you're not trying to recover data. And the same rule applies of course to not writing to the drive you're trying to recover.