Is "dd" a reliable tool to make an SSD image?

Question

I want to back up my SSD using the Linux dd command, but I'm not sure how reliable that method will be. I think I read somewhere that dd does not check for or report errors, so obviously if true then it will be a deal breaker.

This will be the command:

sudo dd status=progress bs=512K if=/dev/nvme0n1 of=/media/d/ssd.img

So please explain how reliable the dd command can be for said use case.

And, are there any more reliable and/or easier alternative?

Solution 1

TLDR: Use ddrescue

It supports resume/continue capabilities, has automatic logs, and tons of other options. More at the ddrescue home page.

Example syntax:

ddrescue /dev/sde yourimagename.image sde.log

IF you want to (given your comment mentioning restoring) restore the image from the command above onto another drive of the same exact size:

ddrescue -f yourimagehere.image /dev/sde restore.logfile

Furthermore, it is faster than dd is -- at least it does look like it is when comparing speed of ddrescue and dd + pv.

Solution 2

The original comment about 'not check for or report errors' probably came because by default 'dd' does not pad out bad reads, so for block-orientated devices not only the bad block but all subsequent blocks will be incorrect (because they're no longer aligned). As others have said, this is fixable using the 'conv=noerror,sync' option, which tells dd to ensure blocks are remain on block boundaries. It should be joined by a block-size setting matching the filesystem block size, which is often 4096 bytes but can be lower.

I would agree with Johan Myréen's comment about using file backup, because the granularity of the backup is much smaller - an error backing up one file doesn't necessarily affect the others. You could also use a file system that uses error-correction on the file data (such as zfs, btrfs, and some configurations of others), so at the very least you know when errors happen and hopefully can fix them.

Another way to detect bad backups would be to use a message digest hash code, e.g. 'sha256' on the raw device (unmounted!!) and on the dd backup file... they should of course be the same.

Finally, best practice in backups is never to rely on only one backup... keep a minimum of 2!

Solution 3

A disk cloning utility - such as clonezilla will make a compressed copy of your disk, including partition tables etc but omitting unallocated space and also unused space in well-known file systems such as ext4 etc.

Obviously it is much faster to only copy data and not the unused space.

If you suspect that your NVME storage cannot be relied upon for read operations, then perhaps you might want to consider software such as rsync to extract most/all of your data (in addition to a cloning operation).

Comments

[user472052]

Does it work well with NVME drives? My goal here is to create an image of the SSD, nvme secure erase it, and then send the device to the service center. What I want know if I will be able to restore it later with no errors.

[Nordine Lotfi]

Yes, it should work fine for NVME. ddrescue is used on many different type of drives and filesystem, feel free to check the documentation if you'd like. (that i linked in my post). I updated my post with more info on restoring using the image from the previous output. @user472052

[user472052]

One more thing: do I need to add the ./ part to refer to the output file? Can I just write disk.img disk.log and omit the ./ part?

[Nordine Lotfi]

oh, no that was a small error on my part -- you don't need it :) (if you noticed, i didn't use ./ on the second command, so yeah) @user472052

[frostschutz]

there is a risk of data corruption with dd conv noerror sync - add conv=fullblock or use ddrescue

[Darren]

I surprised to see no mention of clonezilla in the comments and answers.

[Peter Cordes]

Plain dd is fast if you use bs=128k or something, to make read/write system calls in 128kiB blocks instead of the default 512-byte sectors. (About half of your CPUs L2 cache size is a good tradeoff between per-system-call overhead vs. the kernel's copy_to_user / copy_from_user cache hits). Piping it through pv obviously costs more CPU time, although if you run pv to have it attack to an already running process (looking at file-positions in /proc/<PID>/fdinfo) then it avoids that memory bandwidth cost. I assume ddrescue chooses a reasonable block size by default.

[Austin Hemmelgarn]

@PeterCordes OTOH, because of stuff like the -S option you can make ddrescue even faster than plain dd, as well as being able to do some other useful things like pre-allocating the output file or bypassing the OS’s page cache.

[Peter Cordes]

If you just want progress stats, GNU dd status=progress shows what it's doing. You don't need pv for that. (And even on any Unix, you could just ls -l the output file whenever you want.)

[Nordine Lotfi]

That's good to know! I've been using ddrescue for quite some time myself and never knew this flag :D Thanks for mentioning this @AustinHemmelgarn

[Peter Cordes]

@AustinHemmelgarn: GNU dd has conv=sparse. IDK when that feature was added. It doesn't have a preallocate option; you have to use fallocate(1) and dd bs=128k conv=notrunc,sparse. Although I think sparseness detection works in units of the bs, so you might want a smaller block size like 16k. For a device that might have read errors, certainly ddrescue is a very good choice, but other than that you don't need it. If you know its options then by all means use it. Good point about tricks like sparsifying the output, especially if you just used fstrim on your FS before backup.

[Kamil Maciorowski]

"As the name suggests"? Out of curiosity: how does the name suggest anything? I've seen the one or the other d explained as "disk", "drive", "data", "dump", "destroyer". How to get from dd to " disk image backup" is a mystery to me. If I didn't know dd, its name would suggest me nothing.

[qwr]

It was supposed to be a joke because dd famously has many nicknames but I guess the way I wrote it the joke doesn't come across at all

[Ljm Dullaart]

unix.stackexchange.com/questions/6804/what-does-dd-stand-for

[Peter - Reinstate Monica]

The use case of the OP as described in the comments (sigh) is to produce a disk image.

[Peter Cordes]

@AustinHemmelgarn: Also note that pre-allocate + sparse is less useful than one would like. The unwritten space stays allocated, and even freeing it later with fallocate -d (dig-holes) means the non-zero parts of the file have gaps, making disk / fs read-ahead less effective at getting data from the next non-hole. And the free space not consumed isn't contiguous. One of the other of sparse or prealloc are certainly useful, though, so it's nice to have both features available all in one tool, like how rsync has --preallocate and --sparse options.

[Rashini Gamalath]

about the speed, dd bs=10M is usually much faster than stock dd (10M is usually NOT the optimal speed for any drive, but it

[Hermann]

If you care about speed, ditch dd and go for the file-system specific tools (e2image, ntfsclone,…). You can also backup or replicate the partition table with sfdisk (MBR) or sgdisk (GPT).