Copying an SD card using dd does not copy exactly

Question

I am trying to make a copy of my SD-Card so I can move it to my 64GB SD-Card. I have done this with the SD-Card of a Raspberry Pi, no problems there.

The SD-Card consists of two partitions: BOOT(fat32) and linux(ext4)

I have tried to make an image of the whole SD-Card using:

sudo dd of=Images/orangepi.img if=/dev/sdd bs=1M status=progress

And putting it back on a SD-Card:

sudo dd if=Images/orangepi.img of=/dev/sdd bs=1M status=progress

I could not mount the image since it consisted of 2 partitions. So I imaged BOOT and linux separately using:

sudo dd of=linux.img if=/dev/sdd2 bs=1M status=progress 
sudo dd of=BOOT.img if=/dev/sdd1 bs=1M status=progress

As you can see in the screenshot I added, the image created(on the right) from the SD-Card does not match the SD-Card(on the left).

My question is: why does this happen and how do I make a proper image of my SD-Card?

My SD-Card's home folder has a folder called Music containing folders with mp3 files.

My image has a x-font.ttf with the name Music. Folders seem to change into random files when imaged.

The SD-Card is a working Ubuntu disk for my orangepi PC and is working at this moment.

$ sudo apt install dcfldd
$ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 465.8G  0 disk 
└─sda2   8:2    0 465.8G  0 part /media/Shared
sdb      8:16   0 238.5G  0 disk 
├─sdb1   8:17   0   500M  0 part 
├─sdb2   8:18   0 116.8G  0 part 
├─sdb3   8:19   0 117.3G  0 part /
├─sdb4   8:20   0     1K  0 part 
└─sdb5   8:21   0   3.9G  0 part [SWAP]
sdc      8:32   1   7.5G  0 disk 
├─sdc1   8:33   1    64M  0 part /media/fhfs/BOOT
└─sdc2   8:34   1   7.4G  0 part /media/fhfs/linux
sdg      8:96   0 465.8G  0 disk 
└─sdg1   8:97   0 465.8G  0 part /media/fhfs/0c91eeb6-7199-47b6-a603-04432a091fdc
sr0     11:0    1  1024M  0 rom  
**ls -lha /dev | grep sd**
brw-rw----   1 root disk        8,   0 Oct 18 14:54 sda
brw-rw----   1 root disk        8,   2 Oct 18 14:54 sda2
brw-rw----   1 root disk        8,  16 Oct 18 14:54 sdb
brw-rw----   1 root disk        8,  17 Oct 18 14:54 sdb1
brw-rw----   1 root disk        8,  18 Oct 18 14:54 sdb2
brw-rw----   1 root disk        8,  19 Oct 18 14:54 sdb3
brw-rw----   1 root disk        8,  20 Oct 18 14:54 sdb4
brw-rw----   1 root disk        8,  21 Oct 18 14:54 sdb5
brw-rw----   1 root disk        8,  32 Oct 20 18:11 sdc
brw-rw----   1 root disk        8,  33 Oct 20 18:11 sdc1
brw-rw----   1 root disk        8,  34 Oct 20 18:11 sdc2
brw-rw----   1 root disk        8,  48 Oct 18 14:54 sdd
brw-rw----   1 root disk        8,  64 Oct 18 14:54 sde
brw-rw----   1 root disk        8,  80 Oct 18 14:54 sdf
brw-rw----   1 root disk        8,  96 Oct 18 14:54 sdg
brw-rw----   1 root disk        8,  97 Oct 18 14:54 sdg1

$ sudo dcfldd if=/dev/sdc2 of=linuxdcfl.img hash=md5,sha1 hashlog=hashlog.txt
242944 blocks (7592Mb) written.
243056+1 records in
243056+1 records out
**sudo dcfldd if=/dev/sdc2 vf=linuxdcfl.img verifylog=verify.log**
0 - 0: Mismatch
Total: Mismatch

I tried dcfldd and got a mismatch, no error log though. verify.log is empty. hashlog just has the sha and md5 sums.

Solution 1

ddhas a long history of creating exact bit for bit duplicates. diff can be used to prove this quite handily

Note: you don't mention what version of Ubuntu you are running. The only reason that makes a difference is that the status switch usage has changed.

Ubuntu 14.04 Excerpt from man dd

 status=WHICH
              WHICH info to suppress outputting to stderr; 'noxfer' suppresses
              transfer stats, 'none' suppresses all

Ubuntu 16.04 excerpt from man dd

status=LEVEL
              The  LEVEL of information to print to stderr; 'none' suppresses everything but error messages, 'noxfer' suppresses
              the final transfer statistics, 'progress' shows periodic transfer statistics

All that aside, the only things I can think of that would result in your image file having a different bit pattern than your source is either:

User Error:

A) An attempt to image a mounted partition (an exceedingly bad idea)

B) Failure to sync leaving data behind in the kernel buffer.

or

Hardware failure:

C) A failing area on the disk where you stored the image. This implies impending failure of the drive (I hope you have backups, if not, hop to it!)

D) A dodgy connection providing poor connectivity to either the source or target media device

You would be wise to check the smart status of the drive you stored the image on.

The fact that dcfldd also resulted in a mismatch leads me to believe that you have either a failing cable or a failing storage media (whether on the input media or the output media)

Solution 2

This is a comment on how fragile SD/microSD chips can be.

I used the above comment about badblocks to test 2 new 512GB microSD chips. Given their size, I went with a USB 3.0 card reader on a USB 3.0 port. The chips got hot, really hot. I then did a benchmark using Disks to confirm their speed, using my standard test of 1000 samples (rather than the standard 100), again on USB 3.0 reader/3.0 port. 1 passed, the other got to ~600 samples and the write speed dropped from ~90 MB/s to under 20 MB/s. That write speed didn't change back, so I think I "burned"^ the chip because of the power available on a USB 3.0 port.

Given that the badblocks is an intensive, full disk, write/read process that takes a long time I would suggest using a USB 2.0 card reader, or preferably a USB 3.0 card reader on a USB 2.0 port, to minimize the possibility of burning out a large (expensive) chip. It will take longer, but so what. I would then recommend using the standard Disks 100 samples to benchmark (USB 3.0 reader/USB 3.0 port).

I (unfortunately) used a number of smaller 128GB microSD chips to determine the above procedure. Most of which went back under RMA for having failed my testing. These chips just aren't as robust as even USB 3.0 thumb-drives, so caveat emptor.

^ By the way, if you are on a USB 3.0 card reader/USB 3.0 port and your chip is taking a lot longer than another chip of the same size, it may already be "burnt".

Comments

[DJCrashdummy]

i'm definitely not an expert at this topic (therefore just comments with suggestions)! -- i would have a look at the man-page and research the web what can cause a mismatch at dcfldd and what to do with it.

[DJCrashdummy]

i can only guess the partition-table and/or the file-system got some errors or maybe the storage suffered a physical damage (but i doubt this). - have you already checked your partition-table/file-system? if you got read-errors have a look at ddrescue (package gddrescue).

[Frank Schrijver]

The sd-card is a working ubuntu system for my orangePi pc and i can still use the sd-card without a problem. I dont think its possible to boot from a partition with a corrupt partition-table/file-system, or is it? thanks for your input by the way

[tripleee]

Your question does not indicate where sdd was mounted at the time you attempted to image it, but as per your comments, it apparently was. I second the answer which says to umount and make sure it stays unmounted for the duration of the imaging.

[Elder Geek]

@pbhj It's entirely possible to boot from a system with a corrupted file-system provided the area(s) of corruption aren't required for booting.

[Frank Schrijver]

i just avoided looking for a way to mount a image with multiple partitions. There is a solution to anything in ubuntu/linux that is not my problem. My problem is figuring out what I am doing wrong.

[Frank Schrijver]

I did try imaging the whole SD-Card to my new SD-Card, with my first two lines of "sudo dd". It ends up exactly like my screenshot. My screenshot is the mounted image(not the target SD-Card, but the same nonetheless).

[boutch55555]

So it could be caused by the switch from SDHC to SDXC, or some wear protection happening in the old card. If you ls -i on both, it should give you different outputs. Is only the home folder affected or the whole partition ?

[boutch55555]

Also, did you make sure nothing was mounted when you created or wrote the image ? sudo umount /dev/sdc1 && sudo umount /dev/sdc2

[Frank Schrijver]

I unmounted my partitions and tried imaging, same results.

[Frank Schrijver]

it is not a SDHC SDXC problem since the image created is already different than the SD-Card