“Hash sum mismatch” error due to identical SHA1 and MD5 but different SHA256

Question

When running apt full-upgrade or apt update I'm getting several logs that are more or less the same as this one, where there is an identical filesize, SHA1 and MD5 but different SHA256:

E: Failed to fetch store:/var/lib/apt/lists/partial/jp.archive.ubuntu.com_ubuntu_dists_focal_main_binary-amd64_Packages.xz  Hash Sum mismatch
   Hashes of expected file:
    - Filesize:5826751 [weak]
    - SHA256:af226b4496cbb524bd4814d102047ae77769836203274dffc91cb543d5da13cc
    - SHA1:aef5c36ce45bd5c3154a1bb03c62b6cfb33e2bc6 [weak]
    - MD5Sum:7ef83228ec207df10acac48fbdd81112 [weak]
   Hashes of received file:
    - SHA256:e2c7fc5a2d86f75f03612fec614dcf84d3d502976558fbe40928c1dd120bb05e
    - SHA1:aef5c36ce45bd5c3154a1bb03c62b6cfb33e2bc6 [weak]
    - MD5Sum:7ef83228ec207df10acac48fbdd81112 [weak]
    - Filesize:5826751 [weak]
   Last modification reported: Thu, 23 Apr 2020 16:40:26 +0000
   Release file created at: Thu, 23 Apr 2020 17:33:17 +0000

Whereas others have the same MD5, but different SHA1 and SHA256:

E: Failed to fetch http://jp.archive.ubuntu.com/ubuntu/pool/main/l/linux/linux-headers-5.4.0-29_5.4.0-29.33_all.deb  Hash Sum mismatch
   Hashes of expected file:
    - SHA256:edde13dcd52c51e2404d37731948d491e6e215b22f599239326809754f8d5633
    - SHA1:3e46354d6aaa92f82d5392a80f6b72c3fcc78a91 [weak]
    - MD5Sum:4ccf38c666c836f29fc3602314de47b9 [weak]
    - Filesize:10936232 [weak]
   Hashes of received file:
    - SHA256:9af606f85850ba3010ac8743f62dca91bbef3e5e60da5a7df62dac53eb5a4fb7
    - SHA1:61c842d896676ac9af0fef9f9f48bc4a3c39cce6 [weak]
    - MD5Sum:4ccf38c666c836f29fc3602314de47b9 [weak]
    - Filesize:10936232 [weak]
   Last modification reported: Thu, 30 Apr 2020 08:23:38 +0000

This problem is similar to what is asked here, and indeed I tried several of the solutions offered there (and elsewhere) but they did not help. However I am wondering specifically about why the SHA-1 and MD5Sum are identical, and only SHA256 is different. Does this mean that there is a security problem somewhere along the line? Or is my system reading SHA256 incorrectly? Something else?

Thanks

Solution 1

Disable Windows Subsystem For Linux and Virtual Machine Platform if you're using Windows and VirtualBox. Happened to me when trying to install lubuntu 20.04 using VirtualBox 6.0.4 and WSL2 was enabled.

Solution 2

Try this before running apt:

$ sudo bash
# mkdir /etc/gcrypt
# echo all >> /etc/gcrypt/hwf.deny

Because apt uses the sha256 method from libgcrypt20, but optimized too much. We can opt out of these optimizations using the configuration file /etc/gcrypt/hwf.deny.

---

info gcrypt section 16 says:

/etc/gcrypt/hwf.deny

This file can be used to disable the use of hardware based optimizations, see hardware features.

info gcrypt section 2.7 then says:

Libgcrypt makes use of certain hardware features. If the use of a feature is not desired it may be either be disabled by a program or globally using a configuration file.

Solution 3

EXPLANATION and Solution: Quick Fix

This issue is caused by the Windows Hypervisor Platform. This issue cannot be resolved for now (asfar as I know).

A partial fix is at hand though. And I say"partial" because it involves disabling the platform (also known as"Hyper-V") which will probably break other virtualization solutions you have installed since this is enabled manually. Anyway, here's how to disable it and get your VM running again,

1. Shut down the Virtual Machine.

2. Press Windows logo key + X, then hit A to run Command Prompt(powershell) as administrator.

3. Type bcdedit /set hypervisorlaunchtype off

4.When you see"The operation completed succesfully", reboot your windows. After reboot, boot your VM and update/upgrade.

Solution 4

The file from the jp... archive is correct also, so it must be your copy of sha256sum that is borked. Confirm the sha256sum executable you are running is in /usr/bin, and if it is not, remove the identified executable which is earlier in your path than /usr/bin. If it is, download the coreutils package from the primary Ubuntu archives, and reinstall coreutils.

View more solutions

Comments

[ubfan1]

You should ask the package maintainer, they probably made an error.

[RyanQuey]

@ubfan1 that's the thing, unless I'm mistaken these are core Ubuntu packages (I'm running on 20.04 lts). Moreover, in another VM I'm running, I'm not getting these same errors

[ubfan1]

I just downloaded the pkg from ports.ubuntu.com/pool/main/l/linux/… and the sha256sum hash is correct.Try again.

[RyanQuey]

That's a good idea, confirming the SHA256 sum that way. It's still breaking for me though.

[RyanQuey]

Could you expand on why this would solve the hash sum mismatch, and if there's any external documentation you can add? Thanks

[Marcin Gordziejewski]

VirtualBox is using so called "Virtual Machine Platform" as a virtualization provider if it is enabled. I've no idea why this happens, but for me when running apt update I got mismatch of ONLY SHA256 - SHA1 and MD5 were the same. This gives me an idea there is a bug in this version of Virtual Machine Platform and for some reason SHA256 calculation is done incorrectly. This is just my guess because it solved the issue for me. forums.virtualbox.org/viewtopic.php?f=6&t=92440 It was "Turtle Icon" I got when it didn't work - hence my guess that this changes how virtual CPU behaves.

[RyanQuey]

Thank you so much...this was what I really needed. I was able to get rid of my green turtle as well. Posting this link here for a more full and more recent explanation and set of instructions: forums.virtualbox.org/…

[RyanQuey]

For me, this answer was helpful in determining that my sha256sum was not working correctly. However, this answer here is actually what was able to fix it for me. I didn't mention anything about VirtualBox and using a Windows 10 host in my question though. I think it wouldn't be fair to this answer to change the question now though, so just upvoting both and accepting the one that fixed it for me.

[Akito]

That is not a solution! The whole point why I use this Windows version is WSL2. It has to both work on the same system.

[Marcin Gordziejewski]

Currently cannot tell why this is happening - so a quick remedy is doing what I said. There probably might be a solution out there in the wild to solve it - but I just don't know of it.

[RyanQuey]

@Akito it may not be a good solution, but it is the officially recommended solution put forth by VirtualBox. It seems like the only other option is not using VirtualBox, as far as I can tell

[cclloyd]

This is what worked for me. Disabling WSL like the accepted answer says didn't help.

[Striar]

Use this if you are running into issues with virtualbox + ubuntu + (hyperv or wsl or windows container platform)

[prm296]

Just tested this solution in Ubuntu 20.04 hosted in a Windows 10 install of Oracle VM VirtualBox. Nothing worked before this, even tried to disable wsl.

[user2622016]

Worked for me! Windows 10 host (the thing was broken by newest Windows update), Ubuntu 20.04 guest. Haven't disabled WSL or anything - this solution worked.

[geisterfurz007]

Asking out of curiosity: What exactly does this do? It solves the issue indeed but reads like I would disable all (?) checks for something which doesn't sound that great from a security perspective.

[caduceus]

@geisterfurz007 it just disables the use of dedicated hardware features provided by the chip. Its in the docs for dev.gnupg.org/source/libgcrypt

[David Beneš]

Thanks @MRX Clay. You saved my day! In fact, you saved my night ;) The issue happened to me after enabling Windows Sandbox feature and did not go away after disabling it. I had two virtual machines broken on two different physical hosts. And by broken I mean that all checksum operations returned wrong results. I was unable to use npm, serverless or apt. Now it all works again.

[Pushpendra]

I'm sorry is there anyway I can make it work in Ubuntu server installer ? I tried moving to bash terminal from installer help and did these steps and it looks working. But again when I'm going to installer and face same problem. Tried couple of other things like changing chroot to /target then try same steps but looks not working. And I can see setup fails on has mismatch. Also while booting of installer I can see some integrity fail errors. Any help would be appreciated.

[tomd]

This solution also worked for me. On new PC with latest verions of windows. Not a problem with an old machine (2years, Windows 10) with Virtual Machine Platform turned off. Curiously, turing VMP off on new machine did not solve the problem

[Sir_Scofferoff]

This answer allowed Apt to update but several hash errors were still encountered during the Upgrade process. I had also used the hwf.deny suggestion before without success. I had disabled WSL and enabled VMP without a reboot.

[Terry Wang]

Interesting, hit the problem when running Ubuntu 20.04 LTS VM using VirtualBox on macOS (2019 MBP), the CPU may have some additional hardware acceleration features that the creator (very old 2007 MacBook3,1 running Manjaro) does NOT have, so when VirtualBox exposed the features to the VM, it leverages it by default and therefore caused the problem which I have not seen elsewhere.

[matthewcherrey]

This is the solution. Trying to install Lubuntu impish (21.10) in virtual box on windows 10. I also had WSL2 installed but didn't want to remove it.