CUDA install issue: Invalid cross-device link

Question

I am trying to install CUDA on Ubuntu 18.04, on a headless server (no graphical), following official guide, but I face errors:

# remove previous
apt-get update
apt-get remove --auto-remove nvidia-cuda-toolkit 
apt-get purge nvidia-cuda-toolkit 
apt-get purge --auto-remove nvidia-cuda-toolkit 
apt-get autoremove
# install new
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget http://developer.download.nvidia.com/compute/cuda/11.0.1/local_installers/cuda-repo-ubuntu1804-11-0-local_11.0.1-450.36.06-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804-11-0-local_11.0.1-450.36.06-1_amd64.deb
sudo apt-key add /var/cuda-repo-ubuntu1804-11-0-local/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda

Issues at the end of the process sudo apt-get -y install cuda:

Errors were encountered while processing:
 /tmp/apt-dpkg-install-7DzKEt/135-nvidia-compute-utils-450_450.36.06-0ubuntu1_amd64.deb
 /tmp/apt-dpkg-install-7DzKEt/140-nvidia-utils-450_450.36.06-0ubuntu1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Doing sudo apt-get -y install cuda again:

 Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    cuda is already the newest version (11.0.1-1).
    You might want to run 'apt --fix-broken install' to correct these.
    The following packages have unmet dependencies:
     cuda-drivers-450 : Depends: nvidia-compute-utils-450 (>= 450.36.06) but it is not going to be installed
                        Depends: nvidia-utils-450 (>= 450.36.06) but it is not going to be installed
     nvidia-driver-450 : Depends: nvidia-compute-utils-450 (= 450.36.06-0ubuntu1) but it is not going to be installed
                         Depends: nvidia-utils-450 (= 450.36.06-0ubuntu1) but it is not going to be installed
                         Recommends: libnvidia-compute-450:i386 (= 450.36.06-0ubuntu1) but it is not installable
                         Recommends: libnvidia-decode-450:i386 (= 450.36.06-0ubuntu1) but it is not installable
                         Recommends: libnvidia-encode-450:i386 (= 450.36.06-0ubuntu1) but it is not installable
                         Recommends: libnvidia-ifr1-450:i386 (= 450.36.06-0ubuntu1) but it is not installable
                         Recommends: libnvidia-fbc1-450:i386 (= 450.36.06-0ubuntu1) but it is not installable
                         Recommends: libnvidia-gl-450:i386 (= 450.36.06-0ubuntu1) but it is not installable
    E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

Doing apt --fix-broken install:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following additional packages will be installed:
  nvidia-compute-utils-450 nvidia-utils-450
The following NEW packages will be installed:
  nvidia-compute-utils-450 nvidia-utils-450
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
313 not fully installed or removed.
Need to get 0 B/489 kB of archives.
After this operation, 1570 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 file:/var/cuda-repo-ubuntu1804-11-0-local  nvidia-compute-utils-450 450.36.06-0ubuntu1 [123 kB]
Get:2 file:/var/cuda-repo-ubuntu1804-11-0-local  nvidia-utils-450 450.36.06-0ubuntu1 [366 kB]
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 77110 files and directories currently installed.)
Preparing to unpack .../nvidia-compute-utils-450_450.36.06-0ubuntu1_amd64.deb ...
Unpacking nvidia-compute-utils-450 (450.36.06-0ubuntu1) ...
dpkg: error processing archive /var/cuda-repo-ubuntu1804-11-0-local/./nvidia-compute-utils-450_450.36.06-0ubuntu1_amd64.deb (--unpack):
 unable to make backup link of './usr/bin/nvidia-cuda-mps-control' before installing new version: Invalid cross-device link
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Preparing to unpack .../nvidia-utils-450_450.36.06-0ubuntu1_amd64.deb ...
Unpacking nvidia-utils-450 (450.36.06-0ubuntu1) ...
dpkg: error processing archive /var/cuda-repo-ubuntu1804-11-0-local/./nvidia-utils-450_450.36.06-0ubuntu1_amd64.deb (--unpack):
 unable to make backup link of './usr/bin/nvidia-debugdump' before installing new version: Invalid cross-device link
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/cuda-repo-ubuntu1804-11-0-local/./nvidia-compute-utils-450_450.36.06-0ubuntu1_amd64.deb
 /var/cuda-repo-ubuntu1804-11-0-local/./nvidia-utils-450_450.36.06-0ubuntu1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Here we are.. how to fix unable to make backup link of './usr/bin/nvidia-cuda-mps-control' before installing new version: Invalid cross-device link ?

Thanks a lot !

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic

Solution

At the time of this writing you need to resolve the conflict manually:

sudo dpkg -i --force-overwrite /var/cache/apt/archives/libnvidia-compute-450_450.36.06-0ubuntu1_amd64.deb

followed by:

sudo apt --fix-broken install

Source: https://forums.developer.nvidia.com/t/problem-with-apt-and-nvidia-440-on-ubuntu-20-04/115281/4

Comments

[nobody]

Oh, 313 not fully installed or removed. Make a backup first. dpkg -l | egrep -v '^ii|rc'

[reasons]

What dpkg -l | egrep -v '^ii|rc' it supposed to give?

[nobody]

shows packages in unclean state.

[ubfan1]

Do you need CUDA 11, which may actually require the 450 driver? This is a common problem (Nvidia cuda packages depend upon the too (old,new) Nvidia driver they supply, See the suggested solutions on this site for using the default (440) drivers on CUDA 10.2..