Can you run Docker natively on the new Windows 10 (Ubuntu) bash userspace?

Question

My understanding was that the primary limitation of running docker on other OSs was the Linux Network containers that made it possible. (Certainly for Macs).

Recently Microsoft announced a beta of a Ubuntu linux user mode running natively on Windows 10. This can run binaries compiled in ELF format on Windows (unlike cygwin which requires a compilation.)

My question is: Can you run Docker natively on the new Windows 10 (Ubuntu) bash userspace?

Solution 1

You can use Docker Desktop for Windows as the engine and Docker for Linux as the client in WSL on Ubuntu / Debian on Windows. Connect them via TCP.

Install Docker Desktop for Windows: https://hub.docker.com/editions/community/docker-ce-desktop-windows If you want to use Windows Containers instead of Linux Containers both type containers can be managed by the Linux docker client in the bash userspace.

Since version 17.03.1-ce-win12 (12058) you must check Expose daemon on tcp://localhost:2375 without TLS to allow the Linux Docker client to continue communicating with the Windows Docker daemon by TCP

Follow these steps:

cd
wget https://download.docker.com/linux/static/stable/`uname -m`/docker-19.03.1.tgz
tar -xzvf docker-*.tgz
cd docker
./docker -H tcp://0.0.0.0:2375 ps

or

env DOCKER_HOST=tcp://0.0.0.0:2375 ./docker ps

To make it permanent:

mkdir ~/bin
mv ~/docker/docker ~/bin

Add the corresponding variables to .bashrc

export DOCKER_HOST=tcp://0.0.0.0:2375
export PATH=$PATH:~/bin

Of course, you can install docker-compose

sudo -i
curl -L https://github.com/docker/compose/releases/download/1.24.1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

Or using python pip

sudo apt-get install python-pip bash-completion
sudo pip install docker-compose

And Bash completion. The best part:

sudo -i
apt-get install bash-completion
curl -L https://raw.githubusercontent.com/docker/docker-ce/master/components/cli/contrib/completion/bash/docker > /etc/bash_completion.d/docker
curl -L https://raw.githubusercontent.com/docker/compose/$(docker-compose version --short)/contrib/completion/bash/docker-compose > /etc/bash_completion.d/docker-compose

I've tested it using the 2.1.0.1 (37199) version of Docker Desktop using Hyper-V:

$ docker version
Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89e8a
 Built:             Thu Jul 25 21:17:37 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:17:52 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
Look both client and server say **OS/Arch: linux/amd64**

Volumes

Take care when adding volumes. The path C:\dir will be visible as /mnt/c/dir on WSL and as /c/dir/ by docker engine. You can overcome it permanently:

sudo bash -c "echo -e '[automount] \nroot = /'>/etc/wsl.conf"

You must exit and reload WSL after making the change to wsl.conf so that WSL reads in your changes on launch.

UPDATE

from: What’s new for the Command Line in Windows 10 version 1803

Unix Sockets Unix Sockets weren't supported on Windows, and now they are! You can also communicate over Unix sockets between Windows and WSL. One of the great things about this is it enables WSL to run the Linux Docker Client to interact with the Docker Daemon running on Windows.

UPDATE

This script and the use of Unix Sockets was included in Pengwin's pengwin-setup.

Regards

Solution 2

As of right now (April 2016) the answer is:

We do not know yet (but probably not).

The facts

• Windows 10 can now run a variety of Linux programs (among them the Bash shell and various text utilities). These are not ports (i.e. recompiled versions, like for example in Cygwin), they are the same ELF binaries that run on a typical Linux system. In this case, they were taken from Ubuntu.
• To make this possible, Windows 10 was modified to accept Linux System calls (syscalls), and to be able to load and run ELF binaries (comment by Scott Hanselman). This means unmodified Linux executables can be run, they will load their unmodified shared libraries as required, and Windows will run them as Windows processes.
• Whenever such a Linux program wants to interact with the kernel, it issues a system call (or lets a library do it). This is (presumably) the only difference to running on Linux: When running on Linux, the Linux kernel handles theses calls; on Windows 10, the Windows 10 kernel does it instead.

The speculation

So the question is whether the syscalls that Docker needs (for chroot and namespaces, among other things) were implemented or not. The answer is to that is likely "no". Docker requires fairly sophisticated (and Linux-specific) functionality for process and resource management, and process isolation. While it is probably possible to replicate all this on Windows, it would be a lot of work, and since the goal of this Windows feature seems to be running Linux userspace programs, it seems unlikely they did all the work (and kept it secret).

However, there is no definitive information either way,as far as I can tell.

Existing Docker ports

Of course, if Microsoft decides they want Docker support in Windows 10, they will probably able to offer it. There is some precedent for porting Docker to a different kernel:

• There is a Docker port for FreeBSD. It is labeled "experimental", but in seems to work in principle. It can use unmodified Docker containers from the Docker repository, meaning it actually provides a Linux-like host environment for the images.
• There is a project underway for porting Docker to Windows (specifically, Windows Server 2016) - see this Docker blog entry from August 2015. However, unlike the FreBSD port above, this will be a port that allows Docker to run Windows images on Windows, not Linux images on Windows. Thanks to Rоry McCune for pointing this out.

Solution 3

No, this is not possible.

Docker needs multiple things in order to run containers:

• chroot
• Namespaces for:
  • PID
  • Users
  • Network
  • Mounts
  • UTS
  • IPC

These are all kernel features that are implemented in Linux. Unfortunately, most of them do not have a similar feature in Windows to use as a replacement (nor in the Linux Subsystem that Microsoft implemented in the Windows kernel). All these need to be provided by the OS.

Solution 4

The first insider preview was released yesterday. I've attempted to install docker but it fails with the following:

So it would appear, that for the first preview it does not currently work. However as many people have speculated, it may work in a future release.

Solution 5

As of the Creator Update (released publicly on June 13, 2017) you can run native Windows executable directly in WSL. This means if you've already installed Docker for Windows you can simply invoke the docker binaries installed under C:\Program Files. Since they end in .exe the easiest option is to create aliases. Something like the following in your .bashrc should work:

DOCKER_BIN='/mnt/c/Program Files/Docker/Docker/resources/bin'
for f in "$DOCKER_BIN"/*; do
  alias "$(basename "$f" | sed 's/.exe$//')"'="'"$f"'"'
done

This creates aliases for all the files in the DOCKER_BIN directory:

$ type docker
docker is aliased to `"/mnt/c/Program Files/Docker/Docker/resources/bin/docker.exe"'

$ docker --version
Docker version 17.03.1-ce, build c6d412e

One caveat: you'll get an error message like "Unable to translate current working directory" if run from a Linux directory. Just cd into a Windows directory (e.g. /mnt/c/Users/YourUsername) and you should be good.

View more solutions

Comments

[MSalters]

It's not "just" a bash userspace. It's a real, fairly complete Linux userspace, but without X Windows i.e. text only. Saying "bash" communicates the text-only limitation quite well..

[Michael Hampton]

Did I miss something? Did this actually get distributed? At the moment I only know of it as vaporware.

[Rory McCune]

I think it won't be clear until they release this (AFAIK it's not even available to windows insiders as yet) however it's worth noting that Microsoft and Docker are working on bringing Docker to Windows natively, to be released alongside Windows server 2016

[sleske]

@RоryMcCune: Interesting. However, according to this Docker blog entry from August 2015, this will be a port that allows Docker to run Windows images on Windows, not Linux images on Windows.

[Rory McCune]

indeed that's the purpose of the Windows native docker. The nature of containerisation is that you can't run systems with other kernels, without adding in some virtualisation or (possibly) this new subsystem that Microsoft are developing

[Spooler]

For a GNU system to run on a Windows kernel is certainly possible. It would be a LOT of work for a very lateral shift in quality. I would be surprised if it happened, or if people adopted it.

[Ashish Karpe]

Is it possible to run windows docker image on ubuntu aws instance ? or is there some alternative ? ......root@ip-172-31-22-20:~/docker/windows# docker pull microsoft/nanoserver Using default tag: latest latest: Pulling from microsoft/nanoserver bce2fbc256ea: Pulling fs layer 4806a44e00a0: Pulling fs layer image operating system "windows" cannot be used on this platform root@ip-172-31-22-20:~/docker/windows#

[kenorb]

Related: docker error: /var/run/docker.sock: no such file or directory.

[MSalters]

Actually, Windows does have namespaces for Users, Mounts and IPC. User namespaces are required for Active Directory, mount namespaces and IPC namespaces are required for multi-user operation. Fundamentally, the kernel Object Manager in Windows has always had namespaces, from the very first Windows NT release, so it's not that strange.

[MSalters]

And with Remote Desktop Services, the Session objects actively use these namespaces to provide concurrent operation. That doesn't mean you have all the required infrastructure, but major parts are there. As for chroot, realize that the Ubuntu environment already has a different root than the WIN32 one.

[sleske]

Also note that there is a Docker port for FreeBSD, so there is a precedent for porting Docker to a different kernel.

[Ryan]

@sleske is right, this question isn't answerable at the moment, and saying "no it can't" without any real indication of what the devs working on the linux space in widows are up to is pretty presumptuous.

[ny195]

I don't know enough to state with confidence that this answer is totally wrong, but the way it is phrased makes me a bit skeptical of its validity. In particular stating "Bash is a simple user space program and cannot provide any of these" and referring to the Window Subsystem for Linux as "the new Bash feature" makes it sound like this answer is based on the totally false assumption that all Microsoft did was port bash to Windows. That's not what happened. They developed a whole Linux kernel interface running on top of the Windows kernel: msdn.microsoft.com/en-us/commandline/wsl/about

[sleske]

I added my own answer, so this information is not buried in a comment.

[sleske]

Good idea to actually try this. One thing: Could you please add the text of the screenshot as actual text (Ubuntu terminal supports copy&paste). "Real" text has numerous advantages (easier to read, supports screen readers, crawlable by search engines)

[Hawkeye]

Awesome! Please keep trying.

[Timothy Meade]

It looks like this might be related to how memory allocation works in golang (which Docker is written in): groups.google.com/forum/#!msg/golang-dev/EpUlHQXWykg/…

[wieczorek1990]

Mine just hangs forever after the command.

[sleske]

Update: There's an article on the topic on an MSDN blog now: Windows Subsystem for Linux Overview.

[Carlos Rafael Ramirez]

This is complemented with the fact that now you can run docker client for linux in bash, so avoiding using PowerShell if you are not use to

[Hawkeye]

It will be interesting to see if they scale this up to docker swarm.

[Sami Kuhmonen]

"Bash is a simple user space program" but it runs in a whole Linux subsystem and has nothing to do with the features provided by it. But still at the moment chroot is not supported.

[Joel Pearson]

I think this answer has missed the point of the question and this line gives it away I've tested it using the 1.12 version of Docker for Windows using Hyper-V. @crramirez is running the virtualised version of docker for Windows as a full VM. This question is about the Windows System for Linux (WSL) which is a Linux emulation layer. Unless it is implied he installed Ubuntu Bash on Windows, but he never mentions it specifically.

[Zach Russell]

I just tested it and it works for me on 1.12 :) Super excited to now be able to do some serious development on windows.

[mpen]

Tried your instructions in Bash on Ubuntu on Windows, got "Cannot connect to the Docker daemon. Is the docker daemon running on this host?". Are you missing some steps?

[Carlos Rafael Ramirez]

@mpen Before executing my steps, first be sure you have Docker for Windows installed and running

[Kiran]

I am seeing the same problem as @mpen. I have docker installed and running when I try from powershell or command prompt. I tried using admin command/powershell terminals as well.

[Carlos Rafael Ramirez]

Check if the docker notification icon is moving or is static. If it is moving docker is not ready yet for connection. If it is static well it is another problem.

[Kiran]

Thanks Carlos, I followed the instructions here: docs.microsoft.com/en-us/virtualization/windowscontainers/… There is no icon installed

[Carlos Rafael Ramirez]

I recommend you to install Docker for Windows Beta. It includes Windows Containers as well and it takes care of everything is needed to have Docker running. The stable version doesn't include yet Windows containers

[Carlos Rafael Ramirez]

download.docker.com/win/beta/InstallDocker.msi

[hdave]

As of February 2017, this is still the only way.

[dashesy]

I installed the latest CE edition, as well as running the docker-for-windows, and only had to export DOCKER_HOST=tcp://0.0.0.0:2375. It seems it is working but on build I get "sending build context to docker daemon"!

[Carlos Rafael Ramirez]

I have had problems with docker build and docker cp in this mode. So I recommed using the windows client for these two cases.

[Jón Trausti Arason]

The main problem with using Docker for Windows is that inotify isn't working. This means if you are using file watchers such as fresh then they won't work.

[Rüdiger Schulz]

This does not seem to pick up bash environment variables into docker-compose YML files. Any idea for that?

[dimo414]

That makes sense, since you're invoking the Windows docker binary, and just doing so through the Linux shell. I'm not sure if there's a good way to accomplish that.

[Rüdiger Schulz]

When I have mounted files with docker-compose in this setup, I get the error "Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type". Did anyone have that as well and found a solution?

[Jaime]

After June 2017, you can run the docker for windows executables from the bash shell. Instead of installing the docker client you can use the docker.exe directly to query/interact the docker subsystem. However, note that the docker for windows will use the windows variables and configurations.

[Nick Sweeting]

Looks like it works now: tutorials.ubuntu.com/tutorial/…

[James]

I downvoted simply because the higher voted answer is more useful then speculation of what may be, it covers what is.