lxc-create hangs and finally fails

Question

I was trying to place some juju charms on linux containers and the containers failed to start. To debug the issue I sshed into the node where this happened and tried creating an LXC container manually like

ubuntu@slot13:~$ sudo lxc-create -t ubuntu -n pavan
Checking cache download in /var/cache/lxc/trusty/rootfs-amd64 ...
Installing packages in template: ssh,vim,language-pack-en
Downloading ubuntu trusty minimal ...
I: Retrieving Release

It isn't making any progress at all. Its stuck here for a long long time. After a really long time it says,

ERROR: Unable to fetch GPG key from keyserver

and continues to hang. Finally after 20-30 mins, it gives up like

E: Failed getting release file http://archive.ubuntu.com/ubuntu/dists/trusty/Release

Where are the log files corresponding to lxc-create command? How can I debug this issue?

EDIT: I figured out how to see the debug logs and hence ran the below command a few times

sudo lxc-create -t ubuntu -n pavan --logfile=test.txt --logpriority=DEBUG

test.txt contains only this

lxc-create 1414897265.204 ERROR    lxc_container - Error: pavan creation was not completed
lxc-create 1414897407.757 ERROR    lxc_container - Error: pavan creation was not completed
lxc-create 1414897407.759 WARN     lxc_log - lxc_log_init called with log already initialized

But still it hangs and the debug logs aren't offering much help.

Solution 1

Are you behind a proxy? if yes, then you may set your proxy in bash. Issue following,

export http_proxy="http://<proxy_host>:<port>"

Solution 2

Please append --no-validate flag to skip gpg validation:

lxc-create -t ubuntu -n pavan -- --no-validate

Refer https://github.com/lxc/lxc/blob/master/templates/lxc-download.in#L240

Solution 3

Thanks Felipe for the workaround - for a complete fix, required also doing:

mkdir /var/lib/lxc/juju-trusty-lxc-template/rootfs/var/log/juju

Details:

1. ran lxc-create as per above Felipe's updateList item

2. deploying any service was consistently failing, juju status showing:

   agent-state-info: 'container failed to start and was destroyed: jjo-local-machine-1'
   

3. found at /var/lib/juju/containers/jjo-local-machine-5/container.log :

   lxc-start 1427066682.951 ERROR    lxc_conf - conf.c:mount_entry:1711 - No such file or directory - failed to mount '/var/log/juju-jjo-local' on '/usr/lib/x86_64-linux-gnu/lxc/var/log/juju'
   

4. Creating the directory fixed it, further deploys ok:

    mkdir /var/lib/lxc/juju-trusty-lxc-template/rootfs/var/log/juju
   

Solution 4

Problem

If you are behind a firewall that restricts internet access, you could get this error:

$ lxc-create -t download -n my-container
Setting up the GPG keyring
ERROR: Unable to fetch GPG key from keyserver.
lxc-create: lxccontainer.c: create_run_template: 1297 container
  creation template for my-container failed
lxc-create: tools/lxc_create.c: main: 318 Error creating
  container my-container

Fix

Connect to the key server through port 80:

$ lxc-create -t download -n my-container \
  -- --keyserver hkp://p80.pool.sks-keyservers.net:80

Details

The key server, hkp://p80.pool.sks-keyservers.net:80, is from the lxc source code:

apt source lxc
grep -ri hkp: .

References

• Launchpad bug 1331920
• Launchpad bug 1609479

Solution 5

Actually juju uses the 'ubuntu-cloud' template to grab an image, the command executed to create the template in my system running trusty is:

lxc-create -n juju-trusty-lxc-template \
  -t ubuntu-cloud \
  -f /var/lib/juju/containers/juju-trusty-lxc-template/lxc.conf \
  -- --debug \
  --userdata /var/lib/juju/containers/juju-trusty-lxc-template/cloud-init \
  --hostid juju-trusty-lxc-template
  -r trusty

I suggest you to run something like that and see what error gives you.

View more solutions

Comments

[researcher]

Hello,Please can you explain more the problem and the soltuion, because i have the same error ?

[researcher]

root@localhost:/home# iptables -t nat -IPOSTROUTING -s 10.0.3.1/16 -j MASQUERADE root@localhost:/home# sudo lxc-create -t ubuntu -n u1 -- -r trusty -a amd64 Checking cache download in /var/cache/lxc/trusty/rootfs-amd64 ... Installing packages in template: ssh,vim,language-pack-en Downloading ubuntu trusty minimal ... I: Retrieving Release ................................ it is blocked

[researcher]

@Pradeeper : Please, i'd like to test your solution. export http_proxy="http://<proxy_host>:<port>" but what values can i choose for "port" and "proxy_host" is the IP @ ?

[Pradeeper]

You have to issue this command in a terminal. If you don't use specif port on your proxy and use port 80 then ignore the port section above. If above fixed your issue, then you can add that line to your .bashrc file or /etc/profile

[sam]

Note it's important to use -- --no-validate not --no-validate: the last one isn't valid. I edited the answer with the correct example.