Juju stuck in "pending" state when using LXC

Question

So I'm trying to get started with Juju, and tried to do this locally using LXC.

I followed the instructions here: How do I configure juju for local usage?

Unfortunately this doesn't seem to work for me.

status shows the following:

$ juju status
machines:
  0:
    agent-state: running
    dns-name: localhost
    instance-id: local
    instance-state: running
services:
  mysql:
    charm: cs:precise/mysql-1
    relations:
      db:
      - wordpress
    units:
      mysql/0:
        agent-state: pending
        machine: 0
        public-address: null
  wordpress:
    charm: cs:precise/wordpress-0
    exposed: true
    relations:
      db:
      - mysql
    units:
      wordpress/0:
        agent-state: pending
        machine: 0
        open-ports: []
        public-address: null
2012-05-10 14:09:38,155 INFO 'status' command finished successfully

As you can see the agent-state is 'pending' and there is no public address where I'm able to access the newly created site. Am I missing something here?

UPDATE: Tried destroying the environment an doing everything again (multiple times). This is the output for debug-log:

~$ juju debug-log 
2012-05-11 08:50:23,790 INFO Enabling distributed debug log.
2012-05-11 08:50:23,806 INFO Tailing logs - Ctrl-C to stop.
2012-05-11 08:50:42,338 Machine:0: juju.agents.machine DEBUG: Units changed old:set([]) new:set(['mysql/0'])
2012-05-11 08:50:42,339 Machine:0: juju.agents.machine DEBUG: Starting service unit: mysql/0 ...
2012-05-11 08:50:42,459 Machine:0: unit.deploy DEBUG: Downloading charm cs:precise/mysql-1 to /home/andre/.juju/data/andre-local/charms
2012-05-11 08:50:42,620 Machine:0: unit.deploy DEBUG: Using <juju.machine.unit.UnitContainerDeployment object at 0x9c54b6c> for mysql/0 in /home/andre/.juju/data/andre-local
2012-05-11 08:50:42,648 Machine:0: unit.deploy DEBUG: Starting service unit mysql/0...
2012-05-11 08:50:42,649 Machine:0: unit.deploy DEBUG: Creating master container...
2012-05-11 08:54:33,992 Machine:0: unit.deploy DEBUG: Created master container andre-local-0-template
2012-05-11 08:54:33,993 Machine:0: unit.deploy INFO: Creating container mysql-0...
2012-05-11 08:56:18,760 Machine:0: unit.deploy INFO: Container created for mysql/0
2012-05-11 08:56:19,466 Machine:0: unit.deploy DEBUG: Charm extracted into container
2012-05-11 08:56:19,569 Machine:0: unit.deploy DEBUG: Starting container...
2012-05-11 08:56:22,707 Machine:0: unit.deploy INFO: Started container for mysql/0
2012-05-11 08:56:22,707 Machine:0: unit.deploy INFO: Started service unit mysql/0
2012-05-11 08:56:23,012 Machine:0: juju.agents.machine DEBUG: Units changed old:set(['mysql/0']) new:set(['wordpress/0', 'mysql/0'])
2012-05-11 08:56:23,039 Machine:0: juju.agents.machine DEBUG: Starting service unit: wordpress/0 ...
2012-05-11 08:56:23,154 Machine:0: unit.deploy DEBUG: Downloading charm cs:precise/wordpress-0 to /home/andre/.juju/data/andre-local/charms
2012-05-11 08:56:23,396 Machine:0: unit.deploy DEBUG: Using <juju.machine.unit.UnitContainerDeployment object at 0x9c519cc> for wordpress/0 in /home/andre/.juju/data/andre-local
2012-05-11 08:56:23,620 Machine:0: unit.deploy DEBUG: Starting service unit wordpress/0...
2012-05-11 08:56:23,621 Machine:0: unit.deploy INFO: Creating container wordpress-0...
2012-05-11 08:58:24,739 Machine:0: unit.deploy INFO: Container created for wordpress/0
2012-05-11 08:58:25,163 Machine:0: unit.deploy DEBUG: Charm extracted into container
2012-05-11 08:58:25,397 Machine:0: unit.deploy DEBUG: Starting container...
2012-05-11 08:58:27,982 Machine:0: unit.deploy INFO: Started container for wordpress/0
2012-05-11 08:58:27,983 Machine:0: unit.deploy INFO: Started service unit wordpress/0

This is the result for the status command (with verbose flag):

~$ juju -v status
2012-05-11 08:51:53,464 DEBUG Initializing juju status runtime
2012-05-11 08:51:53,625:4030(0xb7345b00):ZOO_INFO@log_env@658: Client environment:zookeeper.version=zookeeper C client 3.3.5
2012-05-11 08:51:53,625:4030(0xb7345b00):ZOO_INFO@log_env@662: Client environment:host.name=andre-ufo
2012-05-11 08:51:53,625:4030(0xb7345b00):ZOO_INFO@log_env@669: Client environment:os.name=Linux
2012-05-11 08:51:53,625:4030(0xb7345b00):ZOO_INFO@log_env@670: Client environment:os.arch=3.2.0-24-generic-pae
2012-05-11 08:51:53,625:4030(0xb7345b00):ZOO_INFO@log_env@671: Client environment:os.version=#37-Ubuntu SMP Wed Apr 25 10:47:59 UTC 2012
2012-05-11 08:51:53,626:4030(0xb7345b00):ZOO_INFO@log_env@679: Client environment:user.name=andre
2012-05-11 08:51:53,626:4030(0xb7345b00):ZOO_INFO@log_env@687: Client environment:user.home=/home/andre
2012-05-11 08:51:53,626:4030(0xb7345b00):ZOO_INFO@log_env@699: Client environment:user.dir=/home/andre
2012-05-11 08:51:53,626:4030(0xb7345b00):ZOO_INFO@zookeeper_init@727: Initiating client connection, host=192.168.122.1:41779 sessionTimeout=10000 watcher=0xb7780620 sessionId=0 sessionPasswd=<null> context=0x9242ee8 flags=0
2012-05-11 08:51:53,627:4030(0xb6b90b40):ZOO_INFO@check_events@1585: initiated connection to server [192.168.122.1:41779]
2012-05-11 08:51:53,649:4030(0xb6b90b40):ZOO_INFO@check_events@1632: session establishment complete on server [192.168.122.1:41779], sessionId=0x1373ae057d90007, negotiated timeout=10000
2012-05-11 08:51:53,651 DEBUG Environment is initialized.
machines:
  0:
    agent-state: running
    dns-name: localhost
    instance-id: local
    instance-state: running
services:
  mysql:
    charm: cs:precise/mysql-1
    relations:
      db:
      - wordpress
    units:
      mysql/0:
        agent-state: pending
        machine: 0
        public-address: null
  wordpress:
    charm: cs:precise/wordpress-0
    relations:
      db:
      - mysql
    units:
      wordpress/0:
        agent-state: pending
        machine: 0
        public-address: null

Solution 1

I was running into the same error, and with the help of the good folks in #juju, was able to determine that having my firewall turned on on the host machine was blocking zookeeper from connecting back to the host.

Try running:

sudo ufw disable

and then:

sudo juju destroy-environment

and then fire things back up. Also, if this is the first time you're bootstrapping an environment on your machine, note that it takes some time for the initial charm downloading to complete, so give it 15-20 minutes after you deploy a unit.

This is also now an open bug, as juju should be handling this situation automatically.

Solution 2

If this is the first time you've bootstrapped your local environment it will take several (depending on how long it takes to download about 400MB of Server Image data) to build the first master image. In your "data-dir" path (defined in your environments.yaml file) there is a machine-agent.log which describes this process:

2012-05-09 10:04:03,848: juju.agents.machine@INFO: Machine agent started id:0
2012-05-09 10:05:08,175: juju.agents.machine@DEBUG: Units changed old:set([]) new:set(['mysql/0'])
2012-05-09 10:05:08,176: juju.agents.machine@DEBUG: Starting service unit: mysql/0 ...
2012-05-09 10:05:08,222: unit.deploy@DEBUG: Downloading charm cs:precise/mysql-1 to /home/marco/.juju/local/marco-local/charms
2012-05-09 10:05:08,314: unit.deploy@DEBUG: Using <juju.machine.unit.UnitContainerDeployment object at 0x9cccbec> for mysql/0 in /home/marco/.juju/local/marco-local
2012-05-09 10:05:08,375: unit.deploy@DEBUG: Starting service unit mysql/0...
2012-05-09 10:05:08,376: unit.deploy@DEBUG: Creating master container...

A few moments later you'll see the following:

2012-05-09 10:09:40,699: unit.deploy@DEBUG: Created master container marco-local-0-template
2012-05-09 10:09:40,699: unit.deploy@INFO: Creating container mysql-0...
2012-05-09 10:10:31,429: unit.deploy@INFO: Container created for mysql/0
2012-05-09 10:10:31,483: unit.deploy@DEBUG: Charm extracted into container

Which details that several minutes later the master container was created.

Finally, not every "local" boostrap works, try running juju destroy-environment and then re-run juju bootstrap

Solution 3

I had this same issue. I found in master-customize.log apt-get failures due to corrupt packages in apt-cacher-ng (I'm not sure bit I think this happened because my laptop suspended during the download). I was able to correct the problem by visiting http://localhost:3142/acng-report.html, checking:

• Validate by file name AND file directory (not recommended),
• then validate file contents through checksum (SLOW), also detecting corrupt files,
• then truncate damaged files immediately.

and clicking on Start Scan and/or Expiration. Then I was able to destroy the juju environment and successfully redeploy it.

Comments

[Admin]

Can you edit your question and link the examples you are following?

[Admin]

Also can you add the output of 'ps auxf' ? That should show that the wordpress node is still pending because it is still installing some of its components. On a slow connection with a slow hard drive and low RAM, it can take 10+ minutes to get these nodes installed.

[Admin]

There is a helpful tool in the juju bzr branch that will feed us back some information. Can you grab it using bzr branch lp:juju and then sudo misc/devel-tools/juju-inspect-local-provider and then also run sudo lxc-ls and run the above tool for each of the images that are listed, so we can see the output of all the logs inside the containers.

[David]

Thanks for the reply. I'm restarting the process and keeping an eye on the logs to see what happens.

[David]

After juju destroy-environment, do I have to re-deploy the charms? Or are they essentially already 'installed'?

[Michael Gundlach]

@Andre You'll have to re-deploy. Doing a destroy-environment will essentially delete the environment and everything that was running on it.

[David]

Still no luck unfortunately. I've tried this multiple times and waited to make sure that everything finished. I updated my original post with the verbose status and debug-log.

[David]

Disabled the firewall, and it worked immediately. I hope they get this bug sorted out, makes me feel nervous to disable my ufw, but at least I can now experiment and play with juju until such time that I have a server to play on :)