Run `apt-get update` before installing other packages with Puppet

Question

I'm trying to create puppet module which automates installation of zend server CE, this is not important here, but steps are as following

1. update /etc/apt/source.list
2. download repos key via wget
3. do apt-get update
4. do apt-get install zend-server-ce-5.2

I have init.pp file

class zendserverce {

# https://github.com/puppetlabs/puppetlabs-stdlib
file_line { 'debian_package':
    path => '/etc/apt/sources.list',
    line => 'deb http://repos.zend.com/zend-server/deb server non-free'
}

exec { "wget http://repos.zend.com/zend.key -O- |apt-key add -":
    path => ["/usr/bin", "/usr/sbin"]
}

exec { "apt-get update":
    command => "/usr/bin/apt-get update",
    onlyif  => "/bin/sh -c '[ ! -f /var/cache/apt/pkgcache.bin ] || /usr/bin/find /etc/apt/* -cnewer /var/cache/apt/pkgcache.bin | /bin/grep . > /dev/null'",
}

package { "zend-server-ce-php-5.2":
    ensure => "latest"
}

}

Seems that puppet runs commands in different order then I need. Is there any way how to for tell him to run in my desired order?

The output of such snippet is

  [0;36mnotice: /Stage[main]/Mc/Package[mc]/ensure: ensure changed 'purged' to 'latest'[0m
  [1;35merr: /Stage[main]/Zendserverce/Package[zend-server-ce-php-5.2]/ensure: change from purged to latest failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install zend-server-ce-php-5.2' returned 100: Reading package lists...
  Building dependency tree...
  Reading state information...
  E: Couldn't find package zend-server-ce-php-5.2 at /tmp/vagrant-puppet/modules 0/zendserverce/manifests/init.pp:28[0m
  [0;36mnotice: /Stage[main]/Zendserverce/Exec[wget http://repos.zend.com/zend.key -O- |apt-key add -]/returns: executed successfully[0m
  [0;36mnotice: /Stage[main]/Zendserverce/File_line[debian_package]/ensure: created[0m
  [0;36mnotice: Finished catalog run in 6.75 seconds[0m

So it says: Couldn't find package zend-server-ce-php-5.2

Can anyone guide me what is wrong?

Solution 1

You need to specify the dependency relationships. The easiest/cleanest approach is to use the require parameter which is available for all resource types.

package { "zend-server-ce-php-5.2":
  ensure  => latest,
  require  => Exec['apt-get update'],
}

etc..

Solution 2

Since Puppet 2.6.0 a new feature "relationship syntax" was introduced.

An example in Puppet 2.6.0 and above would look like this:

exec { "apt-update":
    command => "/usr/bin/apt-get update"
}

Exec["apt-update"] -> Package <| |>

Every time a package command is executed, the dependency (in our case 'apt-update') will be triggered fist. You can even define longer chains.

Solution 3

I tried previous variant but it doesn't work for me on Ubuntu 10.04

Finaly I prepared the following script, that runs update everytime the repository is older than one week:

exec { 'apt-get update':
    command => "/usr/bin/apt-get update",
    onlyif => "/bin/bash -c 'exit $(( $(( $(date +%s) - $(stat -c %Y /var/lib/apt/lists/$( ls /var/lib/apt/lists/ -tr1|tail -1 )) )) <= 604800 ))'"
}

Hope it helps.

Solution 4

I prefer to put apt-upgrade into a separate stage running before the main stage, so I don't have to hard-wire any dependencies. Check here: http://docs.puppetlabs.com/puppet/2.7/reference/lang_run_stages.html.

A simple example would look like below. It implies you have a separate class for doing the actual apt-update:

stage { "init": before  => Stage["main"] }

class {"apt-update": 
  stage => init, 
  apt_mirror => $apt_mirror 
}

Check my sample LAMPP-box on github to see how the pieces fit together: https://github.com/joerx/vagrant-lampp

Note: be careful with apt-upgrade, as some base boxes break by things like kernel upgrades.

Solution 5

In Puppet 3 this can be done by realizing virtual resources using resource collectors

# so you don't have to fully qualify paths to binaries
Exec { path => ['/usr/bin'] }    

# virtual resource
@exec { 'sudo apt-get update':
   tag => foo_update
}

# realize resource. filter by arbitrary "foo_update"
# tag and relate it to all Package resources
Exec <| tag == foo_update |> -> Package <| |>

View more solutions

Comments

[Jaro]

I read docs.puppetlabs.com/guides/language_guide.html#run-stages and puppet provides stages so you can explicitly specify ordering of execution. Run stage were added in Puppet version 2.6.0, you now have the ability to specify any number of stages which provide another method to control the ordering of resource management in puppet.

[Brandon Cook]

This may be "easiest/cleanest" but is it maintainable? If the Exec needed to change then you have to go back and replace every declaration, the point being this approach isn't referential.

[Aktau]

I actually use stages for this, but I have to admit that this trick is pretty nifty!

[spuder]

This doesn't work for me on 3.2.4 unless I change to fully qualified paths. @exec { '/usr/bin/sudo /usr/bin/apt-get update...

[Brandon Cook]

Good call I tested this having long ago added Exec { path => ['/usr/bin'] }

[yungchin]

Thanks, this was super helpful! For other novices like me wondering about that last line: "<|" is collector notation, see docs.puppetlabs.com/puppet/3/reference/lang_collectors.html

[Thomas Welton]

Awesome. This is a pretty awesome feature I didn't know about. Thanks for sharing.

[kaiser]

Link is dead like a mouse.

[joerx]

Fixed the link, but the relevant part of the manifest is commented out now. You can still see it though. Nothing is static ...

[r3cgm]

This seems to run on every puppet update whether a package install is necessary or not. Is there a way to tighten this up so that apt-get update only runs when a package is missing from the system?

[DrDol]

@r3cgm What is your package 'ensure' setting? If it happens also for 'ensure' => 'installed', the relationship syntax could be changed to: Exec["apt-update"] -> Package <| ensure == latest |> But then for every non 'ensure => latest' package, apt-update will not run.

[r3cgm]

Thanks @DrDol! We have ensure set to 'present' but may be moving toward an ensure 'latest' model soon. I think we kind of have a catch 22 issue here. If we bind an apt-get update against "ensure == installed" then it'll run the apt-get update every time before seeing if the package is there. That's the original scenario we're trying to avoid; running the apt-get update all the time. But if we bind apt-get update against "ensure == latest" only then whenever we find a missing package we'll have missed out on doing the apt-get update before doing the install and we might install something old.

[Kevin Cox]

@BrandonCook Then you can give your exec a new name. exec{update: command => "apt-get update"} then reference it as Exec[update].

[Felix Frank]

It's actually not possible to have Puppet sync the exec only if the dependent package resources need syncing. Such things only work by sending events through notify/subscribe. This in turn would only work if the packages do synchronize before the exec. A middle ground may be found in a schedule to have the update run only once per night. - Generally note that the collector syntax has a drawback - it will realize all virtual package resources that may end up being part of the manifest.

[mc0e]

If you do this for a number of repos, then you might wind up doing more apt-get updates than you'd like to. Puppet's "stages" are a feature to be careful of, but this is the best use case I know of.