chef install and update programs from source

Question

I have a program that I build from source. For this I'm using the script resource. What is a good way to implement the logic for installation and update? Right now I just have installation implemented with the built-in not_if conditional.

script "install_program" do
  not_if {File.exists?('/program')}
  interpreter "bash"
  user "root"
  cwd "/tmp"
  code <<-EOH
    wget http://www.example.com/program.tar.gz
    tar -zxf program.tar.gz
    cd tarball
    ./configure
    make
    make install
  EOH
end

Solution

First and foremost, if you have the means to host an internal package repository, I generally recommend that you build native packages for your target platform(s), and use the package resource to manage them, rather than building from source. I know that is not always available or feasible, so ...

The method which you make a "./configure && make && make install" style installation script idempotent depends on the kind of software you're working with. Most often, it is sufficient to check for the target file's existence. Sometimes, it is desirable to determine what version is required, and which version the program will output when executed with the proper command-line option. I will use your resource above as a starting point for these examples. Note that you can use bash as a shortcut for script resources that have interpreter bash.

Assumptions: The program is installed to /usr/local/bin/program and takes an argument --version presumably to display the version number. I put the cd, configure, and make commands together with && because presumably if one fails we shouldn't attempt to continue execution.

bash "install_program" do
  not_if "/usr/local/bin/program --version | grep -q '#{node[:program][:version]}'"
  user "root"
  cwd "/tmp"
  code <<-EOH
    wget http://www.example.com/program-#{node[:program][:version]}.tar.gz -O /tmp/program-#{node[:program][:version]}.tar.gz
    tar -zxf program-#{node[:program][:version]}.tar.gz
    (cd program-#{node[:program][:version]}/ && ./configure && make && make install)
  EOH
end

Instead of using wget it is a bit better to use the remote_file resource as this is idempotent on its own. Note that the checksum parameter is added, with the value as an attribute. This parameter tells Chef not to download the remote file if the local target file matches the checksum. This is a SHA256 checksum. Also, this resource will notify the script to run immediately, so after it is downloaded. The script is set with action :nothing so it only gets executed if the remote_file is downloaded.

remote_file "/tmp/program-#{node[:program][:version]}.tar.gz" do
  source "http://www.example.com/program-#{node[:program][:version]}.tar.gz"
  checksum node[:program][:checksum]
  notifies :run, "bash[install_program]", :immediately
end

bash "install_program" do
  user "root"
  cwd "/tmp"
  code <<-EOH
    tar -zxf program-#{node[:program][:version]}.tar.gz
    (cd program-#{node[:program][:version]}/ && ./configure && make && make install)
  EOH
  action :nothing
end

Also, /tmp may be erased on your system upon reboot. It is recommended that you download to another location that isn't deleted, such as Chef's file cache location, which is the value of Chef::Config[:file_cache_path]. For example:

remote_file "#{Chef::Config[:file_cache_path]}/program.tar.gz" do
  ...
end

For further examples, you can see "source" recipes in several cookbooks shared by Opscode here: http://github.com/opscode/cookbooks. php, python, gnu_parallel, and nagios cookbooks all have "source" recipes.

Comments

[Evgeny]

NOTE: A bug with remote_file can sometime ungzip a file with a .gz extension, like tar.gz. So tar -zxf will fail because it is not really a gzip.

[Rahul Patil]

@jtimberman could you please explain more, where from program and version variable are set

[jtimberman]

they're arbitrary attributes i used as an example. They're set as node attributes. docs.opscode.com/essentials_node_object.html#attributes

[pitchblack408]

I know this post is old, but how can you ensure that that the script doesn't run over an over again. If it is installed from the /tmp folder then your trick for stopping the install occur again wouldn't work.

[David Grayson]

How can you ensure that the script runs at all? Suppose when you run chef the first time that the remote_file resource succeeds but the install_program resource fails. Then when you fix the issue and run chef again, the remote file is already present so it doesn't bother executing any actions on it, and it doesn't notify the install_program resource, so the program doesn't get installed. I'd like to find a better pattern for installing software from source in chef.

[jtimberman]

Right, you really can't ensure it, or it's really hard because of the nature of what you're doing. Installing programs from source is very hard to make reliable and repeatable. That's why the first thing in my answer is that I suggest building packages and running an internal package repository. Package Cloud (packagecloud.io) is a great service that will do this for you for yum and apt repositories.