How to maintain site.pp with many nodes?

Question

I keep all my nodes in one file, site.pp - but as I add more and more nodes, it's very difficult to maintain them.

Import directive looks very promising, but as I understand docs, it's necessary to restart puppermaster every time something changes. For me it's unacceptable.

Is there any other way to do that? Instead of using big comments to separate nodes/groups. Now i use just rdoc.

I'll be glad with any suggestions :-)

My current puppet directory structure looks like:

• manifests/site.pp
• manifests/extdata/ (for extlookup)
• modules/module1
• modules/module2
• files/pubkeys...

I deploy puppet configuration using git/rsync to only override files that changed.

Solution 1

I have a site.pp that looks like this:

import "nodes/*.pp"

and have a nodes/ directory in manifests/

So that I have a set of nodes in "workstations.pp" "webservers.pp" and so on..

Solution 2

Define your nodes outside of manifests. I'd recommend extlookup's successor, Hiera, but really any external node classifier would suffice to move your node data out of manifests.

This is the recommended way to handle node definitions these days - from the docs:

Most users in most situations should use include-like declarations and set parameter values in their external data. However, compatibility with earlier versions of Puppet may require compromises.

---

Hiera's included in Puppet 3.0 - it needs to be installed separately in older versions. To set up Hiera to handle your node definitions, you'd want to do something along these lines:

site.pp (the whole thing):

hiera_include(classes)

hiera.yaml:

:backends:
  - yaml

:hierarchy:
  - %{clientcert}
  - os-%{osfamily}
  - common

:yaml:
   :datadir: /etc/puppet/hieradata
   # A good alternative if you want different node data based on environments:
   #:datadir: /etc/puppet/environments/%{environment}/hieradata

:puppet:
   :datasource: data

Now, Puppet will look in /etc/puppet/hieradata to pull data on your nodes. Say you have an ntp class you want on everything, and an apache class you only want on one specific node:

/etc/puppet/hieradata/common.yaml:

classes:
  - ntp

/etc/puppet/hieradata/nodename.example.com.yaml:

classes:
  - apache

This array is aggregate - the nodename.example.com node will get both the ntp class from the common file and the apache class from its own file.

Hiera also handles your class parameters for you. Say you have your apache class expecting a port parameter:

class apache ( $port ) {
  ...

You can set this in your Hiera data files as well. If you want it to default to port 80..

/etc/puppet/hieradata/common.yaml:

classes:
  - ntp

apache::port: 80

But you want to override that for nodename.example.com, setting it to 8080:

/etc/puppet/hieradata/nodename.example.com.yaml:

classes:
  - apache

apache::port: 8080

Or, you can use that os-%{osfamily} from the hiera.yaml file for settings based on facts about a given node - the osfamily fact in this case.

/etc/puppet/hieradata/os-debian.yaml:

apache::package_name: apache2

/etc/puppet/hieradata/os-redhat.yaml:

apache::package_name: httpd

(note that the parameter lookup behavior is a bit different if you're running a version older than 3.0, see here for details)

This way, you have the ability to set included classes and parameter/variable settings at different scopes (all nodes, some nodes based on a fact, or one specific node) all in different files.

Comments

[Kralik]

Looks very promising. Thank you for your answer, I'll definitely do some research about hiera and your solution.

[Stefan Lasiewski]

Keep in mind that import 'nodes/*.pp' is deprecated in Puppet 3.5 and will be removed in Puppet 4.

[Felipe Alvarez]

@stefan can you provide a solution or a workaround to import being deprecated?

[Stefan Lasiewski]

@Felipe There is no simple alternative that I know of. Classifying nodes using files like site.pp or import 'nodes/*.pp has fallen out of favor in the Puppet community, particularly by PuppetLabs. I'm still not clear what the best alternative is, but it seems that the favored workarounds are Heira or use an ENC such as The Foreman or Puppet Enterprise. Discussion of those alternatives are fairly detailed and worth a whole discussion. I did try: ask.puppetlabs.com/question/10975/…