Thoughts and learnings in my journey through all things technology. Developer operations, software development, server administration and anything else that I decide should be written.

Tags


Foreman: Enterprise Puppet without the Cost

19th August 2014

Overview

System automation at one point used to be a nice to have for system administrators and was thought to only be useful in large corporations. The introduction of Puppet, Chef, Salt, Ansible, etc has changed automation and configuration to become a requirement for most DevOps / System Administration teams.

This blog post is about my experience with the Foreman.

Preface

I've had the experience to work for several companies with and without automation frameworks in place. I've had exposure with Puppet for a long time but it was not ever really utilized by my system administration teams. When I went to NOOK we were moving forward with Puppet in a big way; every new project and existing projects were being massaged into Puppet and it was also a necessity for me to be able to have my development team utilize Vagrant. So thus I was able to kill two birds with one stone.

Since that time, I've been using Puppet for every project I work on as well as ensuring that our systems are configured in the way that they were designed. At first I attempted to utilize Puppet Enterprise; during that time I found their installation process to be difficult, often failing and brittle at best. I would have to dig in and resolve configuration and determine actual requirements of Puppet Enterprise. Then came Foreman which I utilized at another position. Foreman installed with a breeze, has a healthy community, a better interface and best of all did not have the instanity of the cost that Puppet Enterprise comes with.

Getting Started

NOTE: I have in the past had issues with the installer and loading the AWS provisioner so I am unable to speak to that area at this time.

Getting started with the Foreman is as simple as downloading the installer and running the wizard. Since I am running ubuntu servers, I will show the instructions from that standpoint.

echo "deb http://deb.theforeman.org/ trusty 1.5" > /etc/apt/sources.list.d/foreman.list  
echo "deb http://deb.theforeman.org/ plugins 1.5" >> /etc/apt/sources.list.d/foreman.list  
wget -q http://deb.theforeman.org/pubkey.gpg -O- | apt-key add -  
apt-get update && apt-get install foreman-installer

Now that we have the installer it is time to answer some questions about our installation, however, before we go forward here, we need to ensure that our server is in order.

Server Setup

A simple checklist that I utilize when creating my server for Foreman. Most of these relate directly to your puppet installation.

  1. Setup your hostname

    • /etc/hosts file should be FQDN, hostname, localhost

      vim /etc/hosts
      127.0.0.1 puppet.mydomain.com puppet localhost
      
    • /etc/hostname should be FQDN

        vim /etc/hostname
        puppet.mydomain.com
      
  2. Reboot the server; while generally not required it's a good time to do it.

  3. Run the installer

       foreman-installer
    
  4. Login to https://puppet.mydomain.com

    • Username: admin
    • Password: changeme

Next Steps

Now you have a functioning puppet master, on the machine you should now run:

puppet agent -t  

This will now register the host within the puppet master.

You can now start adding hosts and managing modules within the puppet master.

Adding Hosts

If you are utilizing provisioning much of this is already handled. However, this is the simple steps that I have taken to add in a host and ultimately have it configured.

Host Setup

SSH to the target host machine and ensure that the following items are setup:

  1. Setup your hostname

    • /etc/hosts file should be FQDN, hostname, localhost

      vim /etc/hosts
      127.0.0.1 host.mydomain.com host localhost
      
    • /etc/hostname should be FQDN

        vim /etc/hostname
        host.mydomain.com
      
  2. Reboot the server; while generally not required it's a good time to do it.

  3. Setup the Puppet Client

       wget https://apt.puppetlabs.com/puppetlabs-release-trusty.deb
       sudo dpkg -i puppetlabs-release-trusty.deb
       sudo apt-get update
       apt-get install -y puppet
    
  4. Add in your Puppet Server under the [main] block

       vim /etc/puppet/puppet.conf
       server=puppet.mydomain.com
    
  5. Generate the Certificate Request

       puppet agent -t
    
  6. Sign the Certificate through Foreman

    • Infrastructure -> Smart Proxies -> Certificates
    • Press the sign button near the target host
  7. Ensure you can communicate with the puppet master

       puppet agent -t
    
  8. Assign Puppet Modules and/or Roles (See Puppet Modules below).

    • Check your work (puppet agent -t)
  9. Once your host is all configured to your liking it's time to ensure the puppet agent is continually running:

       sed -i'' 's/START=no/START=yes/g' /etc/default/puppet
       puppet resource service puppet ensure=running enable=true
    

Puppet Modules

One area that is often missed when people are utilizing puppet is that they do not manage their modules. I generally audit each module I add into my stack as well as make several modifications to them. Some which end up in the upstream and others that do not.

The best place to find modules is Puppet Forge. I generally have my own repository that I clone the module, make any necessary changes and then commit them locally.

What I mean by this; is I ensure that they run under puppet-lint through the use of Jenkins. I also have jenkins automatically deploy modules to the puppet master when complete.

Manually Adding Modules

Many people utilize existing modules as they are and assign them directly. While I do not subscribe to this practice it does not mean that it is not a generally accepted scenario.

puppet module install -i /etc/puppet/environments/production/modules saz/ntp  

Now you will need to import the module(s) into Foreman.

Importing Puppet Modules

One necessity is to import puppet modules when you add or update modules. This action is taken in the administration area under Configure > Puppet Classes. Then pressing the "Import from ..." button in the upper right. Upon completion your classes will appear in the Puppet class list.

Following a Pattern

A quick note here on a way of organizing your infrastructure that I have found successful has been dividing into Roles and Profiles. This presentation on Designing Puppet goes a long way into the explaination.

This allows you to have a profile such as profile::webserver and then assigning it to a role such as role::productionweb_server. By doing this roles can have multiple profiles assigned and enable you to re-use many of your definitions.

Attaching Classes to a Host

Once everything is imported, attaching a class to a host is insanely simple. Navigate to Hosts and click the Edit button next to the host you would like to configure.

Select your puppet class and press Save. Within the next 30 minutes (or however long your servers are set to check-in) your class will execute. If you are looking for immediate results SSH to the server and run:

puppet agent -t  

Conclusion

While this is brief overview of how I utilize Foreman there are many other facts that I did not highlight such as:

  • Provisioning
  • Smart Variables
  • Smart Proxies

There are many features in Foreman and even additional automation framework support (Chef). It is a DevOps tool that has greatly improved the support of our systems. We can now easily create new boxes, configure and replicate existing machines and I can keep my sanity that I know they are setup properly.

I am a VP of engineering for a small start up. I have over a decade of experience in engineering, database administration, server administration and management.

View Comments