Module 1 Overview of Chef and Workstation setup

 After completing of this webinar series you will be able to

o Automate common infrastructure tasks with Chef
o Describe Chef’s architecture
o Describe Chef’s various tools
o Apply Chef’s primitives to solve your problems

How to learn Chef

 You bring the domain expertise about your business and infrastructure
 Chef provides a framework for automating your infrastructure
 Our job is to work together to teach you how to model and automate your infrastructure with
Chef

Chef is a language

 Learning Chef is like learning the basics of a language

 80% fluency will be reached very quickly
 The remaining 20% just takes practice
 The best way to learn Chef is to use Chef

Overview

 Overview of Chef
 Workstation setup
 Node setup
 Chef resources and recipes
 Introducing the node object
 Setting node attributes
 Roles
 Community Cookbooks

Overview of Chef

 After completing the lesson, you will be able to

o Describe how Chef thinks about infrastructure automation
o Define the following terms :
 Resources
 Recipe
 Node
 Run List
 Search
 Items of Manipulation
Networking Powershell scripts
Files Users
Directories Groups
Symlinks Packages
Mounts Services
Registry keys Filesystems

Chef is infrastructure as code

 Programmatically provision and configure components

 Treat like any other code base (Version Control System)
 Reconstruct business from code repository, data backup and compute resources

Configuration Code

 Chef ensures each node complies with the policy

 Policy is determined by the configurations in each node’s run list
 Reduce management complexity through abstraction
 Store the configuration of your infrastructure in version control

Declarative interface to resources

 You define the policy in your Chef configuration

 Your policy states what state each resource should be in, but not how to get there
 Chef-client will pull the policy from the Chef Server and enforce the policy on the node.

Managing Complexity

 Resources
 Recipes
 Nodes
 Search

Resources

 A resource represents a piece of the system and its desired state

o A package that should be installed
o A service that should be running
o A file that should be generated
o A cron job that should be configured
o A user that should be managed
o And more
 Resources are fundamental building blocks of Chef configuration
 Resources are gathered into recipes
  Recipes ensure the system is in the desired state
 Configuration files that describe resources and their desired state
 Recipes can :
o Install and configure software components
o Manage files
o Deploy applications
o Execute other recipes
o And more

Nodes to adhere to policy

 The chef-client application runs on each node, which

o Gathers the current system configuration of the node
o Downloads the desired system configuration policies from the chef server for that code
o Configures the node such that it adheres to those policies.

Run list specifies policy

 The run list is an ordered collection of policies that the node should follow
 Chef-client obtains the run list from the Chef Server
 Chef-client ensures the node complies with the policy in the Run List

Search

 Search for nodes with Roles

 Find topology data
 IP addresses
 Hostnames
 FQDNs

How Application grows

 First everything on same server

 Lets move DB to other server as we grow
 Lets have another DB as redundancy as DB grow
 Lets add another server as Application scaling
 Our business is growing. To avoid performance issues add load balancer
 To avoid larger failure add another load balancer
 Our DB is responding slowly. So to avoid slow response add cache DB for faster DB read
response
 Put Round Robin DNS
 Are we monitoring?
 Do we have logging enabled (Centrally or on all servers)?
Manage Complexity

 Determine the desired state of your infrastructure

 Identify the resources required to meet that state
 Gather the resources into recipes
 Compose a run list from recipes
 Apply a run list to each node in your environment
 Your infrastructure adheres to the policy modeled in Chef

Configuration Drift

 Configuration drift happens when :

o Your infrastructure requirements change
o The configuration of a server falls out of policy
 Chef makes it easy to manage
o Model the new requirements in your Chef configuration files
o Run the Chef-client to enforce your policies

Recap

 In this section, we have

o Described how chef thinks about infrastructure automation
o Defined the following terms :
 Resource
 Recipe
 Node
 Run List
 Search

Three Components of Chef Configuration engironment.

 The chef server

 Your work station
 Nodes
Organizations

 Provide multi-tenancy in Enterprise Chef

 Nothing is shared between Organizations – they are completely independent
 May represent different
o Companies
o Business Units
o Departments

Sign on to chef website and create a new Chef environment, login and receipes, organizations.

Module 2 Node Setup Resources and Recipes

 You bring the domain expertise about your business and infrastructure
 Chef provides a framework for automating your infrastructure
 Our job is to work together to teach you how to model and automate your infrastructure with
chef.
 Learning Chef is like learning the basics of a language
 80% fluency will be reached very quickly
 The remaining 20% just takes practice
  The best way to learn Chef is to use Chef
 [Link] Learn Chef video
 [Link] Documentation.

Nodes

 Nodes represent the servers in your infrastructure these may be

o Physical or virtual servers
o Hardware that you own
o Compute instances in a public or private cloud

Lesson Objectives

 After completing the lesson, you will be able to

o Describe in detail what a cookbook is
o Create a new cookbook
o Explain what a recipe is
o Describe how to use the package, service and template resources
o Upload a cookbook to the Chef Server
o Explain what a run list is, and how to set it for a node via a knife
o Explain the output of a chef-client run
 A cookbook is like a “package” for Chef recipes
o It contains all the recipes, files, templates, libraries etc. required to configure a portion
of your infrastructure.
o Typically they map 1:1 to a piece of software or functionality

The problem and the success Criteria

 The Problem : We need a web server configured to server up our home page.
 Success Criteria : We can see the homepage in a web browser.

Desired State : Our Policy

 Apache web server should be installed

 Apache should be running and configured to start when the machine boots
 Our home page should be displayed
 Please not in this course we are teaching Chef primitives, not web server management
 This is probably not the Apache HTTP server configuration you would use in production.

Module 3 Working with Node object and Roles

 After completing of this webinar series you will be able to
 o Automate common infrastructure tasks with Chef
o Describe Chef’s architecture
o Describe Chef’s various tools
o Apply Chef’s primitives to solve your problems
 You bring the domain expertise about your business and infrastructure
 Chef provides a framework for automating your infrastructure
 Our job is to work together to teach you how to model and automate your infrastructure with
Chef

Introducing the Node Object

 After completing the lesson, you will be able to

o A not is any physical, virtual or cloud machines that is configured to be maintained by a
chef
o The ‘node object’ is the representation of that physical node within Chef (e.g. in JSON)
 Nodes are the objects that you manage with Chef
 They have a few different properties
o Attributes
o Run_list
o Chef_environment

The Chef Node Object

 Chef server stores node object data

 It becomes searchable through the API with knife and with recipes
 Some of the data comes from ohai, which takes an inventory of the system and emits JSON data
 You can add data to the node through attributes in cookbooks, roles, directly on a node, etc

Node Attributes Syntax

 Node attributes are hashes (of hashes)

 Reference hash keys with square brackets and keys as strings
o Node [“hostname”]
o Node [“kernel”] [“machine”]

Setting Node Attributes

 After completing the lesson, you will be able to

o Describe where and how attributes are set
o Declare an attribute with a recipe and set its value
 Attributes represent information from your node as well as input variables in your cookbooks
 Attributes hold static node data (e.g. platform, memory installed, disk space etc)
 Allow you to abstract out data in from cookbooks (e.g. directory paths, users, software version
numbers)
  All attributes are set on the “node object”, and are indexed for search on the server
 Attributes can be set at various levels (in increasing order of precendence)
o Automatically on the node itself(by Ohai)
o In roles
o In environments
o In cookbook recipes
o In cookbook attribute files

Roles (Roles based attributes and merge order precedence)

 After completing the lesson, you will be able to

o Explain what roles are, and how they are used to provide clarity
o Discuss the role JSON DSL
o Explain how merge order affects the precedence hierarchy

What is a Role?

 So far, we’ve been just adding our recipe directly to a single node
 But that’s not how your infrastructure works – think about how you refer to servers
o “It’s a web server”
o “It’s a database server”
o “It’s a monitoring server”
  Roles allow you to conveniently encapsulate the run lists and attributes required for a server to
“be” what you already think it is
 In practice, Roles make it easy to configure many nodes identically without repeating yourself
each time.

Components of a role

 Roles must have a name

 Roles may have a description
 Roles may have a run_list, just like a node
 Roles may set node attributes
o Default_attributes
o Override_attributes

[Link]

Attributes can be set multiple places

 This is for flexibility

 Set a “Sane default” that will be used in a cookbook
 Override it easily in a role (higher priority)
 There are many places where attributes come from
 In practice, you will use 2-3 most of the time.
 The others are there when you need them.

Checkpoint

 We now have a webserver role

 We could apply just this role on more nodes to scale out our virtual hosting service
 Roles are a great way to assign attributes for specific purposes.

Module 4: Common configuration with Data Bags

In last module

 Login to the node in your Chef training lab

 Install Chef nodes using “knife bootstrap”
o Included a run_list so that the server was a web server when the bootstrap process
completed
 Read and wrote node attributes

Where did my node go?

 We still need a CentOS machine to manage

 The one we launched last time has likely expired
  Launch a new one using the Chef Lab
o Hopefully, you’ve already done this. We’re not going to spend time walking through it
now.

Data Bags

 Data bags are generic, arbitrary stores of information about the infrastructure
 Data bag items are JSON data
 Our apache cookbook provides a good baseline
 We will drive site-specific virtual hosts with data bags.

A new recipe for virtual hosts

 We’ll create an apache::vhosts recipe to manage the virtual hosts we created in data bag items
 There’s number of new things to talk about in this recipe
 We will take this nice and show 

Think about what we just did

 We had two virtual hosts….

 But we could arbitrarily add more…
 Tigers on 82, Lions on port 83, oh my!!

CheckPoint

 Our cookbook has two recipes, default and vhosts

 Additional data bags can be added, expanding our Virtual hosting empire

Module 5 Environments
In the last module

 Refactored our apache cookbook to include multiple sites

Lesson Objectives

 After completing the lesson, you will be able to

o Describe what an environment is, and hoe it is different from an organization
o Set cookbook version constraints
o Explain when to set attributes in an environment

Environments

 Every organization starts with a single environment

 Environments reflect your patterns and workflow
o Development
 o Test
o Staging
o Production
o Etc
 Each environment may include attributes necessary for configuring the infrastructure in that
environment
o Production needs certain Yum repos
o QA needs different Yum repos
o The version of the Chef cookbooks to be used
 We cannot share cookbooks between organizations
 Best Practice: If you need to share cookbooks or roles, you likely want an environment rather
than an organization.
 Environments allow for isolating resources within a single organization.

Rollbacks and Desired State Best Practice

 Chef is not magic – it manages state for declared resources

 We just rolled back to an earlier version of the apache cookbook
 While the recipe applied fine, investigating the system will reveal Apache is still configured as it
was in the 0.2.0 cookbook

Module 6: Community Cookbooks Windows and Further Resources

After completing the lesson, you will be able to

 Find, preview, and download cookbooks from the Chef Community site
 Use knife to work with the community site API
 Download, extract, examine and implement cookbooks from the community site.
 We’ve been writing some cookbooks so far..
 Hundreds already exist for a large number of use cases and purposes. Many (but only a fraction)
are maintained by CHEF software, Inc.
 Think of it like [Link], [Link], or other focused plugin-style distribution sites.

The problem and the success Criteria

 The Problem: The chef-client application should run on a regular basis on our nodes.
 Success Criteria: Chef-client is running as a service on our node.

You can download cookbooks directly from the site

 It doesn’t put them in your chef repository

 It isn’t fast if you know what you’re looking for (Click, Click…)
 It isn’t necessarily fast if you don’t know what you’re looking for
 You’re already using knife for managing cookbooks and other things in your Chef repository
  Knife includes a “cookbook site” plugin with some sub-commands:
o Search
o Show
o Download
o …. And more!!

Download and use Chef-Client Cookbook

 Documentation for cookbooks doesn’t need to be extensive, but a README should describe
some important aspects of a cookbook:
o Expectations (Cookbooks, platform, data)
o Recipes and their purpose
o LWRPs, Libraries, etc..
o Usage notes
o Read the readme first!!
 So, you just downloaded source code from the internet.
 As root.
 To load in the magic machine that:
o Makes your computers run code
 Read the entire cookbook first!!

Best Practice: Delete the validation certificate when it isn’t required

 Once Chef enters the actual run, synchronizing cookbooks, it has registered its own API client
with the validation certificate.
 That certificate is no longer required. We do this first because in case the run fails for another
reason, we know at least the validation certificate is gone.
 The main point of the “chef-client” cookbook is managing the “chef-client” program. It is
designed that it can run as a demonized service.
 The least surprising thing for most users is that the default recipe starts the service.
 You can manage the service in a number of ways, see the cookbook’s [Link].
 Ideally, you don’t have to modify the contents of a cookbook to use it for your specific use case.
 Look at the attributes directory for things you can override through roles to affect behavior of
the cookbook.
 Of course, well written cookbooks have sane defaults, and a README to describe all this.

Additional Tools for Windows

 Chef-Client works on windows

 Windows-specific resource types
o Batch
o Powershell_script
o Registry_kay
  Knife plugins
o Knife windows
o Knife azure

Usual Suspects

 Most resources work the same way they would in a Unix or Linux like environment
 Including
o Package
o Service
o Template
o User
o Group

Special Attributes

 File based resources have attributes that are unique to Microsoft windows
o Inherits – for file inheritance
o Mode – for octal modes
o Rights – for access control lists (ACLs)

Boorstrap a windows node

 Install the knife-windows gem

 Configure windows remote management (WinRM)
 On the target node
 Knife windows bootstrap winrm….

Module 7: The first Chef-Client Run

After completing the lesson, you will be able to

 List all the steps taken by a chef-client during a run

 Explain the basic security model of Chef
 Explain the concepts of the resource collection
Private Keys

 Chef server requires keys to authenticate

o [Link] – private key for API client
o [Link] – private key for ORGNAME-validator
 Next, let’s see how those are used…

About the resource Collection

During the compile phase, chef

1. Loads all cookbooks from the run list

2. Reads each recipe to build the resource collection

During the execute phase chef takes the resource collection and for each resource it will

1. Check if the resource is in the required state

a. If ‘yes’ – do nothing
b. If ‘no’ – bring resource in line with required state
2. Move on to next resource.
Recipe order is important

 Recipes are executed in the order they appear in the run list
 These recipes are invoked in the following order
1. Recipe [ntp::client]
2. Recipe [openssh::server]
3. Recipe [apache::server]