Chef is an open source cloud configuration management and deployment application. It's meant to help anyone orchestrate servers in a cloud or just in a departmental data center. Instead of system administrators sweating over management programs that were designed for single, stand-alone servers, Chef allows DevOps to spin off dozens or hundreds of server instances in less time than it takes you to read this article.
Chef enables programmers and system administrators to work together. Instead of developers writing applications and then waiting on Ops staff to figure out how to deploy the software, Chef serves both communities. Chef moves the process from a herky-jerky release cycle to a continuous delivery model by enabling an effective and automated workflow—a central goal of the DevOps movement.
Net result? You get more work from your existing IT crew, and everything in IT moves faster with fewer errors. When defects do crop up (because this is the real world), they get fixed sooner.
The “cloud configuration management” part of it, enabled by Chef, is done by turning infrastructure into code, making the process flexible, versionable, human-readable, and testable. Infrastructure as code enables management of both cloud and on-premises resources. Ultimately, Chef is a framework to automate and manage infrastructure and applications. Specifically, Chef translates system administration tasks into reusable definitions, known as cookbooks and recipes. In a recipe, Chef authors define a system's desired state by writing configuration code. Chef then processes that code along with data about the specific node where the code is running to ensure that the desired state actually matches the state of the system.
Wait, why do you even need this?
Aren’t system administrators and developers doing just fine with shell scripts and manual installation routines? Actually, no. The old system administration and deployments methods don’t scale worth a damn.
Let’s look at a concrete example. Say the CEO orders changes to the company’s e-commerce site. Naturally, you (as a developer or Ops professional) decide to do A/B testing with both the existing e-commerce program and the new one. Using old-school methods, you need to set up servers, bring up the e-commerce software, load the data, and arrange for a group of test users. That takes time.
Automation makes the process far more scalable. Using Chef, you simply clone the existing platform to a test platform. There’s no need to set up servers or clusters by hand. Your test platform can be the public cloud you’re already using for production. The entire setup process can take minutes instead of hours, days, or weeks.
Next, you deploy the revised program to a user community from which you get immediate feedback. If the new site design doesn’t work, you don’t need to roll back manually. Instead, you use Chef to automatically roll back all your users to the old production program before you try again.
Or let’s say you want to try something new in your cloud setup. Chef helps you by providing recipes for common deployment scenarios. Want to deploy NGINX to see if it works better than Apache as a web server? Use the NGINX cookbook to deploy it to your Linux servers.
There’s no need to become an NGINX expert. You roll out NGINIX on a test bed, port your web programs using your own Chef recipes, and start benchmarking your transactions. Mission accomplished.
This is possible because so much is virtualized, especially in the cloud. A DevOps program like Chef bridges the gap between development and operations in the virtual IT world and automates administration processes.
The result? Simplified automation and deployment, dependability, and automated resource validation.
Chef basics
You can use Chef in any data center or cloud infrastructure. It runs on several platforms, including Windows; enterprise Linux distributions; AIX; FreeBSD; Solaris; Cisco IO; and Nexus. Chef supports cloud platforms including Amazon Web Services (AWS), Google Cloud Platform, OpenStack, IBM Bluemix, HPE Cloud, Microsoft Azure, VMware vRealize Automation, and Rackspace.
The Chef workflow starts with a team’s developers and system administrators defining a task to automate. The recipes and cookbooks capture the process, and are tested using tools like Test Kitchen, ChefSpec, and Foodcritic. Once they’re working properly, the recipes and cookbooks are deployed to the Chef server using the knife and chef command-line tools.
A typical Chef deployment comprises three core components: the Chef server, workstations, and nodes.
On workstations, you create and edit the Chef recipes, which are stored in cookbooks. As the name suggests, a Chef cookbook is a collection of related recipes that include configuration data and policy distribution. The workstation also stores a library of your infrastructure’s configuration documents. When you’re happy with your recipes, you save them to the Chef server using Chef Knife.
The Chef server is the repository for recipes, cookbooks, node policies, roles, environments, and node metadata. Policies map the organization’s business and operational requirements, process, and workflow to settings and objects stored on the server. Roles define server types, such as “web server” or “database server.” Environments define the type of process, such as dev, staging, or production. In short, the Chef server holds all your infrastructure’s configuration data.
The most important part of any cookbook—whether from Chef or Julia Child—is its recipes. You can think of recipes as small programs written in a domain-specific language (DSL) dialect of Ruby. Because the recipe DSL is a Ruby DSL, anything that can be done using Ruby can also be done in a recipe. That approach makes it easy to get started with Chef, yet still allows power users the flexibility they need to create code for customized solutions.
You don’t need to start from scratch; that’s the point of beginning with a recipe as a template. Over 3,000 cookbooks help you to set up web servers, application servers, database servers, and dozens of other types of servers. You then customize them for your particular needs.
The desired states of your servers are called “scenarios.” For example, a scenario might describe everything you need to install and configure a LAMP stack for a given type of server. Recipes within a cookbook specify which resources to use and the order in which they are to be used.
You store completed recipes in a cookbook on a Chef server. The recipe is then synced with the node in its next check-in, which happens on a regular, preset schedule.
A REST API makes these available to nodes—your physical and virtual servers—and workstations. The recipes and associated data and instructions are then automatically deployed to the nodes at your command.
The Chef client runs on the nodes (the actual application or web servers), using knife and APIs to talk to the Chef server.
The client, in turn, performs the node’s assigned actions. The deployment configuration work is done as much as possible on nodes, not on the server. This helps avoid server bottlenecks and enables Chef to scale for massive node deployments.
That sounds a little complex, so let’s consider an example. For instance, if you wanted to deploy an application based on Django, you’d build a cookbook and its recipes on your workstation. In this case, besides Django, you’d typically need the Apache web server, a MySQL database, memcached, RabbitMQ, and a Linux distribution.
After you write the recipes that detail what and how to deploy these services in a Django cookbook, you store the cookbook on the Chef server. The cookbook’s instructions are deployed from the server to the nodes with the Chef client. Once your Django setup is in place and running, you write the recipes and cookbooks to deploy your Django application.
When the administrator or developer makes a change on the workstation, it is transmitted to the server. Once there, you arrange for it to be sent to the node. That makes it possible for you to roll out patches and updates across your entire infrastructure by changing a recipe, without any need to interact with each machine individually.
As with most open source tools, there's an established company behind it. Chef is available in four versions: Chef Basics (formerly Open Source Chef, the free open source version); the software-as-a-service (SaaS) Hosted Chef; and Chef Automate (with an on-site Chef server). There is also a version of Chef just for Amazon Web Services, AWS OpsWorks for Chef Automate, which provides a fully managed Chef server in AWS. The pricing for each reflects its support and promised availability.
Whichever way you go, deploying Chef is trivial. The company provides free virtual servers for Red Hat Enterprise Linux (RHEL) 7 or CentOS 7; Windows Server 2012 R2; and Ubuntu 14.04, with a variety of clouds and virtual machines. Or you can also download Chef. The Chef Development Kit gives you tools to develop and test your infrastructure on Debian, RHEL, MacOS, Ubuntu, and Windows.
Chef versus Puppet
Chef is one of the big four of DevOps tools, in company with Ansible, Puppet, and Salt. Each has its good points and deserves in-depth analysis of its own, which is not provided here.
I can’t talk about DevOps tools without at least saying a bit about Puppet, arguably Chef’s chief rival. Puppet uses a model-based approach, in which you describe a desired configuration state in a Puppet manifest. Puppet then directs the servers to achieve the desired end state.
Perhaps the clearest difference between Chef and Puppet is that Chef caters to creative developers while Puppet works best for cautious system administrators. Or think of Chef as being heavily reliant on code, taking a programming approach to large-scale system administration. Puppet’s model-based approach asks system administrators to describe the desired configuration state in a Puppet manifest, whereupon Puppet directs your servers to achieve your desired end state.
Which is best? It depends on your team. Investigate the features of each tool in depth, and consider attending a conference where you can get hands-on experience. It’s more a matter of style than substance. Each works extremely well for the right users.
Of course, Chef isn’t for everyone. If you don’t know Ruby, it has a steep learning curve. It’s also not simple. You can create large, complex code bases. Those are easier to troubleshoot than old-style sets of shell programs, but they can still be time-consuming to debug.
The alternative to Chef (or something like it) is a lot more manual work. You would have to deploy complex applications to thousands of servers with their associated patches, security holes, and updates using your good old system administrator tools. Who needs that? Chef lets you spend your time doing things that are far more productive.
I think you’ll find Chef serves up a very tasty diet for long-term corporate IT success.
Mastering Chef: Lessons for leaders
Chef applies a continuous delivery model to the software release cycle by automating its workflow.
System administration tasks are saved as cookbooks and recipes, reusable configuration steps toward a desired state.
Chef has wide industry support across the cloud ecosystem, and works with just about every platform you can think of.