No sooner than I drafted my last post, Opscode released released a huge update to Chef, 0.10.0.

Being on on vacation during the lead up to the release, I missed a few of the pre-release announcements, so while I’d heard that some big features, such as environments were coming, I hadn’t realized how many cool features were due for release in 0.10.0 and I thought I’d mention a few of my favorite and perhaps lesser known new features here.

Knife Plugins

Co-incidentally, the 0.10.0 release came out the same week that we started work on our first knife plugin, and the new architecture has made the process very straightforward. You don’t need to delve into to depths of your gems to extend knife, you can simply throw the plugin in a .chef/plugins/knife directory in your home folder or in your cookbook repo. There’s a quick guide to get you started on the Opscode blog.

Encrypted Data Bags

One of the first questions I get asked when talking about Chef generally relates to security. The new release being able to encrypt the contents of data bags is a big step forward on this front. This means you can now encrypt sensitive values which are stored in your data bags, such as database passwords. To keep things secure, you can also configure the decryption keys at a node level so that only nodes that should have access to the data can see it. @lusis also added a patch to support for storing decryption keys at a URL instead of a local file, a nice addition which should find its way into an upcoming release.

Chef Expander

This behind the scenes update will probably stay out of your way for the most part, but it’s cool all the same. Indexing is now taken care of by a new tool called chef-expander, which replaces the old chef-solr-indexer. The cool thing about chef-expander is the ability to setup a cluster of worker nodes to farm out the indexing process. Small installations are fine with just one process running, but it’s nice to know that this can scale horizontally as your infrastructure grows.

Cookbook Versioning

In my recent post cookbooks as gems I mentioned a few features I’d like to see in the cookbooks architecture, the main one being a straightforward way of managing different versions of cookbooks. Version 0.10 addresses this and more, including the ability to freeze cookbooks when uploading to the Chef server and the option to set cookbook version constraints in environments. Judging by the upgrades to the cookbook features in knife and their latest post on the community cookbook site cookbooks management is definitely a high priority in Chef I’m very excited to see how this evolves as Chef moves beyond 0.10

Upgrading

If the above features aren’t a good enough argument to upgrade straight away, check out the release notes for a full account of what 0.10.0 brings to the table. The 0.10.0 server is compatible with 0.9.x clients, and the process for upgrading both the server and clients is trivial. Instructions can be found on the Opscode wiki.

Update: Many of the questions below have been resolved in the 0.10.0 release of Chef and Opscode have also provided a great overview of the current state of the community cookbooks repository.

User Story:

• As a sysadmin
  • I’d like to version cookbooks and download custom cookbooks
  • so that I can better manage cookbook dependencies.

I’ve been thinking a lot about what the future of the Opscode community cookbooks site / API might look like.
Every way I think about it, I keep coming back to a model similar to that of Ruby gems and I’m interested in knowing if this view is shared and to what extent this parallel makes sense.
I think the cookbooks site as it stands is great and in some senses, the cookbooks site is really the heart of chef. Without having such an easy path to ‘vendor’ the apache2 cookbook for example, the experience of first time chef users might not be the wonderful experience it is today. There are however some cases which users might come across which don’t (at least obviously) have a solution in the current cookbooks site, and it would be great to see if the community can solve some of these. The scenarios that immediately come to mind are:

• “The new Y cookbook broke my X cookbook, I need the X cookbook to be dependent on the old version of Y until I can fix it ”
• “My friend just sent me a cookbook she wrote, which depends on a version of the apache2 cookbook which she modified, I’d like to install her apache2 cookbook and use it alongside the default community apache2 cookbook”
• “I’ve seen cookbooks on github, such as at 37Signals. Can I also use knife to install and manage these cookbooks?”

Rather than get into too much detail about those specific cases, I thought I’d provide an example of what a rubygems-esque cookbook management process might look like

knife cookbook install apache2 # installs the apache2 cookbook along with dependencies
knife cookbook install apache2 --version=1.02 # installs a legacy version of the apache2 cookbook
knife cookbook install agoddard-apache2 # installs my fork of the apache2 cookbook along with dependencies
knife cookbook install agoddard-custom_weird_app # installs my cookbook and dependencies for an obscure app that only I use but which I want to manage the same way
knife cookbook install custom_weird_app # the above cookbook when the obscure app becomes more mainstream

and how these might look in a recipe

include_recipe "apache2"
include_recipe "agoddard-apache2"
include_recipe "apache2", "=1.02"

..and..

%w{ apache2 agoddard-apache2 apache2-1.02 }.each do |cb|
  depends cb
end

I’m sure there’s things I’m missing here, but I’d love to see where this concept leads.. cookbooks already support versioning but I’m not sure if there’s a simple way of maintaining two versions and having legacy cookbooks support the older one. It’s obviously currently possible to manually install any cookbook you like, though you then loose the added benefits of using the knife site vendor command, such as branching and dependency resolution.

Another area this might help is in keeping cookbooks up to date. Currently if a user wants to update a cookbook, they will fork the cookbook, apply their changes and then send the maintainer a pull request. If the maintainer is unavailable, it’s hard for the changes to get back to the community. If in this case the user submitting the changes could simply upload their modified cookbook to the cookbooks site, prefixed with their username (to distinguish it from the main / official cookbook) then the cookbook will be available to the community while they wait for the official version to be patched, tested etc. This isn’t ideal, but it’s more ideal than an update to your chef server breaking an important cookbook which is reliant on a deprecated feature of the server, for example.

I’m sure there are many other approaches to this same issue and I’d love to know what others think. The cookbooks site is a fantastic feature and when I look at it, I feel like I’m looking at the beginnings of something big, like github or rubygems and I’m excited to see where it goes.