Chef Frustrations

I've spent the last week working on implementing chef. The experience is frustrating to say the least. Instead of whining I wanted to take the time to write out some of my pain points and hopefully offer some constructive fixes to what I see as the wall in the learning curve.

Now to be clear up front. Most of my problems aren't with Chef, Ruby, or most of the core product; it's with implementing it. To be more precise I think the failure REALLY is documentation.

Anti-pattern One: Getting Started (into a corner)

Also known as the "Just enough to be dangerous but not useful" anti-pattern

I really liked the new learn chef. I have to give them a ton of credit for all the work but underneath all the new splash and presentation it's still the exact same old Chef 101 it was two years ago; it teaches you the barest of all basics and then drops you off to

I know that most would feel that statement isn't fair, since it teaches you all about the design and system behind how chef works, and that it does; but it still feels like not enough to be useful and here is why.

Anti-pattern Two: We Have no Patterns...

Learn Chef teaches you how chef works but not really how to use it at any level of scale; There is no real world usage taught anywhere. It teaches you to set up a Chef Enterprise server and then re-inventing the wheel with a homemade apache or ntp cookbook, and push it all to a vm but you would rarely do this in practice right?

When you leave Chef's documentation you learn about many very important Chef Patterns;

  • wrapper cookbooks
  • berkshelf way
  • one repo per cookbook vs monolithic repo
  • application cookbooks
  • service cookbooks

Why doesn't chef teach us these? Is this something we save for consultants to teach us at thousands of dollars an hour? Is it that Chef wants to avoid teaching patterns in order to remain as flexible as possible1?

It's not just chef either. Go to and tell me how to use this tool assuming you've never done such before. If I was trying to remember a few commands or learn a new trick on top of something this tools docs would be great but it's missing the meat of what this tool is designed for and how to use it. A lot of chef's tools are treated this way.

Anti-pattern Three: ...So please learn everyone else's anti-patterns

This is my biggest frustration, OPD; Other People's Docs. As someone who has been working in Systems for 10+ years I have lived and learned so much from everyone else's blogs, which is why I feel the need to blog all my own lessons and information.

I feel that chef relies too much on OPD though. Especially because chef is such a fast moving target. It's amazing how many people who use chef that I talk to that use it in some odd, bizarre, and or generally 'not correct' way. It's usually because they learned a bad habit from a predecessor or found a bug in a long ago version and found some OPD that convinced them that "oh no you have to run everything chef-solo with your own special bootstraps, that is the ONE TRUE WAY™". I'm not saying that patten doesn't work but I doubt it's the best way for many infrastructures2.

I plan on documenting plenty of chef like things myself; in fact I plan on posting as much of my own OPD as possible but with how fast chef evolves as a product and with the large variance of methods for different environments I really hope people take everything with a grain of salt and read the date on the post when consitering my advice.

Here is a great example; where about 2014-07 I went into #chef and asked about some methods for setting things up and was linked to this blog which is treated like a defacto example of how to do things. But read all those updates... and then notice how it's using a lot of deprecated methods. I was linked to an article that could be titled "How to develop some really bad habits, but learn important things while you are at it." It's not Mischa's fault, It doesn't seem like he is a docs writer for Chef. Honestly I feel the best thing that could be done is this document be updated to the latest methodologies and tacked on to the end of learn chef as "One good method to get your enviroment up and going".

As a chef user do you even know about chef-dk? you probably should take a break from what you are doing, read this and then do this. Seriously don't you feel much better? This also should be on the end of learn chef guide. Hell this should probably be the first half of the learn chef guide.

I get that maybe they don't want to declare a "chef way" to do things... but at least give us some better hints.

Next Actions

Just to recap;

  • I believe chef's biggest weakness is documentation, which creates a wall in the learning curve to hit right after "I can now build and deploy a test apache on a linode" and "I can build and deploy this in a staging enviroment"
  • I think there should be a learn chef 200 series that goes over;
    • Using a wrapper cookbook, and the different types of abstraction you often see with these.
    • Teaching everything chef-dk adds; bootstrapping, runtests, and automated integration testing.
    • Highlighting several useful patterns for cookbook development.
    • Using more of chef's tools; ex ohai
  • If chef is going to rely on the community for docs maybe it should create a way where they can contribute to the main docbase just like they do code.
  • go here, have your life changed
  • If you are in the Las Vegas, NV area come hang out at #lvdevops on freenode and tell me how I make you feel
  • I'm going to spend another week or two trying diferent ways to structure my cookbooks and see what works.

  1. I believe this is a horrible anti-pattern in documentation. If you believe your power is flexibility then you should highlight that but still outline some predominate patterns for your top two or three use cases. 

  2. I know it's not the best way because they are deprecating chef-solo for chef-zero, which is good but it's a great example about the speed that Chef is changing. 


Where have you been?

Actually I should have called this where have I been. This seemed catchier though.

In short the answer is I have been at my new job at They have keep me as busy as can be. Because of this major shift I changed a lot of habits; I stopped writing here as much, I stopped actively contributing to Pelican1, and I also stopped posting to github.

Be not too afraid. I still write plenty of code and fille around with everything. I just stopped posting most of it to github. I mostly use my own private stash instance. There is some publically accessable code for those who are interested. The reason is that stash is SIGNIFICANTLY cheaper for my private repos so I just put most of my stuff there. I have been rethinking that lately since I miss a lot of the github community style.

I haven't been blogging much since I have just been focusing most of my documentation and writings to work, and got in the odd habit of putting eveything in my private wiki. For the sake of sharing and hashing things out I am going to refocus on using this blog to document out useful things. Hopefully this means you cna excpect floods of useful things. Maybe some smaller posts too.

On the other side I have been busy with my girlfriend, cycling, dba courseware, studying Japnaese, and my unstoppable anime habit. I have my own house now and a workbench set back up so I may blog about those projects as well...

Anyways. No one loves a vanity post but I wanted to at least put an update out

  1. I'm really sad about this but my personal time dropped sharply between a new job and new girlfriend. 

Coming back to vim

It's time for my monthly or so post! I wanted to go through and post about my OpenBSD firewall I built but that's not 100%. Also I'm not ready to go on about anything amazing with puppet because without my lab being done puppet isn't useful so lets go back to talking about my dev environment!

I know Justin has been asking for this for a little while.

Preface: Going "back" to vim

As a sysadmin at work I use vi a lot. Not even vim; vi. We have lots of unix boxes that default to vi as the installed editor and we don't just go installing vim on everything. Personally I use vim a good amount on my machine since I spend a lot of command line time anyways. I know more than just a few of the commands but I really only consider myself a second or maybe third year vim user1 since I never used it full time to write code. I live the motion and use things like ci[ and C-v 5j x but I still fail to use multiple registers, buffers, or tabs… or even the leader commands.

I'm an amateur software developer at best; I have serious aspirations about seeing if I have the chops to go pro but right now I'm honing edges. Irregardless of how developer or not I may be, I am developer lazy so I spend money on tools that make my life easier. I've been using PyCharm to help me write utilities and my mini apps and it's just the best. Sometimes I worry about leaning on IDE tools stunting my abilities so I took some time a month back to stand up and step back from PyCharm and instead just use vim…

This is the setup.

Part One: My keyboard

I use a slick trick on my Mac so I have no access to Caps and my CapsLock key acts as BOTH a Ctrl and Escape. If I tap the Caps it's esc, if I chord it with anything else it registers Ctrl… I pinkie reach for nothing. Here is the the instructions, 10.8 approved so YMMV for other versions.

  1. Go to System Preferences -> Keyboard -> Modifier Keys. Set Caps Lock to ^ Control.
  2. Install KeyRemap4MacBook - Software for OS X.
  3. In KeyRemap4MacBook, enable Control_L to Control_L (+ when you type Control_L only, send Escape. Search will help.
  4. Reboot and enjoy.

Part Two: The Development Server.

You didn't think I was just going to vim and take off did you? No.

If I'm going to work from the shell I want to make it so I can work from anywhere while I am at it. I have a Mac OS X server that would love to be my dev box so away I go. Open some ports, SSH keys, virtualenv, python3 from brew… tada! But it's not ready yet.

OpenSSH is my best friend. I keep keys close at all times and use cools scripts on my laptop to help manage them. However SSH is not enough and this is where Mosh: the mobile shell comes in. Mosh isn't a total end to end transport solution but it's high speed udp style and local echo features make it supreme when then connection starts lagging and you don't want it to slow down your code. Best yet? brew install mobile-shell on both boxes… done…

If only we had a windows client already.

If you want some portable keys help check out the following;

Just remember to not make these your ONLY keys, all posable keys should be password encoded and easily revokable so keep a backup and list of your emergency to revoke when it gets lost.

Part Three: The Terminal

I need a sweet terminal so I use zsh with oh-my-zsh and a while bunch of personal mods. Remember the whole lazy part? Yes. Here is the highlights of my zsh configs;

The second major part is tmux. Whatever you are using now… drop it and use tmux. I remapped all my common tmux commands to vi-mode style and C-a for my leader because now ctrl and a are touching. For the full list of my configs which I won't get too deep into check out tmux.conf at master · tbunnyman/dotfiles · GitHub.

My main tmux window generally looks like this

    |    Chat   |           |
    |     or    |           |
    |extra shell|    VIM    |
    |-----------|           |
    |           |           |
    |    IRC    |-----------|
    |           | MiniShell |

Chat is my flux buffer that gets changed between a personal chat and second work buffer. IRC is my ever present wee-chat connection. The mini shell is a little shell I keep in the same dir as vim so I can quick run python -m unittest discover module over and over or whatever. When I'm playing with Flask it's running there. Depending on where my focus is the vertical split is usually about 65% weighted to the work to squish distractions without cutting them all out or I am on the 11' MacBook Air screen instead of a 20+' external display.

I often have a second window but the latest version added C-a z for window zoom and that's GREAT when I really wanna focus on something or blow up the mini-shell while I am debugging something.

Part Four: Into VIM

First and foremost I keep an 8.5x11 copy of Beautiful Vim Cheat-Sheet Poster & Printable Downloads on my desk. It's a nice way to keep reminding me of all the features I NEED to be using and if you don't want to give someone 10USD for it there is a free link right on the page for a low res.

tl;dr the configs

dotfiles/vim at master · tbunnyman/dotfiles · GitHub

A lot of my inital vim config like most was stolen from somewhere but over time I have stripped out everything I didn't adapt in. I started with YADR and then seriously hacked it to death. In the end there is still yadr references but you shouldn't take anything that claims being from yadr in there still is. I'm just lazy about renaming files for scuz.


There you go! I know it feels like I'm skimming the VIM part of the vim writeup but there is really only so much you can do TO VIM itself. It's the development environment you put around it and what you put out with it. Hopefully I will be putting out great things once I learn how to use tags and rope and all that other stuff to get back to ultra fast code sifting and editing.

Part Five: Wishlist

Auto-running tests

Just something that PyCharm and Komono before that spoiled me on. :w running my tests since I very often TDD would save a lot of window jumping


I'll never get deep code intelligence with vim and that's kinda the point but PyCharm saved somewhere around a billion keystrokes when you learned when to hit the auto complete right.

Part Six: Warnings

This allows me to do some awesome stuff and so far I am happy with with it outside of a few small caveats.

  1. It's damn fiddly. So much and learn and fiddle with distracts from the work.
  2. Sometimes cruising around inside of vim, inside of tmux can make for some finger dancing that I don't care for; C-a l C-a k C-w l… until I trip over my own keystrokes. C-a ; is really useful when popping between panes when it comes to mind

Part Seven: Going forward

I can connect in technically from anything I trust enough to plug my key jump-drive into. I currently have bought a YubiKey and am seriously considering switching right over to two-factor OTP which makes me LESS afraid of plugging in the key into something.

Another area I am considering going forward with is I technically won't even need my computer to work. I could just work with my iPad and a keyboard! I'm really sure these articles had just a little bit to do with my idea of moving over to all command line vim. I don't know if I am there but it is tempting;

Was this more in depth than you expected? Do you want more? Lemme know.

  1. See Just Use Sublime Text - Andrew Ray's Github Blog for details on what I mean by that. 

Tagged ,