Monthly Archives: June 2014

Introducing Soos – A Docker Workflow Template for Devs

I was recently handed the devops reins over at Djed Studios, I was starting from a fresh slate and ready to do something cool. With all of the hype surrounding Docker, I was especially eager to give it a shot.

I reached out to Flux7’s Aater Suleman, who spoke at last month’s Devops Days Austin on “Using Docker to Improve Web Developer Productivity.” Unfortunately, the timing of my assumption of devops responsibilities did not give me enough lead time to get into Devops Days, but Aater was kind enough to meet me at a local Starbucks to discuss his presentation. I am forever in debt to the kindness he showed me in this.

Our Stack

Our development stack is fairly standard. MacBook Pros, VirtualBox, Vagrant, Django, and postgresql. Initially, we had rolled with one custom vagrant box (“djedbox”) which was monolithically built using the same SaltStack states that built our production website. This worked, but was very time intensive, and did not give us the flexibility of rebuilding fractions of the development environment.

Want to drop and rebuild the database, with test data, without having to rebuild the python virtual environment too? Well, too bad: our states were so intertwined as to make this infeasible. I hope you are a seasoned postgres admin, because that is what it took. Not all of our team is that comfortable with the unix command line, though, much less with the intricacies of the postgres dropdb and createdb tools, granting privileges, and so on. Initially, we wrapped some of these things in shell scripts, but the scripts had to evolve in sync with the salt states and it turned into a nightmare.

Docker

I threw pretty much everything away when we moved to docker. Salt went right out the window; if I was going to make this move, it was going to be docker on the dev, docker in testing, docker in production. End to end docker. The only difference between our dev environments and our production is that the database in production is its own instance, whereas in development it is another container.

Our devs regularly want to revert back to a known environment, something they can do by axing their current container and starting a new one from a known-good image. The first time a develop runs our scripts, they get a local image from which to base their work and they can quickly and painlessly return to this at any time, regardless of what they do to their currently running container. This is the big win here. Everything else simple serves this goal.

ENTER SOOS

Soos is the gateway into dealing with docker. It is a collection of bash scripts that manage which containers are running, when, and how. It builds new images, runs, stops, and starts existing containers. Launches single-use containers to run tests. Launches long-lived containers for the database and a master web app instance.

I consider Soos more of a template than a drop-in solution. Everyone’s needs will be different; perhaps you’re using Rails or Node instead of Django, or some variety of NoSQL. The publicly released Soos uses sqlite, whereas internally we’re using postres (in the near future, I will share our scripts for the db image).

Getting Started

  • bin/soos up

This is how you get started. soos-up will check to see if you have all the requirements installed and, if not, give you the opportunity to do so. It’ll boot a VirtualBox VM via Vagrant, install the docker provisioner, and pull the ubuntu:14.04 image that we use as our base.

Once this is done, you can visit http://10.1.2.4/ and see your django site. The django development server is in use here, meaning your changes to .py files will be reflected in real time. If you make changes to the db, you can run bin/soos migrate and it will syncdb. If you use south, or something similar, you can edit this to run that as well.

You will see that there are docker scripts in bin/docker. The main soos-* scripts are to be run by the user, via bin/soos [command], but the scripts in bin/docker are explicitly for running inside of a container. Generally, these should only be run by the zoos command.

Dockerfiles

We have a handful of docker files in our repository. The root Dockerfile is intended for AWS Beanstalk. Soos instead barters with the docker files found within support/dockerfiles. bin/soos build defaults to building at support/dockerfiles/app/ (and copies in an requirements.txt file, if found). While there’s no docker files for these, you can see that it could support, for instance, building a db server with bin/soos build --db.

Fork Me!

Please feel free to fork me and send PRs for any ideas you have in how this would help you do your job.