How we improved our development process in 2015

General news, Opinion

How we improved our development process in 2015

The development team at Onespacemedia saw a lot of changes in 2015.   We grew to 4 members, and we launched the largest number of websites of the highest level of quality and innovation that we’ve ever had.  To support our growth and evolution we had to improve every aspect of our day-to-day workflow.

The first step was to update our project template, something we use to springboard every new project. Our project template comes with a well-thought-out file structure and is configured with a whole host of settings which allow us to extend the functionality of any given project, as well as configure different environments in different ways.  Some of the services we added support for by default included Sentry, Opbeat, Mailtrap, Adobe SDK and Google+ (for authentication).  In addition to adding support for APIs and services in the backend, we also added a lot of dynamic front-end variables such as analytics keys. We added conditionals to our base templates to check for the existence of various API keys and automatically output the relevant integration code on the front-end, which helps to reduce repetition and increase overall productivity.

Perhaps one of the biggest changes for us in 2015 was moving our hosting over to DigitalOcean.  Previously we made use of Webfaction, which allows you to get up-and-running fairly quickly, but did not have the flexibility we needed.  In addition, we were having to follow multi-page instructions to get sites configured properly, which was costing many unnecessary hours. This was compounded by excessive downtime and an unwieldy administration system, so we began the process of moving sites across to servers on DigitalOcean (known as droplets) which we configured and maintained.  We made use of Fabric and Ansible to automate the deployment process, and while this did not go as smoothly as we would have liked, we iterated quickly and soon had a solid collection of server management tools which would allow us to deploy, update and interact with servers with ease.  With the improvements to both the project template and the server management tools, we were able to go from having no project files at all, to having a fully hosted CMS within 10-15 minutes - and with the plans we have going forward into 2016, this time should decrease even further.

An advantage of moving onto servers that you control is being able to take advantage of modern libraries and packages without being restricted by what is available to you on shared hosting.  We took full advantage of our new-found freedom and brought our application environment into the 21st century, including making use of Python virtual environments to create reproducible installations and using a service stack of Gunicorn, supervisor and nginx to run our applications.  In addition to improving the back-end stack, we re-evaluated our front-end systems, lead by Dan Gamble in his new role as a dedicated front-end developer - look out for a blog post on this process in the near future.  We stripped away the monolithic Foundation framework and replaced it with a lightweight flexbox driven grid system, and evaluated various different class prefixing schemas before finally settling on one which made the most sense for us.

One of the bigger changes towards the end of the year was to change our Git repository hosting provider.  Our history with Git hosting has been a fairly unusual one; from hosting the Git repo on the application servers, to hosting our own centralised Git remote, then to Bitbucket private repositories - which is what we used for the majority of 2015.  Unfortunately towards the end of 2015 we started to have a fair amount of trouble with Bitbucket - particularly around uptime and reliability - so we evaluated our options for a move.  We already hosted all of our open-source work on Github, so we decided our best option was to also host our client code there. This move allowed us to have all of our code in one place - perhaps somewhat against the general decentralised model of Git, though we have at least 5 versions of the codebases amongst the development team.  One of the other Git-related changes we made was to make use of ‘Git flow’, which sees us maintaining a ‘develop’ branch for work-in-progress and feature branches for when we’re working on new features for a project.  We’ve found this to work extremely well for us and we’ll be continuing to use it for the foreseeable future.

The final change of the year was to begin to introduce a continuous integration flow for our projects.  For the longest time our projects have been fairly self-policed, but as we grow as a team, it’s important to have some ground rules and procedures in place regarding code quality and style, so we decided to make CircleCI part of our workflow.  Whenever a commit is pushed to a project, CircleCI automatically runs tests and checks both the back and front-end code against a set of style criteria.  We decided to use isort and flake8 for the Python code, and eslint and jslint (using standard-js as a baseline) for the front-end code.  So far we’ve found it works well, with the next logical step being that we integrate it into a pre-commit hook for projects.

In my next blog post, I talk about our plans for 2016 and how we are looking to improve our efficiency and workflow going forwards. Read it here.