There are some tools that are considered almost a universal good amongst developers. Revision control systems tend to fall into this category. Everyone ought to be using one.
Continuous integration servers are also often bundled in to this category. It’s the kind of tool people ask you during interviews whether you use and judge you accordingly.
However, with continuous integration build servers, as with any tools really, it is worth considering the affects it has on your team. What are the trade-offs we are making for the benefits it provides?
Why use a CI build server?
There are many reasons that a CI build server might help a team. It forces you to have an automated build since an unattended system will be running it.
It also provides a way to ensure the tests are run regularly for each project.
It makes sure that every code push gets built and ensures people are integrating “atomic” changesets. In short it helps keep your code in an always-deployable state.
Mixed blessings
There are other benefits of build servers that have more complex effects on the team.
Your build becomes asynchronous, which means developers are not blocked while waiting for a build, and the full test suite, including any integration and performance tests to run.
Only the build server(s) need to have a fast, deterministic, working full build process. This saves you from maintaining it across potentially diverse development environments, tracking down elusive environment-specific non-determinism in tests, maintaining all dependencies for integration tests on all development machines.
There is also often tool support for things like build-dependencies – what order do projects have to be built in.
The curse of Asynchronicity
Using a CI server to make a build asynchronous is often a response to frustration at slow build times.
By making your build asynchronous you remove much of the pain from the build being slow, and even non-deterministic.
It can be very tempting for the team to ignore a steadily increasing build time because there are other things that are causing them more pain. What does it matter if it now takes 15 minutes instead of 2 to go from a commit to a deployed system? You can work on other things while it is building.
However, our goal is not to minimise development time, it is to maximise the value we are delivering to an organisation, and minimise the time to deliver increments of that value to get rapid feedback.
We don’t just want fast build times to save development time. It is also necessary to enable us to keep our feedback loop from customers and end users short.
It’s not sufficient to push several times a day and know that it will end up in a production or production-like environment eventually and be happy that you have integrated your changes.
We ought to care about what are are deploying. If we push a new user-visible change we can then go and get some immediate feedback on it from our customer. If it’s not a user-visible change we should still be caring about its affect on things like performance of the environment it is deployed to. We want feedback from our releases – otherwise what’s the point of releasing regularly.
We want to be able to change things in response to this feedback. That means that we actually do still care about the time it takes to build and deploy our code. If our build is asynchronous it’s tempting to start something else while it is happening and forget about getting that feedback, or have to context-switch to act on feedback once the build is done.
If we had a synchronous build we would have constant incentive to keep that build fast as we would feel the pain as it grows in length (particularly when it exceeds the duration of a coffee break). It is also difficult to ignore non-determinism in the build because it increases the build time.
Synchronicity forces us to keep the feedback loop for changes short, if we choose to make the build asynchronous we need to build in another mechanism to make slow and failing builds painful to the team.
Maintaining the build
Another somewhat subtle effect of build servers is that you have fewer environments from which you need the full test suite to run. For integration tests this means fewer dependencies to manage and environment configuration to maintain.
It is easy for the CI server to become the only place that all the tests can be run from and the only machine capable of building a deployable artifact.
This means the CI server is now critical infrastructure as you can’t work without it. This can happen because you lose some of the pressure to automate the configuration of the build environment because you only have to do it in a single place. It’s also easy to embed configuration needed for a build manually in non-standard configuration via a CI server web interface.
One way to ensure this doesn’t happen might be to throw away your CI server after every build and spin up a new one. That would ensure it’s very painful for you if you haven’t fully automated the configuration of your build system.
Consider the trade-offs
You can get the benefits of a CI server through a team being disciplined at building and deploying every change. Similarly, you can also avoid the potential associated traps if you build in some other feedback mechanism to encourage you to keep builds fast. There are ways of working effectively with or without a CI server.
My point is that it is always worth thinking through the effect introducing new tooling will have on your team. Even something that seems to be all beneficial may have subtle effects that are not immediately obvious. Think about what your goals are, and whether the tools help achieve those goals or make your life easier at the cost of your goals.