I have no doubt that the managers who place such requirements have really good intentions. They most likely care a lot about their projects. They do not want any broken windows. However, such edict often has negative consequences.
- It leads to a different form of TDD—Threat Driven Development. The programmers are threatened rather than being empowered. I don't want to fear making change to the code on your project. To the contrary, I want to make a lot of changes as I feel fit and rely on my tests and the builds to alert me when I err. The quick automated tests I run on my machine will act as my first line of defense. I expect the continuous build to be my second line of defense.
- They may tend to withhold check-ins. If I'm talked down when I break the build, I am going to be overly cautious and check in less frequently. This slows me down. I don't want to be careful, I want to be productive. I have not lowered the quality to gain speed. To the contrary, I have slowed down already to improve the quality. I don't need to be any slower due to fear of breaking the build.
- It may lead to duplication of effort. I will be forcing myself to run my tests on each supported configuration that the continuous build is checking my code against. This is a wasted and duplicated effort. Each developer has to spend time and effort setting up and then maintaining these different configurations on their own machines or desks.
I care a lot about my projects. I write fairly decent amount of test cases. I run my continuous builds, not on one platform, but on different platforms my application is expected to run. I use these tools to increase my productivity and agility.
Let's ask the Why question: Why do I care to use the Continuous Integration build? The number one reason is to avoid "But it worked on my machine" syndrome. I want to know that when my programmers think they've made progress, that is reflected by a healthy exercise of the application on different machines. It tells me that all the relevant code is checked in, that the code compiles (or in a state ready to execute), runs as currently expected not just on the developers machine, but also on different supported platforms and configurations. If my application has to run on Windows XP, Vista, a flavor of Unix, and needs to work with Oracle, SQL Server, etc., the build machines (physical or virtual) will exercise the application on these different configurations. The feedback I get from these builds will tell me that my application run and continues to run on different configurations that my application is required to support.
In spite of all my good intentions and efforts, there are times when I've broken by builds. Here are a few examples where I did really break my builds.
A. The code I wrote ran fine on one version of Windows, but the API I used does not work the same on another version of Windows. This difference was in a fine print that I did not notice. Unless I ran my test on all supported versions of windows before I checked in code, I would not know this.
In order to run all my tests on all supported versions of Windows, I either have to have multiple machines on my desk, have access to multiple machines I would run over to and test my code on, or have multiple virtual machines I should run on my own machine. There is a cost, effort, and time associated with this when each programmer has to do this before every checkin. Also each of us need to put the effort to maintain these configurations.
On the project with above failure, this failure happened once over the entire period of the project. We have two options on hand. Each time I changed my code, I can take time to run all my tests on different combination of OS, databases, libraries, devices, etc. that my application is supposed to run with. Or, I run tests on my machine, and gain a good bit of confidence that my code works. Then I'll let my CI build machines that are readily waiting to exercise my code to take care of the platform and configuration differences. If it fails due to these differences, then I can immediately know what went wrong and go take care of it. If all goes well (as it does most of the time) then I am not spending any time and effort worrying about these edge cases knowing the build machine is providing me the safety net.
B. The second time I broke the build is when there was a difference in representation of a data type between two version of the database. This case is similar to the above. Such failure was rare and was caught by the build machine within minutes and fixed within minutes after that.
C. The most recent build break occurred a few weeks ago. I added a new unit test, added a little code to satisfy that test. I ran my automated tests on my machine. With a green bar on my hand, I checked in the code. I was quite surprised when the CI build failed. It turned out that when my tests were run standalone, they worked fine. However, when invoked by the build process, it failed due to a class loading conflict. Took me a few minutes to resolve the issue so it runs fine both locally and on the build machines.
I'm sure others have their shared of "Why my build broke" stories to tell (and I sure would be delighted to hear).
So, if I have to work so I never break the build, that is a tall order. I simply can't meet that expectation. Yes, I do not want to be careless and break my builds. However, I should not be punished (or shamed) for breaking the build either. Reserve that when I break the build but do not care to fix it within a reasonable amount of time.
I invite my developers to go ahead and break the build. I then tell them that now they are responsible to fix it, real soon (within minutes). Let them learn from the experience. Soon you will see that the builds don't break so often, but there are frequent checkins at the same time Of course, motivating programmers to check in frequently is another story for another blog, but at least you've not helped hold them back.
Let's not focus on the mechanics. Let's keep the focus on the overall goal—to develop a successful product.
