How Google TestsSoftware - Part Three
Lots of questions in thecomments to the last two posts. I am not ignoring them. Hopefully many of themwill be answered here and in following posts. I am just getting started on thistopic.
At Google, quality is notequal to test. Yes I am sure that is true elsewhere too. “Quality cannot betested in” is so cliché it has to be true. From automobiles to software if itisn’t built right in the first place then it is never going to be right. Askany car company that has ever had to do a mass recall how expensive it is to bolton quality after-the-fact.
However, this is neither assimple nor as accurate as it sounds. While it is true that quality cannot betested in, it is equally evident that without testing it is impossible todevelop anything of quality. How does one decide if what you built is highquality without testing it?
The simple solution to thisconundrum is to stop treating development and test as separate disciplines.Testing and development go hand in hand. Code a little and test what you built.Then code some more and test some more. Better yet, plan the tests while youcode or even before. Test isn’t a separate practice, it’s part and parcel ofthe development process itself. Quality is not equal to test; it is achieved byputting development and testing into a blender and mixing them until one isindistinguishable from the other.
At Google this is exactlyour goal: to merge development and testing so that you cannot do one withoutthe other. Build a little and then test it. Build some more and test some more.The key here is who is doing the testing. Since the number of actual dedicatedtesters at Google is so disproportionately low, the only possible answer has tobe the developer. Who better to do all that testing than the people doing theactual coding? Who better to find the bug than the person who wrote it? Who ismore incentivized to avoid writing the bug in the first place? The reasonGoogle can get by with so few dedicated testers is because developers ownquality. In fact, teams that insist on having a large testing presence aregenerally assumed to be doing something wrong. Having too large a test team isa very strong sign that the code/test mix is out of balance. Adding moretesters is not going to solve anything.
This means that quality ismore an act of prevention than it is detection. Quality is a development issue,not a testing issue. To the extent that we are able to embed testing practiceinside development, we have created a process that is hyper incremental wheremistakes can be rolled back if any one increment turns out to be too buggy.We’ve not only prevented a lot of customer issues, we have greatly reduced thenumber of testers necessary to ensure the absence of recall-class bugs. AtGoogle, testing is aimed at determining how well this prevention method isworking. TEs are constantly on the lookout for evidence that the SWE-SETcombination of bug writers/preventers are screwed toward the latter and TEsraise alarms when that process seems out of whack.
Manifestations of thisblending of development and testing are all over the place from code reviewnotes asking ‘where are your tests?’ to posters in the bathrooms remindingdevelopers about best testing practices, our infamous Testing On The Toiletguides. Testing must be an unavoidable aspect of development and the marriageof development and testing is where quality is achieved. SWEs are testers, SETsare testers and TEs are testers.
If your organization is alsodoing this blending, please share your successes and challenges with the restof us. If not, then here is a change you can help your organization make: getdevelopers fully vested in the quality equation. You know the old saying thatchickens are happy to contribute to a bacon and egg breakfast but the pig isfully committed? Well, it's true...go oink at one of your developer and see ifthey oink back. If they start clucking, you have a problem.
