Stop Mocking, Start Testing

Project Hosting at Google Code is a large, well-established system written mostly in Python. We'll share our battle-born convictions about creating tests for test-unfriendly code and the larger topic of testing.

When launched, Project Hosting’s testing consisted of the stock Subversion test suite and a handful of ad hoc smoke test scripts that required starting the entire system and manually inspecting the test’s output.

Over six years of codebase evolution, tests have been added with varying degrees of coverage and maintainability. Early system design decisions made adding tests difficult: the first tests added to the system used mock objects unwisely and large numbers of mock objects made refactoring costly in time and effort.

Frustration with the difficulty of enhancing the service led us to reevaluate our testing practice and led to the discovery of better ways to test applications of this complexity. We will share our experiences with testing and discuss designing for maintainability and testability and appropriate use of testing tools such as frameworks and mocks.