How do we test a system that seems untestable?
Untested legacy code is the bane of many a developer's existence. In a
permissive language like Ruby, continued development in an untested legacy
codebase requires adding tests. Users expect certain behavior to survive each release intact.
For some systems, the traditional pyramid of test types (unit, API, and
integration) is insufficient. Unit and API tests must presumably cover hundreds
or thousands of functions; which matter? Integration tests could cover a wide
range of user interactions; which are likely? Familiarity with the project is a luxury we don't all enjoy.
I recently worked on an application that took a complex input (a
production database), ran it through a large SQL function, outputting a
giant CSV file representing thousands of permutations of data. Each stage of
the transformation was complex enough that visual inspection would fail to
determine if something was amiss.
the traditional test types could handle this challenge. Unit tests came
close, but the isolation of the test database could not rival the complexity of
a large, messy production database. Unfortunately, not testing was not an
option. Like every application, development needed to keep moving forward.
So, what did we do?
After much brainstorming, we found a solution: an obscure technique called a
Gold Master Test. I wrote about the
experience not long
Continued discussion led to a talk that I presented at RailsConf
2017 in Phoenix, AZ.
My presentation included finding a clear definition of a Gold Master Test,
writing a test, and working with an existing test.
Test writers of every experience level should leave this talk with a better
understanding of this technique, and a broader conception of what a test can
be. Thank you to Hashrocket for continuing to sponsor my public speaking endeavors.
Photo Credit: Christoph Gockel