Test Sizes (2010)

What do you call a test that tests your application through its UI? An end-to-end test? A functional test? A system test? A selenium test? I've heard all them, and more. Going into the pros and cons of each type of test is a whole other blog entry, but it should be obvious that each type of test fulfills a specific role. A Small test equates neatly to a unit test, a Large test to an end-to-end or system test and a Medium test to tests that ensure that two tiers in an application can communicate properly. The major advantage that these test definitions have is that it's possible to get the tests to police these limits. In Java it's easy to install a security manager for use with a test suite that is configured for a particular test size and disallows certain activities. Because we use a simple Java annotation to indicate the size of the test, it's a breeze to collect all the tests of a particular size into a test suite. These include a requirement that tests can be run in any order which in turn means that tests need high isolation -- you can't rely on some other test leaving data behind.