F.I.R.S.T. Principles
Definition
F.I.R.S.T. stands for Fast, Independent, Repeatable, Self-Validating and Timely. These principles help to write well-crafted unit tests and will make testing easier. Testing can be hard when the production code is not designed to be testable. These principles will show practices which improve testability.
The purpose of these principles is to create clean code¹, and by extension clean tests². Clean tests help unit tests to keep the production code flexible, maintainable and reusable. Our software architecture depends on unit tests because it is them which make changes possible with confidence.
Unit test
For most people, unit tests are defined as tests in isolation from software’s individual components (structure, class or function). However, it can be hard to test in isolation because most of the time components are related to one another (for example, a class often contains another class as a member property). According to Michael Feathers³, unit tests can have many definition. They should be fast (< 100ms / test) and help to localize problems. In order to be fast, they should not talk to a database, communicate across a network, touch a file system, or run in a special configuration setup.
[1] Clean Code is code that is easy to understand, reuse, maintain, extend, test…
[2] A clean test is a test code that is fast and easy to read, change…
[3] Author of the book Working Effectively with Legacy Code.
Fast
This principle says that tests should be fast to run. Here is a story showing why tests should be executed quickly:
- If tests are slow, we will not run them frequently ;
- If they are not run often, we will miss bugs ;
- If they take too long to find, we will struggle to maintain our test code ;
- If it is difficult to maintain, we will take time to fix bugs ;
- If they take time to fix, we will discard them ;
- If they are discarded, we will not be sure our production code works as expected ;
- If our code does not work correctly, we will add bugs at some point ;
- If too many bugs are added, we will fear to make changes in our code ;
- If we fear to make changes, our production code is not reliable anymore.
To conclude, if we don’t keep tests fast, we will discard and lose them. In short, slow tests are the same as no test.
This principle is a first step toward clean tests. It implies that tests must be fast to run, but clean tests also involve tests that must be fast to read, write and maintain.
Let’s see how we can plug this principle to write fast tests.
How should we write Fast tests?
One assert per test
If possible, tests should be verified by only one assert. The aim is to have a minimal number of assert because it quickly and easily helps to know what we are testing. A test focusing on a single concept (Single Responsibility Principle) tends to have fewer number of assert.
Naming
Variables and functions should be short and descriptive in order to improve readability and communication between programmers. Concise naming guides us to write more specific test cases. If naming is not appropriate, the code becomes legacy code¹ to other programmers.
Domain-specific language
Clarity, simplicity and density of an expression make tests easier to read. The use of domain-specific language makes tests that are easier to write, read and by extension maintain.
func testGetDataAsHtml() {
crawler.addPage(root,
PathParser.parse("TestPageOne"),
"test page") request.setResource("TestPageOne")
request.addInput("type", "data")
let responder = SerializedPageResponder()
let response = responder.makeResponse(FitNesseContext(root),
request) as SimpleResponse
let xml = response.getContent() XCTAssertEqual("text/xml", response.getContentType())
assertSubString("test page", xml)
assertSubString("<Test", xml)
}
The code² above is not designed to be read because it contains details interfering with the expressiveness of the test. Creating functions that use APIs from our production code is a solution for a better readability. Here is a refactored form into a much understandable way:
func testGetDataAsXml() {
makePageWithContent("TestPageOne", "test page") submitRequest("TestPageOne", "type:data") assertResponseIsXML()
assertResponseContains("test page", "<Test")
}
Dependency injection
Dependency management is one of the most critical problem in software development.³
The disadvantages of high coupling⁴ are impacts on code writing pace, maintaining time, test speed… Moreover, a programmer can easily add bugs in this type of code. In this case, decoupling is a key because it can improve components’ testability.
When we change code, we should have tests in place. To put tests in place, we often have to change code.³
In order to make testing easier, we should not have test code coupled with production code. Dependency injection⁵ helps to get control over every source code dependency. Components can be injected to another components. So, they can be deployed separately without having a component depending on another one.
Dependency injection in production code speed up test execution because test doubles⁶ can be used. As a consequence, we avoid time consuming tasks such as database interactions, network requests, multithreading…
[1] Legacy code is a personal point of view. It has many definitions such as code without test (Working Effectively with Legacy Code, Michael C. Feathers), or code we need to change and struggle to understand. So, it can be old code or unfamiliar code. It depends on our understanding and our comfortability about changing code.
[2] This code, adapted to Swift language, is from Clean Code, Robert C. Martin, pp 125–127.
[3] Quote from Working Effectively with Legacy Code, Michael C. Feathers.
[4] Coupling is the degree of which a component is independent from others. The more connections a component have with another one, the higher the coupling is.
[5] Dependency injection is a pattern or collection of techniques that supply a dependency to a piece of code.
[6] A test double is an object replacing the one in production in order to isolate our code under test from its surroundings. It is also heplful to simulate special conditions.
Independent
Tests isolate failures means that tests should not depend on each other in order to prevent a single failing test to cause a succession of other failing tests. To do so, they should not be run in any specific order and should be run without affecting each others. So, a test should not set up any conditions for another test. If a test cannot be independent, it means it is not short and specific enough.
How should we write Independent tests?
BUILD-OPERATE-CHECK Pattern
The BUILD-OPERATE-CHECK pattern helps writing more independent tests. In a test function:
- the BUILD part is about building the input data so they are not shared between tests ;
- The OPERATE part is about calling the method we want to test. It clarifies what we are testing ;
- The CHECK part is about asserting the result of the operation.
func testGetDataAsXml() {
// BUILD
makePageWithContent("TestPageOne", "test page") // OPERATE
submitRequest("TestPageOne", "type:data")
// CHECK
assertResponseIsXML()
assertResponseContains("test page", "<Test")
}
Dependency Injection
From the SOLID principles¹, the Dependency Inversion Principle says that a component should not depend on a concrete implementation but on abstraction. Dependency injection is a way to do dependency inversion. Together, they lead to a modular system with independent components. As a consequence, it makes tests more isolated without shared state.
iOS / masOS / tvOS
In a XCTestCase class, we cannot share variables in the class scope. In their lifecycle, each test method is invoked in a separate instance of the test case class.
Tips: in Xcode schemes containing tests, we can execute tests in parallel on simulator and randomize execution order.
[1] SOLID Principles are OOP coding standards that intend to create understandable, maintainable and flexible software design.
Repeatable
A Repeatable test means deterministic outcome. So, tests should be repeatable in any conditions / environments such as production environnement, QA environnement, no network, or different day, place, time…
When dealing with local data sources, tests should not depend on an initial state or leave any residue behind that could prevent running them again.
How should we write Repeatable tests?
Data sources (local database, network requests…) are not reliable because they can change. The goal of repeatable tests is not to test external systems. To meet this goal, test doubles are the solution. Their own input are set up in the BUILD part of the BUILD-OPERATE-CHECK pattern. Test doubles are a way to control their outputs and are implemented through dependency injection. They also simulate special conditions such as missing data, timeout request…
Self-Validating
Tests should be self-validating, meaning that we should not go through a log file or compare data to know if they pass.
How should we write Self-Validating tests?
BUILD-OPERATE-CHECK Pattern
Following this pattern, the CHECK part contains our assertion(s), so we ensure our tests either succeed or fail.
iOS / masOS / tvOS
With the XCTest framework, assertion methods are self-validating: they either succeed or fail.
Timely
Since tests help us to design testable production code, we cannot wait for our code to be ready for production before writing them.
Testing should not be done at the end because otherwise it does not gives choice to programmers. Instead, refactoring the working production code makes tests conform with the FIRST principles.
How should we write tests before production code?
They should follow the Test Driven Development¹ (TDD) Cycle.
According to Kent Beck’s book titled: “Test-driven Development: By Example”, the roots of TDD consists of the following 2 rules:
1. Don’t write a line of new code unless you first have a failing automated test
2. Eliminate duplication
The Two rules imply an order to the task of programming.
According to Robert C. Martin‘s book “Clean Code”, the three laws of TDD consists of the following 3 laws:
First Law — You may not write production code until you have written a failing unit test.
Second Law — You may not write more of a unit test than is sufficient to fail, and not compiling is failing.
Third Law — You may not write more production code than is sufficient to pass the currently failing test.
This process is a 30 seconds long cycle. It leads us into a continuous feedback loop from each cycle, so we always know what we are doing and where we are going.
Those rules and laws define a TDD approach called red, green, refactor. Red symbolizes a failing test; Green a passing test; Refactor the restructuring process that intends to improve the design.
The goal with this process is not to reach 100% code coverage but to be confident enough about our test suite. By gaining confidence, code changes will not affect our production code negatively by adding new bugs.
[1] Test Driven Development is a process where writing tests guide software development.
Conclusion
The reward of applying the FIRST principles is to end up with a code that is clean, easy to maintain, loosely coupled, and with reusable APIs.
In summary, the purpose of testable code is not only testability, but it is also the easiness of dealing with code understandability, reusability, maintainability and extendability.