Testing is to Software as Experiment is to Science

The principle of science, the definition, almost, is the following: The test of all knowledge is experiment.

The sole test of the validity of any idea is experiment.

These quote came from Chapter 1 and Chapter 2 of the Feynman Lectures on Physics.

I’m not a physicist, but the software developer in me resonated with the connection between experimentation and validation of what we know about the world around us. Almost immediately, I related it to the world of software.

Validity is confirmed by experiments (tests)

The universe around us is an immense mystery. We have many ideas about it, but the ideas that are validated by experiment are the ones we cling to more tightly than the ones that remain in the realm of guess and theory.

How do we know that things are valid within a software application? The answer, of course, is by testing that software… experimenting with it, if you will, and observing the outcomes of those experiments.

Your software is a universe of mystery until it’s tested.

Repeated experiments (tests) are most valuable

Experiments that are conducted multiple times are significantly more valuable than a single observation made from a single experiment. Repeated experiments increase the probability that what was observed was not an accident, but was truly the result of correct procedure validating a correct hypothesis. Performing the experiments as identically as possible to one other is the key to trusting the results.

Likewise, repeated testing of software is far more valuable than testing it once and walking away. Repeated tests done identically over time provide the reassurance we need to know for sure that our software behaves as we expect as it morphs to accommodate new ideas and new needs.

Sure, we could spin up the app in a simulator and click/tap through it to see if pressing [Button X] does what its label implies it will do. But I’ve heard it said that if we have to do it more than once, it should be automated.

Testing is one of those things that needs to be done often. Pretty much any time the app’s code changes (even just a little bit!), there is warrant to check and make sure the universe as we knew it still obeys all the laws we previously discovered.

Repeated tests are just as valuable to software as repeated experiment is to science. The more we can automate them, the greater the consistency in our repeated observations of how the software behaves.

Experimentation (testing) is a creative effort

Designing the right experiment to elicit the right conditions to make that one critical observation is a creative process. Sometimes, scientists have to go to great lengths to make their observations. But they all find it worth it when their experiment yields results!

I work with some amazing scientists on a day-to-day basis. Watching them work and hearing about some of the things they have to do in order to test their ideas is so fascinating to me. For example, one of our Principal Investigators is making observations about how microgravity affects a plant’s cell walls. To do that, he had to literally launch his plant’s seeds into space! They’re aboard the International Space Station. Growing plants! In space! I’m blown away by these guys!

While we needn’t launch our code into space just to experiment with it, sometimes we have to get creative in order to make the observations we need to make from our code. Automated testing isn’t easy. Running the app and clicking a button and watching what happens with our own eyes is easy, but it’s tedious and I’m lazy!

Creating the right environment (in code) for a test to verify what it needs can take effort, but in the end, when the test yields results time after time after time, I’ve always found that it’s worth it!

Wrapping up

My hope is that seeing automated unit testing from a slightly different perspective has sparked some thoughts and ideas in your mind. How can you begin experiment with your code with automated tests? How can you begin to make repeatable observations about your code so that you know, for sure, that it works, even after you go about making changes to it as time goes along?

I’ve devoted a bit time to writing about unit testing in Swift since the language was released. I hope this article adds something unique to the collection and that it gets you motivated to validate your software universe with tests!