Some testing best practices
Last updated
Last updated
Software testing will save an organization time and money by reducing software development and maintenance costs. Software testing builds stability guarantees into the development of new features. Testing ensures that a feature is working as expected and users are not encountering bugs.
Development time on new features is reduced by specifying a set of test cases that the new feature must match to be considered complete and deliverable. This gives developers a fixed target to work towards enabling more accurate timeline estimates and lowering the introduction of new bugs. Once these test cases are in place the overall maintenance costs are lowered. The tests can be run against an already delivered feature to ensure that it still behaves as expected.
There are several fundamental levels within software testing, each examining the software functionality from a unique vantage point within the development process. Let’s take a look at each type of testing in turn and examine its practical use.
The foundational level of software testing is unit testing. Unit testing is the practice of instrumenting input and output correctness checks for individual units of code. The measurement unit, in this case, is standalone code functions or methods.
During unit testing, production code functions are executed in a test environment with simulated input. The output of the function is then compared against expected output for that input. If the output matches the expected the test passes. If not it is a failure. Unit tests are a great way to validate derived data functions.
A hypothetical unit test user story example would be something like: “function 2VAL, Given 2 values x and y always returns x+y”. The unit test would then execute 2VAL with two values and confirm that the output was x+y. Unit tests are great for confirming the correctness of code that operates on monetary values.
When a software test case covers more than one unit, it is considered an integration test. When developing a software test case, the lines between unit tests can quickly evolve into integration tests. Often times a unit test may be developed that operates against a third party code dependency. The dependency itself will not need to be tested and the integration to it will be mocked or faked.
Test cases that simulate a full user-level experience are called functional tests or end-to-end tests. End-to-end tests use tools that simulate real human user behavior. Common steps in an end-to-end test:
Click this button
Read this text
Submit this form
Because of the full experience execution context, end-to-end tests verify correctness across all the layers of a software stack.
Exploratory testing is a testing exercise in which testers are assigned a loosely defined task to achieve using the software being tested. This means you can learn a lot about the way people use your product in the wild. Exploratory test sessions can even motivate their users by offering rewards for the most number of issues, best defect, or doing something unexpected with the product.
One of the benefits of exploratory software testing is that anyone can join in to help test because all they need to do is wander about the product in a free form manner. Exploratory testing is not random, yet they aren't scripted like manual tests, either.
When devising a software testing strategy its best to keep the overall product, user, and business strategies in mind. Considerations will need to be made on what the most high value test coverage targets are.
In an ideal world, a software project would strive for 100% test coverage guaranteeing the code is bug-free and works as expected. Unfortunately in the real business world, with timelines and budget constraints, this is not so realistic.
Different testing strategies should be considered depending on the type of deliverable software as well. If the software is a GUI driven application, high level end-to-end tests will be highly valuable. Headless UI free software projects will forgo end-to-end testing and value highly from unit tests.
A general overall strategy for GUI-based user applications is as follows.
Instrument end-to-end tests on all the core user flows, login, signup, checkout, etc
Instrument unit tests on all data sensitive code functions like monetary transaction tools
Instrument integration tests for any points of 3rd party integration to ensure data is flowing to the 3rd and any errors are being propagated correctly
