Maximising Output: Thorough Testing on a Small Budget

Budget constraints play a major role in many projects. Reality often means that cuts must be made and when that happens test resources scale to match. With this in mind, it is important that testing, and by extension quality of the final product does not suffer when the screws start to tighten. The following insights, complete with cliche headings, will ensure you have quality (pardon the pun) processes to deliver products that meet industry and client expectations, resulting in better outcomes across the board. 

Horses for courses

A cricketer doesn’t arrive at the crease with a baseball bat, and your test approach should aim to follow the same principle. There are many different types of testing—regression, feature, smoke and sanity, to name a few. Choosing which is necessary for a build will help reduce unnecessary effort and spend. Regression tests, as an example, are resource and time intensive, meaning they should be kept to a minimum. Smoke and sanity testing however ensures core features are working as expected while also touching on areas that have been changed, which is sufficient in many cases. Our approach with all clients, small budget or not, is to limit regression testing to once per cycle for major updates—e.g, large new feature implementation or React Native upgrades. This naturally flows (like a good test set should) into my next topic. 

Patience is a virtue

The test team is often pushed from many angles, whether it be looming deadlines for product release or rapidly depleting funds. During these stressful times there is often the expectation that test will engage with the product earlier in an effort to accelerate its progression. Be wary when approaching this scenario as it can result in the opposite outcome. Patience should be at the forefront of one’s mind when considering when to implement regression testing specifically. Rushing regression testing when a product or feature is not yet complete only means it will need to be tested again when it is complete, doubling the time required. The QA team should endeavour to push back in these situations, instead advocating for feature testing in the interim. Similarly, once regression testing has taken place the test team should communicate that it does not need to be repeated, as smoke and sanity/feature testing is sufficient. Being at the bottom of the waterfall means you are constantly under pressure, but stand firm. You’ll thank me later.

Measure twice, cut once

It’s better to go slower and be accurate than rush and have to do something twice. Whilst this applies to almost all scenarios, it could never be more evident than it is in the test environment. Testing takes time, more than some people would like on occasion, and it is not an exact science. Cutting corners will only lead to a negative impact on product quality, ultimately resulting in a loss of reputation and customer satisfaction, which is far worse when considering the bigger picture. Companies should seek to promote a culture where the test process is respected and not interfered with, allowing testers to work free of distraction and pressure.

Process, process, process

I once watched a documentary about the creation of Yamaha pianos; it was a revelation. The clarity and precision that underpinned every step of the process was only possible because the process itself was so well defined and monitored. This methodology is not exclusive to manufacturing, and companies should include it in every aspect of software development, especially within the test team. Process outlines the best way to achieve a goal, considering many factors for a rounded perspective. If nothing else it gives you something to fall back on and can be used as a guide in trying times. When testing on a tight budget the benefits of a well-defined process are fully realised. This can be credited to everything already having structure, and thus less time is wasted on meetings and amongst team members getting everyone on the same page. By making a small investment internally to establish processes for future use, a tight budget is much easier to work with and you are much less likely to run over, resulting in better outcomes for everyone. A house crumbles without solid foundations. 

The squeaky wheel gets the oil

Don’t be afraid to speak up. If you don’t it will inevitably come back to bite you. People will assume everything is fine if you say nothing. When the budget is tight make sure to let the product manager know so strategies can be put in place to ensure things go as smoothly as possible. Furthermore, communicating what’s going well and what isn’t ensures that there are no surprises and everything can be monitored closely. It can be daunting to speak up, particularly when things are in the red, but not speaking up will only lead to more complicated difficulties down the line. At the end of the day, test is the advocate for the quality of the product and not speaking up would only lead to a failure to advocate effectively. In essence don’t let yourself (and your team) be blown off the cliff. The fall will hurt significantly more than the effort required to stand strong.

Wandering and going off track is… Hold on where were we?

Alice may have enjoyed her time in Wonderland but going off track usually results in disaster. Tunnel vision and going down rabbit holes is easy and often leads to expenditure on unnecessary issues. At Adapptor we use bug triage meetings to help avoid this pain. Post test bug fixing will only occur after triage meetings to reduce the possibility of low priority bugs being worked on, eating away budget unnecessarily. At the triage meeting every bug is given a priority and a decision is made regarding whether it will be fixed immediately or put in the backlog. Taking the time to look at the bigger picture in this scenario means a clearer view of the overall product is considered. These meetings also help to clear up any concerns testers, developers or product managers might have, meaning when bug fixing begins it is more efficient. 

Final Thoughts — Swords at the ready!

Testing on a small budget presents many challenges. The above methodology outlines how a business or team can maximise output to ensure product quality is maintained. Utilising some (preferably all) of these ideas will help your team to work smarter not harder, allowing for the optimisation of available funds. I'm not saying by following the above you’re guaranteed to come in under budget, just that these are common observations that might help you or your business stay on track. Now get out there. Your swords have been sharpened and the battle is about to begin against a common foe - looking at you scope creep!