The majority of projects in IT are over budget and miss their deadline.  We often notice that IT staff works hard just to stay in one place.  While there are many causes, one of the key factors to mitigate the problem is testing.  In this article, we will define show what happens if you:

  • let testing become an afterthought.
  • let testing to be reckoned with after the software exists.
  • fail to consider all kinds of testing: unit, system, performance, and acceptance testing.

When these happen, the benefit of testing will be very limited: thumbs up or down on software, the final mark of quality, or the lack thereof.  What are the opportunities missed with this view?

Requirements testing

Hardly anyone disputes that a lack of clear, well understood and documented requirements is a land mine likely to blow in the development cycle, often causing overruns in both timelines and budgets.  Yet a lot of companies miss the low hanging fruit in this phase: creating test cases, where each test case corresponds to each functional requirement.   (Think of specifying technical requirements and performance requirements in this phase to improve effectiveness.)  More often than not we find requirement “bugs” using this simple method. At this stage, we can easily clarify and fix them.  Cost of defect fixing is exponential – 1x in requirements, 10x in testing after coding and 100x in production.  This is often referred to as the 1:10:100 rule.*

If we find a bug when the software is live, it is likely tarnish the reputation of the software supplier or service company. We have seen when a bug can even sink it. The example of Sybase in the area of RDBMS demonstrates this best.  (Who remembers today that they used to be Oracle’s strong competitor?)  Yet how many times does a team rush to design? How often do we see people making assumptions and finding out these were incorrect in the latter stages of development?

Frequency of testing and test automation

It’s more a rule than an exception that monolithic projects with a massive delivery at the end – often nicknamed “the miracle happens here” – don’t work.  We believe that small deliveries have the advantage of realizing business advantage early and enabling adjustments.  Whether you practice true ‘agile’ methodology or phased development, testing is an integral part of this cycle.  When we optimize testing using automation tools, it saves both elapsed time and costs.  In small phases, this becomes a necessity, as developers can leave testers in the dust when doing sprints with manual testing, with all of the negative ramifications.  

When we developed automated testing, we realized that most of the tools require scripting and in turn need a qualified scriptwriter. As it takes time to become a scriptwriter, we noticed in many cases teams often end up using developers to write scripts. As a result, we found that scriptwriters end up creating bottlenecks. That would increase costs and create delays.  The solution we found was to use tools which allow test creation through GUIs so that anyone can create such tests.  

Summary

Finding bugs early is cheaper and saves time.  It is key to maximize ‘test coverage’ defined as the percentage of the code covered by test cases.  A simple matrix illustrates the challenge:

It is the major unknown bugs which present the biggest challenge – we don’t know what we don’t know.  How do we minimize this quadrant?  The higher the test coverage the lower the likelihood of unknown bugs.  Automating testing through tools which provide agility in the creation of test cases and updates in conjunction with doing test cases parallel with requirements can significantly change the probability of success of your project while decreasing its cost.  Why not do it?

At NewPush we provide Testing as a Service to help start testing. Our service offers minimum difficulty while reaping all the benefits.  There is no prior knowledge of testing needed. We create automated test cases which can be run at any time.  We also provide project management consultancy and we can create test cases upfront during the requirements phase.

*Ralph Young: Effective Requirements Practices: Defining the Real Customer needs.