Software Testing is an integral partof Software Development and Deployment. Yet, there have been theories of ValueEngineering determining the costs of Testing and “Not” Testing usingvarious mechanism and analyzing various patterns. This review paper aims to analyzethe need and loopholes that exist in the various testing approaches, be it Manualor Automated. INTRODUCTIONTesting Approaches andCosts MetricsTesting is mainly seen as bothTime and Money Consuming activity in the Software Engineering Process. Hence,testing as a process is mostly avoided till later stages or even completelyeliminated in some cases. Cost metrics are invariably dependent on the average numberof errors identified in the software.
Also, indicating presence of bugs in asoftware is completely different than indicating their absence and hence,testing does not guarantee a faultless software. However, not testing is notnecessarily a cheaper approach. Software metrics are adopted to check forfuture software performance and current code quality. These metrics specificallyrefer to the assessment of various testing methods.To adopt an immaculate testing approach,we need to identify a cost metric which is as simple and detailed as possible.The three testing strategies have therefore been compared to analyze which happensto the best – Automated Testing, Manual Testing or Not Testing. “Automated software testing uses automatedtools to run tests based on algorithms to compare the developing program’sexpected outcomes with the actual outcomes.
If the outcomes are aligned, yourprogram is running properly and you are most likely bug free. If the two don’t align,you have to take another look at your code and alter it and continue to runtests until the outcomes align.”1Cost automated=Cost implementation + Costmachine + CosterrorFP”Manual Software Testing is theprocess of going in and running each individual program or series of tasks andcomparing the results to the expectations, in order to find the defects in theprogram. Essentially, manual testing is using the program as the user wouldunder all possible scenarios, and making sure all of the features act appropriately.”1Cost manual= (Cost wS + Cost pay)?t test”Since there is no testing, there is also no need to develop atest. Thus, the implementation costs, as well as the costs for determining therequirements, are saved.
While not executing a test results in savings when thetest would find no bug or even produce a false alarm, there are still timeswhen a test would have detected an error. The overall costs of the escapederrors furthermore depend on the number of bugs which have not been found, dueto not testing. Thus this results in -CostnoTest = Costescaped ?#bugs “1. Automated testing is beneficial in the sense that it eliminates theneed and hence the cost of retesting. However, manual testing gives the added advantageof testing Human Interaction Interfaces and aesthetic comprehension of thesoftware. Not testing, on the other hand, can be costly or rightly avoidabledepending on the type of error which could be as harmless as an aesthetic erroror as costly as any perilous error.Behavior – DrivenDevelopment Tools Based on Test-driven development(TDD) and Acceptance Test-drivendevelopment(ATDD), Behavior-Driven Development (BDD) is an evolutionary step.”Taking into account behavior specifications allows software engineers tothink more clearly about each behavior, relying less on testing by a class orby a method, and having better executable documentation.
” 2 TDD – Almost a type of automated testing, it follows the order of writingtesting code before (to-be) tested code. “Each development cycle consistsof three steps: the creation of unit test, implementation, refactoring” 2.ATDD – Is an extension of TDD wherein the tests are designed to fulfill stakeholders’requirements. Its main purpose is to help developers design test cases whichsuit the initial system requirements.
Combining the two – “BDD is involvement of all stakeholders which is possible viaubiquitous language. Business analysts write down behavioral requirements inthe way that will also be understood by developers who later transform theserequirements into executable tests.”2Analysis of MutationTesting TechniquesMutation Testing is a technique wherein the test code is introducedwith a mutated code, which if recognized by the test code is either killed and if not, it becomes live. The mutation operators howeverneed to be investigated as the efficiency of resulting mutations determine theperformance of the test technique. Cloud Computing and its evolution has playeda major role in increased performance by parallelizing work and therebyreducing time consumption.Detection of TestSmells “It is important for the test cases to adhere to the unit testcriteria. Some of the desired test design criteria as mentioned by Van Rompaey andDu Bois in their paper are, (1) Tests should be consistent in their overall behavior(2) Every test is needed as they verify a part of the system (3) They should be maintained and should satisfy most of theobject-oriented design principles (4) Tests should have repeatable outcomes (5)They should also have the expected outcome defined to check and compare theresults (6) They should be isolated and should conform to a clear and rigorousstructure (7) They should also be independent of external factors (8) Theyshould be automated, persistent and run fast to reduce the turnaround time.
Tests that fail to comply with any of the above criteria show symptoms ofpoorly designed test cases which are termed as test smells.There are basically three test smells that are specific for testcode(s), which are, (1) General Fixture (2) Eager test and (3) Obscure Test.”4General Test – When the test context is too broad letting different partsof the fixture being accessed by different parts of the code. Eager Test – This test context tests the setup multiple times, makingit tough to interpret and even tougher to document.Obscure Test – The test contexts which are difficult to understand makingit incompatible for documentation.Test Smells have various causeswhich can be detected by CharacterizingMetrics which are Metrics and Indicators. DISCUSSIONS – Having reviewed and surveyed papers available online, there have beenmany views leading to a lot of conflicts.
Having read about Testing Techniques and Cost metrics, any observationscan’t be confidently expressed as to which Testing is better. It all depends onthe probability of the test failing or succeeding. Therefore, it is a toughdecision to choose between Automated Testing, Manual Testing and Not Testing andis even tougher to generalize as it all depends on the application.
However,Not Testing should not be a resort but the type of tests can vary depending theprobability of the type of error. On the other hand, Testing too much could meanthat the developer puts lesser effort into codes knowing that testing wouldidentify the bug fixes, which means double costs – Over-paid developer andRepetitive Testing Costs.BDD is a novel approach to the traditional testing techniques whereinBehavioral Analysis of Appliances can be made to guarantee user satisfaction andhence the stakeholders’ requirements. BDD Technique is granular in the sense thatit can be applied at various levels such as Code/Unit level and Acceptance/Integration level as well.
Different dimensions to BDD are Tools with readable I/O, Ubiquitous Language, Automated ATDD, Templates,Unit Testing facilities, etc. Hence, it is observed the BDD combines TDD andATDD using a Ubiquitous language. A recent study defines three dimensions forcomparing BDD frameworks: based on a target group, on the support of characteristicsof BDD and based on specific features of selected tools.2To take a more evolved approach, Mutation testing seems to be moreuseful in real world applications.
To determine a more efficient mutation operatorset in the sense of having a greater inverse mutation score, the inverse mutationscores were analyzed per mutation operator. The biggest increase of the inversemutation score could be achieved if testing would just include the conditionalsboundary mutator. Not all testing techniques are faultless or guarantee a faultless codeoutput. After surveying a set of compiled metrics and indicators that measurecertain properties of the test code, the relative significance of the testsmells in the test code according to the violation of unit test criteria were assessed.The indicators specified for General Fixture and Obscure Test measure the smelland check it with a pre-defined threshold value. Finally, we have developed aconceptual model which depicts the various detection techniques of GeneralFixtures, Eager Tests and Obscure Tests and also the dependencies between them.CONCLUSIONS ANDFUTURE SCOPEIn short, Testing techniques and their adoption can cause what is referred as Technical Debt andtherefore have short-term benefits but could be detrimental in future.Extensive survey of testing techniques has provided a variety ofsolutions which are as follows – Test Data Generators – Like code generators accelerate the softwaredevelopment process, test data generators could be used to accelerate thetesting process.
Research in generating a wider range of input that can mimic ahuman could have a great impact on automated testing tools.Vulnerabilitypresented by the testing technique is for all intents and purposes unavoidable.Other, frequently more troublesome, illustrations incorporate streamlining agentsbeing too aggressive when test code in a framework generally influences itssize, speed, or behavior, and when framework or part test systems create inaccurateoutcomes. An issue that happens amid testing isn’t repeatable when running anon-test session, and an element that worked fine while testing but fails in reallife.Under test, theframework acts uniquely in contrast to in its discharge rendition.
Thiscircumstance normally happens when the test condition influences the conduct ofthe guinea pig. Cases of variables presenting this conduct are restrictivelyordered testing and troubleshooting code, and also exceptional interfaces fortesting. Limit thedistinctions inside the framework under test and the product in its last shape.By and large, you can annihilate the issue, albeit discovery testing throughthe interfaces that the product under test typically gives frequently limitsthe need to particular testing or troubleshooting forms of the product. Anotherway to deal with taking care of this issue is to convey the framework with thetroubleshoot code still in it.The complexity ofcurrent or even futuristic software and communication systems, have renderedthe current testing methods non-viable and hence, is expected to become morecomplicated in the near future. Manual testing is becoming obsolete and integrationwith the overall design processes and tools will prove necessary to keep pacein testing these complex current and future systems.