Technology focussed test automation - pitfall

Ever been in a situation where your test automation project was assigned to someone who was most interested in technology and coding and wanted to get away from the "routine" of testing ? Nothing wrong in being technically inclined and getting bored occasionally with testing! (Read more here on dealing with boredom in testing) 

However, what normally happens is that an engineer or a set of engineers who seem to demonstrate the most propensity to pick up a new tool / technology and run with it while wanting to get away from the regular testing tasks, are handed over the reins of test automation. Oftentimes what is observed is that the output of such an automation effort tends to be less than desirable from a testing perspective. What do i mean ? How can we have poor automation when employing our "star" technical resources ? Note the point that i am making - the probability of ending up with poor automation is higher in such a scenario where the focus is mainly on technology or tools used in automating rather than trying to solve the testing problem well. 

Who would you assign to do test automation ? The answer to that question is a key determinant to test automation success or failure. Agreed, it is not the sole determinant. However, it does play a very significant role. A common situation that one may observe while embarking on test automation is an excessive focus on tools or technology used to automate tests. Now, how could this be a negative factor in automation ? Isn't it a good thing to be keenly focussed on the technology used in automating tests ? Yes and No. Yes, since it is important to identify the right set of tools and technologies to automate your tests. You would not want to embark on an automation exercise only to meet roadblocks as the tool proves incapable of meeting your specific requirements. Now that you have the necessary tools, will it pose a problem if you continue to be focussed on technology used to automate tests ? Focus on technology is not bad in itself unless that focus makes you lose sight of the bigger picture, which is the testing problem you are trying to solve using that technology.

It is easy to become fascinated with the workings of a tool or technology and dive deep while losing focus on the reason for using that tool. Not convinced such a thing could occur ? From my own experiences across several projects, i have come across instances where a person in testing who has an interest in coding, solving technical challenges and often is bored with the "routine nature" of testing is asked to automate tests. This individual usually tends to get excited about the chance to do something "challenging" and "technical" as opposed to "just" testing. 

What happens there on is that often this employee who is automating tests gets caught up in trying to show the world what a great technical resource he/she is and creates suites that may not be - maintainable in the event that this employee leaves the group, documented enough for someone else to understand the suite easily, effective from a testing point of view. True it might have several technical bells and whistles, look complex and shiny but at the end of the day what counts is what the suite can do. If the sole focus while automating has been the technology used, the end result may not justify investments in automation.

Also, the technical resource you might have handed the automation work to could be a wannabe developer who is keen on sharepening his/her coding skills before moving on. Unless you have a well designed, maintainable and useful test automation framework/suite, your investment in automation is not justified. Some folks tend to argue that they have not spent a penny in purchasing or licensing any automation tools and hence have not really invested as much in automation. They have used open source or free tools. However, tool costs are one part of the picture. True they represent a sizeable chunk of the investment costs. However, you must account for employee costs as well as opportunity costs of automation. What could you have done if you had not pursued this path of ineffective or throw-away test automation ? What if you had chosen to put the right resources and manage the automation effectively ? What are the costs in terms of time spent by employee(s) in automating ? What are the costs involved in having to learn how this automation suite works and then maintain it ? Will changes to the product being tested or any of its dependencies break the automation suite and if so what costs will need to be incurred to make the suite usable ? These any many such questions need to be considered when evaluating returns from your investment in test automation.

For automation to succeed, the people involved should be carefully chosen and managed. An ad-hoc or improvise-as-you-go method will not produce a robust framework. Detailed planning, design and development following sound and standard practices and principles is essential. Ultimately, the tool or technology you choose is an enabler and not the reason for automation.
Join my community of professional testers to receive free updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Email subscriptions are managed by Google's FeedBurner service.

Share & Bookmark this blog entry