充分挖掘自动化测试的潜力

本文主要是介绍充分挖掘自动化测试的潜力，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

LogiGear 的白皮书《 Achieving the Full Potential of Test Automation 》中总结了一些自动化测试的经验：

 

Software test automation has the capability to decrease the overall cost of testing and improve software quality, but most testing organizations have not been able to achieve the full potential of test automation. Many groups that implement test automation programs run into a number of common pitfalls. These problems can lead to test automation plans being completely scrapped, with the tools purchased for test automation becoming expensive “shelfware”. Often teams continue their automation effort, burdened with huge costs in maintaining large suites of automated test scripts that are of questionable value.

 

The Benefits of Software Test Automation 自动化测试的好处

Most software development and testing organizations are well aware of the benefits of test automation. A quick glance at the Web sites of any test automation tool vendor will point out a number of the key benefits of test automation. Some of these benefits include:

 

Reduced test execution time and cost （降低测试执行的时间和成本） : Automated tests take less time to execute than manual tests, and can generally execute unattended. A tester must simply start the test, and then analyze the results when the test is completed.

 

Increased test coverage on each testing cycle （提高每个测试周期的测试覆盖率） : Automated tests can allow testing teams to execute large volumes of tests against each build of their application, achieving a level of coverage that would not be possible with manual testing . This increased coverage can help teams uncover bugs in existing functionality much more quickly than through manual testing. Test automation can allow teams to test more features in each cycle (breadth), and also to test features using more permutations of inputs (depth).

 

Increased value of manual testing effort （提高手工测试的价值） : So long as applications are meant for human end users, test automation will never entirely replace the need for human testers. No matter how sophisticated test automation tools become, they will never be as good as human testers at finding bugs in an application. Human testers will instantly notice subtle bugs that are almost never detected by test automation, particularly usability bugs. Automated test tools cannot ‘follow their instincts’ to uncover bugs using exploratory and ad hoc testing techniques. By freeing manual testers from having to execute repetitive, mundane tests, test automation enables them to focus on using their creativity, knowledge, and instincts to discover important bugs.

 

 

 

Pitfalls: Why Test Automation Projects Fail to Achieve Their Potential 为什么自动化测试项目会失败？

 

Despite the clear benefits of test automation, many organizations are not able to build effective test automation programs. Test automation becomes a costly effort that finds few bugs and is of questionable value to the organization.

There are a number of reasons why test automation efforts are unproductive. Some of the most common include:

 

Poor quality of tests being automated 自动化的是质量不高的测试用例

Mark Fewster explains this problem very well:

“It doesn’t matter how clever you are at automating a test or how well you do it, if the test itself achieves nothing then all you end up with is a test that achieves nothing faster. ” [Fewster, Software Test Automation, I.1, (Addison Wesley, 1999)]

Many organizations simply focus on taking existing test cases and converting them into automated tests. There is a sense that if 100% of the manual test cases can be automated, then the test automation effort will be a success.

In trying to achieve this goal, many organizations find that they may have automated many of their manual tests, but it has come at a huge investment of time and money, and produces few bugs found. This can be due the fact that a poor test is a poor test, whether it is executed manually or automatically.

 

Lack of good test automation framework and process 缺乏优秀的自动化测试框架和流程

Many teams acquire a test automation tool and begin automating as many test cases as possible, with little consideration of how they can structure their automation in such a way that it is scalable and maintainable. Little consideration is given to managing the test scripts and test results, creating reusable functions, separating data from tests, and other key issues which allow a test automation effort to grow successfully. After some time, the team realizes that they have hundreds or thousands of test scripts, thousands of separate test result files, and the combined work of maintaining the existing scripts while continuing to automate new ones requires a larger and larger test automation team with higher and higher costs and no additional benefit.

 

Inability to adapt to changes in the system under test 被测试的软件发生变化时自动化测试不能很好地适应（维护性问题！）

As teams drive towards their goal of automating as many existing test cases as possible, they often don’t consider what will happen to the automated tests when the application under test (AUT) under goes a significant change.

Lacking a well conceived test automation framework that considers how to handle changes to the system under test, these teams often find that the majority of their test scripts need maintenance. The outdated scripts will usually result in skyrocketing numbers of false negatives, since the scripts are no longer finding the behavior they are programmed to expect.

As the team hurriedly works to update the test scripts to account for the changes, project stakeholders begin to lose faith in the results of the test automation. Often times the lack of perceived value in the test automation will result in a decision to scrap the existing test automation effort and start from scratch, using a more intelligent approach that will produce incrementally better results.

 

 

 

Generations: Test Automation Evolution 测试自动化的“进化”

 

Software test automation has evolved through several generations of tools and techniques:

Capture/playback tools （录制回放工具） record the actions of a tester in a manual test, and allow tests to be run unattended for many hours each day, greatly increasing test productivity and eliminating the mind-numbing repetition of manual testing. However, even small changes to the software under test require that the test be recorded manually again. Therefore this first generation of tools is not efficient or scalable.

 

Scripting （脚本编程） , a form of programming in computer languages specifically developed for software test automation, alleviates many issues with capture/ playback tools. However, the developers of these scripts must be highly technical and specialized programmers who work in isolation from the testers actually performing the tests. In addition, scripts are best suited for GUI testing and don’t lend themselves to embedded, batch, or other forms of systems. Finally, as changes to the software under test require complex changes to the associated automation scripts, maintenance of ever-larger libraries of automation scripts becomes an overwhelming challenge.

 

Data-driven testing （数据驱动测试） is often considered separately as an important development in test automation. This approach simply but powerfully separates the automation script from the data to be input and expected back from the software under test. This allows the data to be prepared by testers without relying on automation engineers, and vastly increases the possible variations and amounts of data that can be used in software testing. This breaking down of the problem into two pieces is very powerful. While this approach greatly extends the usefulness of scripted test automation, the huge maintenance chores required of the automation programming staff remain.

 

Keyword-based test automation （基于关键字的测试自动化） breaks work down even farther, in an advanced, structured and elegant approach. This reduces the cost and time of test design, automation, and execution by allowing all members of a testing team to focus on what they do best. Using this method, non-technical testers and business analysts can develop executable test automation using “keywords” that represent actions recognizable to end-users, such as “login”, while automation engineers devote their energy to coding the low-level steps that make up those actions, such as “click”, “find text box A in window B”, “enter UserName”, etc. Keyword-based test design can actually begin based on documents developed by business analysts or the marketing department, before the final details of the AUT are known. As the test automation process proceeds, bottlenecks are removed and the expensive time of highly trained professionals is used effectively.

 

The cost-benefits of the keyword method become even more apparent as the testing process continues. When the software under test undergoes changes, revisions to the test and to the automation scripts are necessary. Organizing test design and test automation with the keyword framework eliminates time previously allocated to maintaining large libraries of scripts and rewriting entire scripts anew after major changes to the software under test. With the keyword method, the necessary changes are far fewer. Many changes do not require new automation at all, and can be completed by non-technical testers or business analysts. When required, changes to automated keywords can be completed by automation engineers without affecting the rest of the test.

 

 

 

这篇关于充分挖掘自动化测试的潜力的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！

---