Master the Essentials of UI Test Automation: Chapter Two

Real Life Guidelines that Deliver Results

You’re reading the second post in a series that’s intended to get you and your teams started on the path to success with your UI test automation projects:

1. Introduction
2. Before You Start (you are here)
3. People
4. Resources
5. Look before You Jump
6. Automation in the Real World
7. Improving Your Success

Important note: After its completion, this series will be gathered up, updated/polished and published as an eBook. We’ll also have a follow-on webinar to continue the discussion. Interested?

• Register to receive the eBook
• Watch the follow on webinar on-demand.

Chapter Two: Before You Start

"Plans are useless, but planning is indispensable." This quote from American President Dwight Eisenhower is one of my favorite quotes. Sure, he's talking about the run up to D-Day in World War II, but it's applicable to so many things in life—especially software development.

You need to ensure you're spending your time and effort wisely before you jump in and start hacking away at automated tests. Here are a few things I've found helpful to work through as part of the planning process.

Why Automate?

Why are you considering bringing in test automation to your delivery process? Take some time to get very specific about the problem you're trying to solve. More importantly, make sure you're tackling a concrete business problem. UI automation's a nifty tool, but it's an expensive one to get adept with, and it's a very costly tool to deal with if you use it badly. It's also only one form of test automation, and by itself it's not going to solve much of any delivery "challenges" you're having.

Here are a few business-related areas in which UI automation may be helpful:

• Long release cycles due to time required for manual regression testing
• Testing falling behind development during release cycles/iterations/sprints
• Rework costs due to regressions of high-value features
• High support costs due to escaped bugs around high-value features
• High cost of testing high-value features against multiple browsers and operating systems
• Exploding cost of testing against multiple mobile platforms and browsers

Worst of all, here are two areas that, for me, trump all other concerns:

• Loss of executive-level trust in the team's ability to deliver good software
• Loss of trust and business from your end users or customers

The last two, especially the last one, should concern every team member. It's really bad if you've lost higher-level management's trust. It's potentially disastrous if you're losing customers.

There are some areas for which you should not consider UI automation as a solution:

• Validating cross-browser look and feel
• Guarding against layout and style regressions
• Saving money by cutting number of manual testers

UI test automation may help you solve technical or process problems, but ensure you're first asking why and focusing on solving business problems first.

Do You Have the Environment to be Successful?

Once you've decided that UI test automation is a good choice for you, the next critical factor is whether your team is able to be successful at UI automation. Several different aspects come into play:

Do You Have Stakeholder and Sponsor Buy-In?

Adopting UI automation will require significant changes to how you work. You'll need team members with the right skills, you'll need resources (no, “resources” are not people), and you'll need additional time.

Adding UI automation into your delivery process will decrease your velocity. You have to make sure your stakeholders and sponsors truly understand that. They have to support the decrease in velocity for the improvement in delivered business value.

"Decrease in velocity" is always something that concerns the business side of the organization. I've found it very helpful to tie back to the specific business-level issues discussed earlier in this post. Say this:

"Yes, we're going to slow down a bit, but the goal is to cut the support costs we're incurring through escaped bugs. We're also hoping to gain back revenue we've lost from the decline in license and service renewals."

This links your efforts back to the things the stakeholders really care about: the organization's mission and bottom line.

Do You Have the Right Communication?

Good communication is the foundation of every successful human effort, software notwithstanding.

Are your teams able to get good information about features in a timely fashion? Do designers, developers and testers all talk regularly about how UX/UI work is accomplished? Can your testers get early input on UI design to help make testable screens? Do your testers understand what the stakeholders' highest priorities are for each feature, and do they understand the business value behind those features?

If the answers to any of those questions are "No," you will need to address those issues as you move forward with your automation—or suffer the friction and stress that falls out.

Great communication helps ensure everyone involved in UI automation knows the priorities before they start their work. It helps everyone understand how to balance automation work with focused manual testing, and it helps focus automation work on the right parts of the system.

Do You Have the Right Team Structure?

UI automation rarely succeeds when testers are expected to create all automation in a silo or walled-off room. I already mentioned the importance of early, frequent communication between developers, stakeholders, testers—basically the entire team.

If your teams are fragmented into highly constrictive silos, you will likely see additional friction and difficulties when trying to clear up basic fundamentals such as getting good locators on elements around which you’re building automation scripts.

The best structure for any team is an open environment that empowers frequent communication directly between concerned team members. Co-located teams always function the best; however, geographic dispersion can't be an excuse for poor communication.

Break down communication bottlenecks in your teams. Guide project managers or others to get out of the mindset of requiring communication to flow through them—wheel/spoke communication routed through one person is guaranteed to cause problems as teams try to get more efficient at their work.

Encourage your testers to reach out directly to developers when possible. The developers can clear up issues around tricky asynchronous operations, or other behind-the-scene functionality that's not apparent to someone looking only at the UI.

Setting up a team's structure for success means as few roadblocks and walls to frequent, candid discussions.

What Tools Can You Use?

Notice I've left the actual tooling for last. Yes, yes: the toolset you use is critical, but it doesn't matter what tools you select for automation if you haven't answered the harder questions first.

You can finally jump into tool selection once you've addressed that you've got a clear case for why you're going to use automation, and what problems you're trying to solve.

There are a huge range of UI automation test tools available. Some are open-source, some are free and some are commercial. There are also many types of tools, from drivers to frameworks to entire suites.^[1]

Selecting the right toolset for your team means answering a few more questions:

• What communication mechanisms will you use for tests? Do you need a grammar-based specification? Will recorded tests with coded steps be clear enough? Are 100-percent coded tests clear enough for everyone?
• Who writes the tests? Will testers be solely responsible for test creation? Or, will developers have a hand in it, as well?
• Who maintains tests? Will the team that writes the tests maintain them, or do you have an outside contractor writing tests and handing them off to an internal team? If you have two (or more) different teams, make sure the teams have the skills, aptitude and time to take on the selected tool.
• Who uses the tests? How will you run your tests? Will the suites be triggered manually? Will they be part of a scheduled suite to be run via a Jenkins CI server or Team Foundation Server build? You'll need people who can handle the care and feeding of those environments, and you'll need the infrastructure required, too.
• Who uses test results? Who in your organization needs what level of information about your tests? Keep in mind that your stakeholders often need one set of data, while your team needs another.
• Do we have the skills? Lastly, you'll need team members with the right skillset to build, manage and maintain all the pieces necessary for a successful automation effort. Your team doesn't need those skills right now, but they'll need support to develop those skills in a timely fashion.

Telerik put out a “Buyer’s Guide” in 2014 that answers these and other questions.

People and Resources Come Next

All of these high-level planning concepts will be central to your team getting automation started as quickly and effectively as possible. You shouldn't spend months answering these questions—analysis paralysis can cause entire organizations to lose a lot of time wringing hands over silly details.

Spend enough time to get a sense of comfort that you've at least talked about the major points above. After that, it's time to move on and discuss the people and resources you'll need for a successful effort.

What do you think of the planning topics we've laid out here? Are there other topics you've covered in your own automation efforts? Let us know in the comments.

Next Up: People

^[1] An automation driver is responsible for driving the UI application around. Think of Selenium WebDriver or the Telerik driver. An automation framework sits atop the driver and normally gives teams a grammar-based approach for writing tests (think Cucumber, Fitness, SpecFlow and so on).