Konstantin Petkov's blog

The UI Automation Support Highest Requirements – UIA Verify Tool Update with Preconditions Failure

Konstantin Petkov — Fri, 07 Nov 2008 09:49:56 GMT

Overall, we got quite good impression from the UIA Verify tool, I wrote about last week. Yet, there is an issue that provoked me to download the source and modify the tool behavior.

As a tester, I believe the automated tests should pass only when you get the full test running through all the actions and verifications. The original UIA Verify though marks as passed all the tests which preconditions are not met. That doesn't help much if you need to verify all the scenarios pass successfully, does it?

I've modified the source so now the failed preconditions cause test failures. Basically, there are only two small changes in the test library - in TestObject and UIAVerifyLogging classes. If you are interested, just browse for IncorrectElementConfigurationForTestException within those classes.

In future, I think of modifying the visual results adding a new condition into the whole logic. It will be generally better to separate the precondition failures from the regular ones.

You can download the updated TestLibrary from the link below:

[Download]

UI Automation Support in RadControls for Silverlight Q3 2008

Konstantin Petkov — Fri, 07 Nov 2008 09:47:24 GMT

The new release of�RadControls for Microsoft Silverlight is out with a couple of new controls and tons of improvements in the old components. In case you've missed, the new demos are uploaded and available here. You may need to delete your browser cache if you still don't see the new cool home page:

�

Along with the other improvements, the UI Automation support in Silverlight controls suite is greatly enhanced. Now the following components (I've ordered them alphabetically) officially support UI Automation:

- ComboBox/ComboBoxItem

- Expander

- Menu/MenuItem

- NumericUpDown

- PanelBar/PanelBarItem

- ProgressBar

- Slider

- TabControl/TabItem

- TreeView/TreeViewItem

- Window

What's next?

Our test results look good so far, but we still have many ideas on improving the UI Automation support of those controls. Since our requirements on the quality of UI Automation controls are quite high, we will keep working on that until we get the things perfect.

The team has also started the UIA implementation over the rest of the controls so the next official update (probably in a month) will definitely make the above list longer. Meanwhile, should you have particular request, please drop us a line and we will prioritize the upcoming tasks accordingly.

UI Automation Testing with UIA Verify

Konstantin Petkov — Tue, 28 Oct 2008 02:49:58 GMT

As I mentioned in my previous post "UI Automation/Accessibility in Silverlight 2, tools and resources summary", UIA Verify is one of the current options for UI Automation testing of controls and applications. The tool was published by the Microsoft UI Automation team about 7 months ago and according to the release page, it was downloaded about thousand of times so far. Still, I didn't find any public articles regarding its advantages not because the tool is useless, quite the opposite! So I decided to describe the tool's features briefly and I hope you'll enjoy the tutorial.

First of all, please note that UIA Verify supports WPF as well as Silverlight! This is a huge advantage for us @Telerik as our purpose is to provide complete UI Automation support both for RadControls for WPF and RadControls for Silverlight!

Next, UIA Verify comes with intuitive GUI. Should you run the executable, you see the picture below:

�

The windows tree is represented to the left. You can browse through the windows and the elements selecting those you'd like to test or you can switch the "Hover Mode" on:

�

As you can see from the modes, one can put Visual UIA Verify always on top and use the focus tracking option for the element in interest. Also, there are some modes of highlighting the element plus an easy navigation through the tree of elements.

Now I'm running our latest Silverlight 2 Release Candidate live demos. I load the demos of RadNumericUpDown, one of the first Silverlight components provided with UI Automation support. After I have focused it I hold CTRL and get it selected in UIA Verify tree (with the help of the "Hover Mode" option). The selected element is an editable input and its parent, the actual NumericUpDown control, is recognized as Spinner. So far, so good!

Let's take a look at the element's automation properties. The right panel in the UI gives you all the information regarding the UI Automation support of the element:

So our NumericUpDown is recognized through its automation ID; it is focusable via the keyboard; it has the corresponding class name; it has ControlType.Spinner type set, etc. Yes, this is the same information one can get with the help of UI Spy that I mentioned in my list of tools summary posted a week ago.

The Patterns section shows that our control implements the RangeValuePattern as expected. I expand the tree to check the values of the pattern members. Additionally, I run the SetValue method to see whether the NumericUpDown's value can be set. I successfully get the new value straight into the demo:

�

Now let's focus on the automated tests that are a part of UIA Verify. The tool comes with a base of tests for each automated element! Those are grouped by priority as you can see from the screen shot. There are tests specific to each UI Automation control type as well as some concerning the pattern. You can either run all the tests for this element, control type and pattern, or execute some specific tests:

�

Before running all the tests for our NumericUpDown control, I recommend turning on the "Always on top" mode. This way you'll have the tool always visible no matter which window is focused during the test execution for example. Running the tests opens a new window representing the progress:

�

Finally, we get our Overall test results:

�

The table is clear. Those links to the left load all the passed, failed and the caused unexpected error tests. The failed ones are documented so one can follow the test output. Those two tests fail because of AutomationId duplication along with improperly labeled input:

�

Okaaay, I have to report the issues now. Thanks, UIA Verify!

UI Automation/Accessibility in Silverlight 2, tools and resources summary

Konstantin Petkov — Mon, 20 Oct 2008 00:54:09 GMT

The purpose of this post is to summarize the tools and resources for UI Automation/Accessibility testing in Silverlight 2 available at the present moment as well as to share a few details on what's expected in the nearest future. There are a lot of resources on the web about Microsoft UI Automation framework and Active Accessibility so I'm not going to put any general information here. Still, you can refer to the list at the end of the post for additional information.

The UI Automation testing tools are quite important to developers and testers (like me) that are interested in Silverlight/WPF test and UI automation. Let me start with the UI Spy, a part of the Microsoft Windows SDK, which displays the UI Automation characteristics of the elements on each window in a tree view. The image below shows a typical view from UI Spy over Silverlight page. I've put RadSlider there to look at its UI Automation properties (please note that's RadSlider for Silverlight 2 RTW update we announced last week):

�

UI Spy allows easy search for the exact element of interest by just hovering the control on the page and holding CTRL key. You can also locate the element on the demo by enabling the location option (the tool surrounds your element with red boundaries as shown in the picture). Additionally, read about the Spy tasks on MSDN. You might be also interested in what Tamir Khason shared about UI Spy and Accessibility in WPF some months ago.

UI Spy can be used in conjunction with advanced UI Automation testing tools like White. The project is available on CodePlex. White provides rich API based on UI Automation and supports Win32, Windows Forms, WPF and SWT (Java) based applications. In the latest version labeled 0.17, White does not support Silverlight (how could that be?!), but that is scheduled for the next release, which can come up in a few weeks. I will prepare another post concerning the RadControls for Silverlight 2 UI Automation testing via UI Spy and White as soon as I'm able to test the SL extension of White.

Back in March 2008, the MS UI Automation team posted their testing framework on CodePlex as well. It is called UI Automation Verify (UIA Verify) and comes with a complete test library and GUI tool. There are built-in automation element and control tests, and I'm also excited about the option of extending the available test library. Needless to say, the testing framework usability is enormously increased by that option! I'm looking forward to getting more details about the framework and having some experience to share on this blog.

Let's move ahead to the MSAA tools. You can find some available ones as Active Accessibility 2.0 SDK Tools - Event Watcher, Accessibility Explorer and Inspect Objects. Find the bits for free download here.

Along with the UIA Verify framework mentioned above, Microsoft published their UI Accessibility Checker (AccChecker) on CodePlex. AccChecker comes with GUI tool, API for the test case creation as well as a "Screen Reader" feature, which can catch the UI's programmatic access if broken. Hmm, does "AccChecker" sound the same as "AccExplorer" for you too? Find a comparison between both tools on AccChecker threads here.

Do you want to hear (yes, to hear!) what is displayed on the screen in your Silverlight site and make sure the blind people or those with low vision can hear the expected? Use the Narrator Accessibility Speech engine of your Windows. There is an article describing Narrator as KB at support.microsoft.com. You can find the tool under Programs - Accessories - Accessibility group or you can just type "narrator" in Start - Run dialog box.

I'd like to stop here and invite your comments on the topic. Do you have a favorite tool for UI Automation or do you plan to make your Silverlight site accessible? Feel free to share your experience or impressions.

�

I've listed some additional resources below:

UI Automation Fundamentals

UI Automation of a Silverlight Custom Control

WPF Application Quality Guide - UI Automation Tools

Accessibility and UI Automation in Silverlight 2 by Pete Brown

Project White: Automated UI Testing by Ben Hall

Testing WPF applications with the White UI Test framework by John D'Addamio

UI Automation in Siverlight - Simulating User Interactions by Gabriel Isenberg

UI Automation in Silverlight - Part II (The Easy Way) by Gabriel Isenberg

Sara Ford WebLog's posts on Accessibility

Accessibility Developer Center

Microsoft Accessibility Labs

AIA - Accessibility Interoperability Alliance

�

Hope this summary helps!

�

P.S. I've attached UISpy to my post so you don't have to download the whole Windows SDK update to run the tool.

UISpy.zip�

[Yours UI Automation addicted]

-Konstantin

Joining the Silverlight Testing World

Konstantin Petkov — Mon, 13 Oct 2008 02:44:35 GMT

Back in late 2006 I started this blog, posting content based entirely on the ASP.NET tester's activities. At the end of summer 2008 I was still completely focused on ASP.NET. You never know what fate will bring you in a month, what about a year or two.

Since the beginning of September 2008 I joined the Telerik Silverlight team. My main responsibilities are to build the test strategy, organize the work in all terms of quality assurance and help the team of talented developers release a great suite of Silverlight controls.

You have probably already seen the latest nice additions to the tools like RadGauge for Silverlight, the Page Navigation framework and the Context Menu for Silverlight for example. Many more are coming in the next few weeks. However, it's important to work not just on the new tools and features but to put efforts into the tools quality as well. Experience clearly shows that is the way to completely excite our valuable customers.

Does that mean I'm out of the ASP.NET and you won't see any more ASP.NET content in this blog? Not at all! Along with some folks here I've started a brand new and innovative project related to RadControls for ASP.NET AJAX testing. The project is in early development stage, so I'll keep the secret for a couple of months more. Also, I'm writing some more topics about the test automation of Telerik ASP.NET components suite, so expect a new content on this shortly.

Talking about "automation", in Silverlight we've already announced our goal to provide full UI Automation support of our tools for the official release expected within a couple of weeks. Some implementations, concerning the UI Automation support have already been tested and are being available now in the source, i.e. you can expect new controls covering Accessibility even in the next release candidate.

I've just realized I've already been performing software test automation for almost 8 years. Just call me an addict, but I can't live without test automation. ;) In WPF and Silverlight this is entirely new world, so, last but not least, expect posts on that very topic and the tools available for Silverlight/WPF components automation testing to your humble tester's blog in the next months.

RadCalendar for ASP.NET (AJAX) localization and all the necessary to have culture independent automated tests

kpetkov — Thu, 28 Aug 2008 09:07:20 GMT

First some background:

Recently I’ve been working on an interesting task, which required digging into some details in RadCalendar control localization implementation. Due to a customer bug report, I tested the RadCalendar control in a non-English culture for a specific scenario. I also ran the automated tests we had built for the Calendar control and was shocked at the fact that so many tests simply failed. The reason is that these tests expected some certain Calendar element values, which generally depend on the culture you run the control on. I took the challenge to investigate, implement the necessary methods and update the tests so that those can be culture-independent from now on.

So for those who are interested in RadCalendar localization, this post will clarify which component elements are localized and how. The post lists the main RadCalendar features and properties that the control provides for localization.

I’m listing the RadCalendar elements being localized by the culture in use. I’m also explaining how these elements are formed so that the automated tests can build the necessary values runtime to avoid the culture dependency (attached to the post C# class provides all the helper methods in question). RadCalendar exposes some properties for localization as well as elements formatting, which are also pointed below.

Let’s start with the element people immediately find:

Calendar Title:

This is simple: by default RadCalendar creates its title by the month name and year. The component provides a couple of properties (TitleSettings) to change the title: TitleFormat, TitleAlign, DateRangeSeparator (the last for Multi-Month View mode -- see the next point) as well as TitleStyle group of properties to change the appearance. Read more about the title settings in documentation here.

The automated tests access the title by its ID: <RadCalendarID>_Title. To get the month name by culture, the automated tests call System.Globalization.CultureInfo.DateTimeFormat.GetMonthName() method.

Calendar Multi-Month View Titles

The main as well as the single month title is culture dependent as well. While the main title is easy customizable through the first and the last months, the tests locate the single month titles through the DateTimeFormatInfo.AbbreviatedMonthNames[] array.

Calendar Fast Navigation Months

Clicking on the calendar title, one gets the “Month/Year Navigation Popup” provided for faster navigation to the month/year in interest.

The month abbreviations rounded in blue depend on the culture in use. The automated tests get those from the same DateTimeFormatInfo.AbbreviatedMonthNames[] array. Note that the FastNavigation “Today”, “OK” and “Cancel” buttons in this popup can be localized through the FastNavigationSettings RadCalendar properties as in the demo below:

http://demos.telerik.com/aspnet/prometheus/Calendar/Examples/Functionality/Globalization/DefaultCS.aspx

Calendar Week Days (header selectors)

Those links select all the respective days in RadCalendar when clicked or just highlight the days on hover (the whole week can be highlighted/selected by the week number on left – see the next point). The automated tests access the elements by their title: the day names are available through DateTimeFormatInfo.DayNames[] array.

By the way, here are the rendered headers in that table (shot from Firebug):

Why don’t we use the ID of that <th> element instead? The reason is quite on topic -- the order (so the IDs) depends on the culture. In other words the Top_cs_5 element represents “Tuesday” in English but “Friday” in Bulgarian for example.

Locating the week dates

Talking about the week selectors on the other hand, here is a typical scenario the tests should cover:

Click on week number (guess what – it depends on the culture)
Verify all the days of that week are selected

This is quite interesting in terms of culture-independent test. The test click on the row element, accessed by the ID:

Then we need all the dates in the selected week. The method used for this purpose (see getWeekDates() from the attached C# class for details) locates the first day of the week through the CultureInfo DateTimeFormatInfo.FirstDayOfWeek, then lists the day names available in DateTimeFormatInfo.DayNames[] and calculates the indices difference with the given date to get the first date from the week in question. Taking all the dates from the week is now a trivial task.

Days Cell Title

To be able to click on a certain date in the calendar (simulating user selecting that date), the tests need to locate the day cell. As the order of the Calendar cells depends on the culture (again) and for the sake of readability, the automated tests don’t access the date cells by cell index, but through the localized cell title instead.

The default calendar cell title format is defined as "dddd, MMMM dd, yyyy" in RadCalendar but is changeable by DayCellToolTipFormat property. So the exact day cell title can be accessed through the DateTime.ToString(<format>, CultureInfo) method.

I would also recommend the RadCalendar documentation resources for localization as additional readings:

Specifying a culture in RadCalendar controls

Calendar localization – date format patterns

Localizing strings in RadCalendar

Happy localizing!

-Konstantin

RadCalendarCultureIndependentTest.zip

The Trickiest Tips in RadControls for ASP.NET (AJAX) Tests Automation

Konstantin Petkov — Thu, 21 Aug 2008 07:45:06 GMT

Do you wonder if there are any keys to get easy test automation over the RadControls components?

I've seen people asking about how to get a simple (at a first glance) task done through this or that test automation tool and we generally find the specific resolution with the exact tool in use. However, in most cases the things are quite similar no matter which tool you prefer. Instead of posting particular solutions though, with this post I'm sharing some of the trickiest moments one may came across while working on tests automation of RadControls for ASP.NET (AJAX) and the general solution. That is to say the questions/answers below are component-related and are not something concerning the specific test automation tool.

I won't surprise anyone here pointing that the ASP.NET (AJAX) suite of Telerik controls provides rich client-side functionality. The controls expose JavaScript functions/properties/events that are giving huge amount of options for the final implementation. At the same time, you should not care what's behind the scene at all. Each component's documentation of course describes its API in details.

Ah, did I say something about a key in RadControls ASP.NET (AJAX) tests automation? Here it is - the client-side API.

So, which are the top trickiest moments in those controls test automation I'd like to share? Let me start with a common task regarding the rich text editor - setting custom content. You know, the RadEditor is one of those giant components, which does so much behind the scenes on the client, because of its rich client-side functionality.

RadEditor

Hello, Telerik! We're using the tool X� to test our ASP.NET (AJAX) application. Whenever we encounter a page with the RadEditor on, X cannot enter text in the editor.�� X recognizes the frame where the editor is placed, but it cannot push text to the editor.

I remember I've been looking into a nice test automation tool with a colleague of mine. He didn't like it that much saying: "Look at this, even its recorder cannot work with the Editor!"

Nice try! What I answered was there is no tool which would automatically work with the editor. No, that's not correct, there is no tool, which would automagically work with any rich-text editor. So I don't know about X, but RadEditor supports . all the good tools out there. ;)

What can you do to change the rich text editor content through an automated test? Take a look at its client-side API and the method set_html() (SetHtml() for the ASP.NET version). As long as your tool provides the ability to execute some custom JavaScript, you just need to access the client-side object and call the function with your content as argument and voila, you get the editor HTML updated. You can also get the editor HTML by get_html()/GetHtml() methods.

RadInput

Hey, Telerik! When I try to set a value in a RadInput control with the tool Y, the value I set is immediately cleared out on exiting the field. I have tried numerous ways of setting this value, all with the same effect.

Yes, the RadInput controls perform some JS validation on the client so although those are not so complex components as the RadEditor, you can't just find the input element and set/replace its value. Instead, (guess what?) use the Client-Side API and the set_value() function in particular (SetValue() in ASP.NET version). Again, you can verify the RadInput value by get_value()/GetValue() methods.

Okay, here comes the funny part:

RadComboBox

Good day! How do you select an item of the ComboBox manually through an automated test written with the Z tool? I've written such a test, which clicks on the combobox arrow, the items are displayed, then the test clicks on the item I choose, but it does not get selected! I do it myself over the demo and it is selected as expected! What is going on here?!

As some fellows noticed, the trick is first to highlight the item and then click it. Otherwise the click action over the item does nothing.

That's not specific to Selenium of course as you can see in the WatiN solution shared by my colleague Elena.

Let's stop here for now and see what trick you've stumbled upon while doing some tests automation over your RadControls applications. Did you get an issue and want to share the solution you've found?

Thanks!

Konstantin

I'm back again, what comes next?

Konstantin Petkov — Thu, 07 Aug 2008 02:28:47 GMT

Yeah, I know, it's been quite some time since my last post. The ideas keep coming though and I've just realized there is no better time for a QA to blog than ... the summer. Do you know why? Well, just because of the summer time! You know, devs are humans too and they also need to rest a bit. ;) I can at least stop bugging around for a while and post some nice, interesting content.

I'm afraid the list of ideas for my future posts is already quite long. However, since I'd like to share the most fascinating with you, the readers, I'd definitely like to hear what you are mostly interested in concerning the QA department here, at Telerik.

Do you want to know about the major bugs we pushed together with the Q2 2008 release, which we announced two weeks ago? Sorry, I have to "disappoint" you -- there are no major bugs out there. ;) Seriously, it's great the customers never hear about most of the bugs we, the QAs know about.

Do you want to know more about the test automation of RadControls? I'll soon prepare some more topics on the test automation, particularly about the RadControls for ASP.NET AJAX. I'd also like to hear your opinion about some ideas concerning the test automation our customers can do. Do you want to cover your complex RadControls application with automated tests so that you can easily verify everything's working fine on next update? I can prepare a separate post on the subject discussing some ideas in details that might be quite interesting for some of you, I suppose.

I'd also like shortly to post a topic, which would surely come unexpected from a QA blog. It's something I can't find being discussed anywhere on the web, although I'm constantly trying to get information on the matter from the QA blogs. So get ready for a topic discussing the different kind of responsibilities the QA job can offer.

Hope you'll enjoy the upcoming discussions but please feel free to raise questions you'd like to get answered.

Thanks!

Konstantin

Web components styling tests automation, cross-browser solution

Konstantin Petkov — Mon, 07 Apr 2008 02:00:00 GMT

It's been quite some time since�my last post�concerning the�RadControls automation tests developed with Selenium. We still have a lot to share about our�Selenium adventure, which could help those, interested in similar solutions.

First of all, I believe you are unable to completely verify the visual appearance of your Web application via automated tests. There is no way you can skip the manual check if the app "looks" good and has no visual glitches anywhere. Still, the client-side styling tests automation can be considered to be the first step for the necessary web page appearance verification.

I would separate the main approaches I follow into two: through the rendered HTML and via JavaScript.

�

Exploring the page output through�element locators is a fundamental task in Selenium. For a styling test one can locate the particular element and perform a check for its style attribute.

Let's take this demo for example:
http://www.telerik.com/demos/aspnet/Calendar/Examples/DateTimePicker/Templates/DefaultCS.aspx

I like it because of the RadTimePicker template buttons. Each button is colored either in red or blue depending on whether it is declared within the Time or AlternatingTime template. So let's open the Firebug and inspect a button element:

�

I would go for a CSS selector to locate the element and simply check its attribute through assertAttribute� command:

<tr>

�� <td>assertAttribute</td>

�� <td>css=#RadTimePicker1_timeView button:contains('9:00')@style</td>

�� <td>width: 80px; color: Red;</td>

</tr>

�

Perfect! I run the test in Firefox (my default browser) and it verifies the button's style. Then run it in IE and ... it fails. It simply returns an 'object' in IE:

Actual�value�'[object]'�did�not�match�'width:�80px;�color:�Red;'

Now let's see the second approach. The key is the cssText attribute of the DOM element's 'style'. The following script test does the trick in Firefox for the 'red' button from the above example:

<tr>

�� <td>assertExpression</td>

�� <td>javascript{this.page().findElement("css=#RadTimePicker1_timeView button:contains('9:00')").style.cssText}</td>

�� <td>width: 80px; color: Red;</td>

</tr>

That's good, but in IE it fails again:

Actual�value�'WIDTH:�80px;�COLOR:�red'�did�not�match�'width:�80px;�color:�Red;'

You notice the upper case as well as the missing semicolon at the end. To work around that, one can call toLowerCase method reaching the following cross-browser solution:

<tr>

�� <td>assertExpression</td>

�� <td>javascript{this.page().findElement("css=#RadTimePicker1_timeView button:contains('9:00')").style.cssText.toLowerCase()}</td>

�� <td>width: 80px; color: red*</td>

</tr>

That's it! Looks a bit tricky at a first glance but does a perfect job.

Numerous RadControls tests use the above solution. Hope you like it too!

-Konstantin

[Incoming] South East European Software Testing Conference

Konstantin Petkov — Wed, 12 Mar 2008 06:30:48 GMT

��As a seven-year hard fighting Software Quality Assurance, I was surprised but glad to find out that the upcoming Testing Conference -�SEETEST 2008 will be held just a couple of miles away from the�Telerik Head Office in Sofia, Bulgaria on July 2^nd and 3^rd. It will be the first conference ever in South East Europe aiming to�cover various topics in the area of Software Testing and Software Quality Management. What impresses me is the�top experts in the area coming from all over the world - USA, South and West Europe, Israel and India. Okay, the ad should stop here. ;-)
��I have to admit I was a bit suspicious about the organization of such a big conference in the area of Software Testing right here in Sofia, but so far everything looks good! My preferences for the sessions are not important - there are a dozen of speakers having over 10-15 years of experience in organizing QA processes and teams. I was just wondering if someone there could best answer my question about how the�RadGrid�automated tests should be organized so that we can mostly cover most of the thousand scenarios in its functionality and at the same time we do not have to maintain the test suites all day long. Seriously, what I wonder is whether a specific problem resolutions in software testing can be discussed at a testing conference, without making the session quite boring for the most of the audience. Of course, the general topics are mandatory for each conference; no matter if it is just the_big_one_this_season or it is organized over a new technology or a specific area like the Software Testing. Is it possible at all? Otherwise listening to the masters sharing their experience can always bring some good new ideas.
��Oh, I know a nice restaurant at the top floor (18^th as far as I remember) of the hotel holding the conference, which offers good food and great sight over the city. So as you might have already guessed, we have gathered a strong QA Telerik team to attack the event. ;-) I hope to see some fellows there..

�Konstantin
[A suspicious QA]

Bug-tracker conversion tool: Gemini -> TFS

Konstantin Petkov — Tue, 05 Feb 2008 00:00:00 GMT

Our migration to Team Foundation Server started a couple of months ago with the Source Control adoption. Another step that we performed was to transfer our Countersoft's Gemini bug-tracker projects to TFS Issue Tracking System.

I've been searching around for a tool that would perform the job automagically ;-) and found nothing but other inquiries on the subject without answers. So, I'm posting the code, which helped us migrate our projects. Hopefully someone else can benefit from it too and I can save some of your time up.
�
Well, there is no rocket science here, but the code, I believe, is easy to reuse. Several SQL queries get all the issue information (subject, history, comments, attachments and links) from your Gemini database. Then we create the work items for the respective type, title, description, area, priority (if applicable), state, assignee, attachments and links. We've put custom mapping everywhere so that one can add his/her own lists of Gemini issue types, categories, assignees, states, etc. since Gemini allows easy customization of each project. The issue comments are added into work item's history in the appropriate format.

Some details coming to my mind:

The current query skips the closed Gemini issues
The code disallows work item duplication. Say you start the tool, but it fails somewhere throwing an exception. When you address the issue, restart the tool and it will skip the project work items that have already been created (comparing the item title).
The regular expressions helped us a lot in description formatting (although I should say it's still not perfect), but they may not be necessary at all. Our Gemini distribution includes RadEditor, which adds issue description as HTML. Thus we use the expressions to format descriptions via adding new lines, converting HTML tags, etc.
Attachments are loaded from the database and saved in the file system. TFS WorkItem Attachment requires a path to the file to be added.
Links (RelatedLink types) are added if the linked item has already been� created in TFS. Otherwise, we add a note in the item description, which could help when the item is reviewed.
Some other details are mentioned at the top of each description -- the issue ID in Gemini for reference and the name of the reporter Hm, perhaps one would like to see the submission date as well?

So how would one use that?

This is a simple Visual Studio 2005 project so extract the zip and load it
Add your Gemini SQL database and modify the SQL connection string
Adapt it to your Gemini projects:

� configure the Gemini and TFS project name, area
� add your mappings of:
- �� item types
- �� project areas
- �� list of assignees
- �� issue / item states
- �� priorities
- �� "activated by" field if necessary

��
I've added comments here and there. Hope those help!

The code is available here.

I would like to thank Mr. Hrissimir Neikov (Telerik Reporting team) who got the idea, wrote the base part and helped improving the tool. As to me, well, this was just a great learning experience!

Yours truly,
Konstantin

How precise automated tests catch totally unexpected bugs

Konstantin Petkov — Sun, 04 Nov 2007 11:21:00 GMT

You may remember my post where I mentioned the technique I like regarding writing precise Selenium tests.

I believe the good test has double value if it doesn't verify just the case it is written for, but even catches totally unexpected bugs. How does such test look like?

Several days ago our Web Grid developers were working on resolving a weird bug in the control. As soon as the new build was ready, I started Selenium tests execution to verify that everything with the build is OK. Some of the tests failed, though, throwing an error that a specified element cannot be found. This appears in several tests that check about element existence in Edit mode of the Grid. Click on ? pencil on the right in the following demo to get an example:

http://www.telerik.com/demos/aspnet/Grid/Examples/Programming/AccessingCellsAndRows/DefaultCS.aspx

Imagine a test that for instance checks about the drop down (GridDropDownColum actually) in the edit form or any other element there:

The test above verifies that the specific element is present in the 2-nd cell of the 1-st row in the edit form table. The same element could be verified by the simpler XPath expression below:

//table[@id='RadGrid1_ctl01_ctl05_mainTable']//select

You can use Selenium IDE over the demo to verify that (as I explained here ).

So why should I make this check so precise?

The failed tests threw an error because of another element rendered unexpectedly - the check box from GridTemplateColumn (first column in the above grid example). Since the CheckBox is part of the ItemTemplate only (and not EditItemTemplate - see the source from the ASPX tab below the Grid) it should not appear when the Grid is in edit mode. Of course, there was no check like "verify that a CheckBox does not appear as first element in the edit form", but the precise check in this test (1st row / 2nd cell) actually verifies the exact table structure catching any rendering issues. The test fails since the actual place of the drop down has been changed.� Of course no one will find this bug in production - it was addressed immediately as well.

This is just a simple example on the matter, but you can't imagine all the situations where such precise tests value. So the better our automated tests are the better quality our products offer!

Cheers,
Konstantin

ASP.NET Web Control Performance Test in Selenium

Konstantin Petkov — Mon, 20 Aug 2007 19:53:00 GMT

Since Selenium has been designed as a browser-oriented tool, load/performance tests definitely aren't among its strong points. With the help of Selenium Remote Control � one can make precise performance test in his/her preferred language (executing number of opening page steps for example and calculating the average time), but I will limit this post in a really simple example test based entirely on Selenium Core.

I've used RadGrid as a complex Web Control to check whether its execution time exceeds over an accepted period. What the Selenium Core test needs is just a result available for verification. So adding the execution time as text on the page will best simplify our Selenium test.

In order to fix the control execution precisely, my test class inherits from RadGrid, overriding its Init and Render events.

I check the starting time prior to control initialization (Base.OnInit) and calculate the time as soon as our grid is rendered (Base.Render). I extend the rendering adding the calculated time into a span element below the Grid. The result looks like this:

Now our Selenium test looks much straight-forward. All I do is to perform a simple text check (using assertTextPresent for example) to verify that the rendered text contains "00:00:00", which will assure that the execution time has not passed over a second.

As simple as possible, isn't it?

Of course, one can modify the test by rendering the required text in format per his/her requirements or limit the allowed control execution more strictly.

I have attached a sample web site including the inherited class (in App_Code) and a web page with the test grid demo.

RadGridExecutionTimeWebSite.zip

Pressing keys simulation in Selenium, RadInput on fire

Konstantin Petkov — Sun, 12 Aug 2007 18:04:00 GMT

Have you thought whether it's easy to automate tests, simulating key pressing?� How is this handled in a browser-oriented tool like Selenium?

Fortunately Selenium provides API for keyDown/Up and keyPress as well as a command to fire any event, which triggers correspondingly "on"-handler (of course - "fireEvent"). On the other hand the "type" command just sets the value of an input field, so that one can set new value of the input element without the need to fire any event. What if we need to see how a control behaves in case of some specific user input, though? An input control may accept digits only (like RadNumericTextBox ) or even accepts certain keys pressing (see the MaskedTextBox ).

The keyPress command should simulate a user pressing and releasing a key, so this is our choice. I put an input text element in the form to test key pressing on it (you can find all the examples/tests I use in this post attached in the zip):

�� <input name="theTextbox" id="theTextbox" type="text" />

and add a simple test of keyPress. The command needs the ASCII code of the character, so the test looks like this:

�� <tr>
�� <td>keyPress</td>
�� <td>theTextbox</td>
�� <td>\97</td>
�� </tr>

I run the test in Firefox (my default browser) and it leaves the input with 'a' as value. Then I check the same in IE, but there is no visible effect!

I stumbled upon this issue some time ago and I really did not understand that until Dan Fabulich, Selenium lead developer (I like his post here ), explained to me that the "keyPress" simply fires keyPress event in JavaScript, but its up to the browser whether to type this letter into the input field. Firefox does type it, but in most other browsers keyPress causes value change only if a JS event handler is registered to handle the event. Simply, isn't it?

So the input value should be explicitly set no matter if the event is fired. We can use keyPress/typeKeys (the latter replaces several keyPress calls) along with "type" to workaround that.

The first test solution looks like this:

�� <tr>
�� <td>keyPress</td>
�� <td>theTextbox</td>
�� <td>\97</td>
�� </tr>

�� <tr>
�� <td>type</td>
�� <td>theTextbox</td>
�� <td>a</td>
�� </tr>

Now lets take a form with ASP:TextBox and RadTextBox. The above works with the regular text box, but there is a trick regarding RadInput controls input elements: those render the actual input elements with "_text" suffix. So our test looks like this:

�� 
�� <tr>
�� <td>keyPress</td>
�� <td>TextBox1</td>
�� <td>\97</td>
�� </tr>

�� <tr>
�� <td>type</td>
�� <td>TextBox1</td>
�� <td>a</td>
�� </tr>

��

�� <tr>
�� <td>keyPress</td>
�� <td>RadTextBox1_text</td>
�� <td>\97</td>
�� </tr>

�� <tr>
�� <td>type</td>
�� <td>RadTextBox1_text</td>
�� <td>a</td>
�� </tr>

�� <tr>
�� <td>fireEvent</td>
�� <td>RadTextBox1_text</td>
�� <td>blur</td>
�� </tr>

�� 
�� <tr>
�� <td>typeKeys</td>
�� <td>RadTextBox1_text</td>
�� <td>voila</td>
�� </tr>

�� <tr>
�� <td>type</td>
�� <td>RadTextBox1_text</td>
�� <td>voila</td>
�� </tr>

�� <tr>
�� <td>fireEvent</td>
�� <td>RadTextBox1_text</td>
�� <td>blur</td>

�� </tr>

You have noticed the "fireEvent" call at the end - we need that to set the value of RadInput control by typing-letters-way.

By the way, the easiest way to set values of RadInput controls is to call their SetValue() function via "getEval" command for example:

�� 
�� <tr>
�� <td>getEval</td>
�� <td>this.page().getCurrentWindow()["RadTextBox1"].SetValue("myTextBoxValue");</td>
�� <td></td>
�� </tr>

The testing of RadInput controls client validation behavior is now easy to automate!

KPetkov_InputElementsKeyPressInSelenium.zip

Selenium IE performance issues; Tests optimization

Konstantin Petkov — Mon, 16 Jul 2007 13:57:00 GMT

As started in my previous post Selenium supports XPath expressions, which is actually one of the most commonly used DOM element location strategies. Unfortunately people complain about slower tests in IE (the problem is explained in details and discussed here) because of the XPath as well as of some failing tests in IE, created for example with Selenium IDE in Firefox. These are due to the fact that Selenium includes external library in order to support XPath in IE too and Firefox supports it internally. So in other words, Selenium XPath support definitely differs in IE and Firefox.

Almost two years ago, when I started working with Selenium here at Telerik, I was happy to see my first tests passing and simulating successfully some user actions. I had written hundreds of tests before I found the significant difference in the execution time in IE compared to Firefox. Now I always do my best not simply to write a test covering specific scenario, but to make it as optimized as possible. Is this necessary?�

Look at the screen shots below. This is just a suite of RadGrid Selenium tests.

[FF 2.0.0.4]

[IE 7.0.6730.11]

So those tests pass for about 8 minutes in Firefox, but require ~15 minutes in IE. Well, I still can optimize them a bit.�

Now imagine customers that are waiting for a stable build or even release. Imagine all the RadControls tests available being executed tens of times daily. We definitely need to have tests that run as fast as possible!

What can we do to optimize the tests?�

Here is a list of some solutions I'm after:

�� 1)�� Use the best command for the particular case! I'll take a command, verifying the text of the given element:�

Simple test scenario implementations:
-�� Check the existence of an element, having specified ID and text
-�� Check whether the element with given ID has the expected text�

Both do the same, but the second sounds better, doesn't it? Here's the source:�

assertElementPresent | //span[@id='mySpanId' and text()='mySpanText'] |

assertText | mySpanId | mySpanText

Furthermore, as soon as you know the API and use the best command, creating such tests is much easier.

Consider using store/verify/assertValue as well. This one is an assertText alternative, but for the input elements.

2)�� Avoid using XPath when IDs are available!

Ask your devs to add IDs to the elements you need to use! ;-) This is serious and I have a really fresh example.�

During the past week, our RadInput controls got new rendering! This will help us avoid some layout issues, etc. I asked our devs to add IDs to the input sub-elements like the Go and Spin buttons:

http://www.telerik.com/demos/aspnet/Input/Examples/RadTextBox/Settings/DefaultCS.aspx
http://www.telerik.com/demos/aspnet/Input/Examples/RadNumericTextBox/Formatting/DefaultCS.aspx�

So I no longer need to use complex expressions like:

click | //span[@id=' RadTextBox1_wrapper']/span/a/img[@alt='Go'] |

to simulate pressing those controls. Instead, the action looks much better now:�

click | RadTextBox1_GoButton |

and is definitely faster!

3)�� Use precise expression, to avoid continuous XPath engine iterations

As I mentioned in my previous post, the XPather Firefox Add-on will give you the full XPath to the required element really easily. However, I dislike using the full XPath as the tests become so ugly and maintaining them is a nightmare! The technique I like is something in the middle.�

A. Start from the top of the control, whose element you'd like to locate (most RadControls for example include a div element as wrapper) and go through the tree of its elements to the required one. For instance, I like starting from our Web Grid's table element (see the screen shots/examples from my previous post). This makes reading the test easier as well.

B. Use the parent element, which will make the expression a bit more specific.

Say there are a couple of input elements around the page, having different and unique parents. Creating an expression, which includes the parent like //parentType[parentAttribute]/requiredChild will make the test faster for sure. Point A is actually a private case of this one.

Generally speaking, make your expression as precise as possible, but without breaking test's readability!�

Conclusion: We always talk about product quality, but I believe we should also pay attention to the quality of our automated tests. Making the tests run faster and still keeping them easily readable will be worthwhile.