Using the new Image Verification feature

With the release of Q2 (coming in couple of weeks), our WebUI Test Studio will enable customers to build image verifications against specific elements on their pages. Image verification can be used to verify exact visual matches for both HTML pages and Silverlight applications.

To record an Image Verification step in WebUI Test Studio, follow the steps below:

1. On the recorder toolbar, enable the highlighting surface:

2.  Hover over the element on the page that you wish to visually verify.

For this example, I have navigated to the google.com home page and I’m going to verify the Google logo so that when a new logo is used, the test fails.

3. Open the element menu by clicking on the blue element ‘Nub’ that appears when you hold the mouse for 1 second. On the element menu, you will now notice a new icon (shown below):

 

4. Click the new “Image Verification” icon to start building your verification.

5. You will now be presented with an image verification editor to help you customize the verification of your image (if needed):

There are few options that you can perform to customize your image verification with this editor:

a) "Verify Entire Image" : When checked the entire selection that you highlighted on the surface will be verified. In many cases, you might not need to verify the entire image but a portion of it. As mentioned in the "Help Info" of this window, the more focused the verification, the less noise and the more reliable your test will be.

To refine your image comparison area, simply 'UnCheck' this option. This will enable the "Refine Comparison Area" section to allow you to refine your selection.

Now you can use your mouse to make a selection within the displayed image as shown below.

 

In the editor above, we have selected the "o" (the Red rectangle) in Google logo to make our comparison even more specific in terms of which area to compare within this image. As you make the selection on the image using your mouse, the coordinates of that comparison area will be displayed.

b) Refine Comparison Area: This section will display the exact coordinates of the red selection rectangle. You can use the up/down number selector for each corner to refine the selection down to the pixel level.

c) Tolerate Difference %: This is defaulted to 1%. This value instructs the comparison algorithm how much difference to tolerate between the base image and the actual image being verification. You can increase this margin depending on the scenario you are verifying.

To perform exact verification, simply set this value to zero. In that case, the verification will fail even if 1 pixel is different.

6. Once you are happy with the verification, simply click OK.

7. You are done, the verification is added to your test:

 

It is worth noting here that all our verifications can be converted to waits. Meaning, the verification above, can be converted to: Wait for the image to match. This can be a very powerful synchronization feature. To convert this step to a wait:

1. Right-Click on the verification step and select (Set As Wait):

2. You verification is now converted to a wait:

When the test executes now, the test will wait until the images match according to the defined tolerance before moving on. This can be a very powerful synchronization mechanism. For example, if you are building a Silverlight application and using an image zooming feature, you can automate the zoom-in zoom-out of your application with accurate verifications.

This feature has been requested by many customers using our Automated Testing Tools. Even though Image Verification is a very powerful feature, it has to be used only when all other automation methods are exhausted. When not used appropriately, it can cause tests to fail too often. Use the refine comparison area when possible to limit the area of verification within your image and keep the tolerance between 1-5% to avoid false positives too.

Hope you enjoy this feature….

Faris / twitter @farissweis