Organize Your Tests, Part II: Use Metadata!

In my last post I talked about some ways to organize your tests to help keep things clear and discoverable. My pal Adam Goucher, one of the top Selenium folks around, took me to task me via mail (in a friendly way!) that tags and metadata are a critical part of organizing your tests.

I agree, but with a subtle difference. Tags and other forms of metadata are great for helping you categorize your tests, but I think they come in to play more when you’re trying to execute your tests versus organize your tests on the file system.

I’ve a long background with tags and other metadata thanks to my time working in the SharePoint and social networking domains, so I think I’ve got a moderately solid understanding of the benefits and use of tags and metadata. Tags and metadata let you cut across physical organization and enable you to do creative grouping in a much more dynamic fashion.

That said, I believe the tooling around various testing frameworks and toolsets (Test Studio included) just isn’t where it needs to be to support effectively organizing tests via tags.

However…  You can do wonderful things with tags when you want to execute your tests! All of the various testing frameworks support something akin to NUnit’s Category attribute, allowing you to decorate specific tests, fixtures, or even namespaces with tags/metadata. You can then use these category tags to run only those marked tests.

In Test Studio you can use custom properties to define metadata about each test, plus we also provide things like Test Owner, priority, and a few other fields. You can use these properties and create Dynamic Test Lists which will grab only tests with the applicable values when the test is run. This means you can continue adding new tests with that particular value of metadata, and the test list will adjust each time it’s run.

To define metadata on a test, select the test in the Project view and set any of the various fields shown for that test. You can make use of the CustomProperty fields to categorize your tests as you see fit. My examples below use “BVT” for “Basic Validation Test” – a practice I’ve used in the past to separate out short-running, critical sets of tests from a longer-running suite.

Setting a CustomProperty on a test

Creating a Dynamic Test List

Bottom line: Tags and other forms of metadata bring great value to the test organization party. Just be aware of where your toolset and framework support their use. Tags and other metadata are great additions to test organization, but you still need the fundamental structure I laid out earlier!

About the author

Jim Holmes

Jim Holmes has around 25 years IT experience. He is co-author of "Windows Developer Power Tools" and Chief Cat Herder of the CodeMash Conference. He's a blogger and evangelist for Telerik’s Test Studio, an awesome set of tools to help teams deliver better software. Find him as @aJimHolmes on Twitter.