Silverlight Unit Test Framework: New features & download for RC0

Enhanced interactive results, more log providers, dynamic test methods and attributes, tag expressions: there's a whole lot of fun new test-dev-geek candy in the latest Silverlight Unit Test Framework download on the MSDN Code Gallery (download now available; breaking changes here).

Here's a quick look at a few of the top new features in the bits today:

Enhanced interactive results experience: The built-in test log that appears in the web browser has been improved. It now displays a progress bar, numeric test method progress, and average execution duration at the completion of a test run.

All test methods in the test run have a details expander that shows time and duration statistics, exception method details, the ability to retry/re-run a single test method, and also copy result information to the clipboard.

That's right: you can re-run a single test (passing or failing) without having to restart the debugger, the browser, or do anything other than click the "Retry this test" link - then watch as the result appears again, at the bottom of the test log.

There is a simple log provider that lets you copy all failure results into the clipboard for e-mailing or pasting in bug reports. By clicking the "Copy result details" link, we'll pop up an alert message box. Just press Ctrl + C (Windows) or select the text and press Command + C (OS X) to grab the details to then open a bug in TFS, e-mail a co-worker to nag them to fix a feature, or whatever.

TagExpressions: Easily select and run sets of tests, organize test classes and methods, and add descriptive metadata to speed up validation time. Originally implemented by expert dev Ted Glaza for the ASP.NET AJAX Toolkit, it was the perfect addition to move beyond the 'Exclusive' attribute. More about the expression language will be posted someday soon.

A built-in user interface for working with tags is not yet in today's drop of bits, but it can be set using parameters or modifying the UnitTestSettings object in the meantime:

/// <summary>
/// The Application_Startup method inside the test project's 
/// App.xaml.cs file.
/// </summary>
private void Application_Startup(object sender, StartupEventArgs e)
{
    // A tag expression that selects all tests that are NOT 
    // part of the group tagged with "SlowTests"
    string tagExpression = "!SlowTests";

    UnitTestSettings settings = UnitTestSystem.CreateDefaultSettings();
    settings.TagExpression = tagExpression;
    this.RootVisual = UnitTestSystem.CreateTestPage(settings);
}

What are they? Well, for the most part that's for another post, another day. But tags are attached to methods and classes when the TagAttribute is used (found in the Microsoft.Silverlight.Testing) namespace.

They are also automatically generated at runtime for class names, method names, and full names (namespace + name).

Visual Studio Team Test log provider output (TRX): A built-in, bare-bones log provider built with XLinq. When configured it can output *.trx results that can be opened in the Visual Studio 2008 interface. This is output-only, and not an integration story.

Test Service Provider implementation: A super easy, lightweight implementation of a set of provider interfaces for extending the capabilities of the test harness. Paired with the right environment you can get information about the environment, report results to an external test runner or continuous integration system, and more. The provider framework is simple and extensible, you can add your own providers as you see fit in your test projects and common libraries.

Dynamic test methods: For advanced scenarios and simple test generation, you can implement your own ITestClass that also implements the IProvideDynamicTestMethods interface, located in the Microsoft.Silverlight.Testing.UnitTesting.Harness namespace. The interface defines the contract for a single IEnumerable<ITestMethod> method named "GetDynamicTestMethods."

Dynamic test method attributes: It is now possible to provide attributes dynamically (as if they were attached to the real ITestMethod type, if any). The ITestMethod interface has been expanded to include a GetDynamicAttributes() method that returns IEnumerable<Attribute>.

Custom BugAttribute: The Bug attribute defined in the Microsoft.Silverlight.Testing namespace is similar to the Visual Studio known issue metadata, but it is a custom extension that alters the results of the test run.

The attribute flips the test result, accounting for expected & known bugs. This makes continuous integration testing super easy without the need for extensive analysis. A "Fixed" bool property can be used to keep the metadata once the issue is corrected in a future build.

Upcoming posts

With the test framework download now available, over the next few weeks I'll make sure to fill out the story:

• Updated test templates for RC0
• Other cool new features for testers
• Step-by-step guide for new test developers
• Understanding tag expressions
• Writing your own simple log provider
• Digging into the psuedo-asynchronous testing story built into the framework

Hope these new features are useful to you, let me know!
- Jeff