Squak !
18Apr/10

Continuous Integration Testing for the Windows UI

Testing Windows UI’s… The search…

Testing Windows User Interfaces ... I've been trying to find a good solution to this for years. It's not easy.

Every time there is a change in a UI, there is a high probability of introducing a code bug. So, I want to test it. Ideally, I'd like to have automated tests. But, the automated UI testing tools I've found all have huge holes in them of one sort or another.

Either they are too expensive for the project I'm working on, or don't work for the UI tools I am using, or break every time a minor UI change is made, or say they test the UI but actually don't, or take lots of time to write tests in, or have a steep learning curve, or have useless documentation.

I've spent hours... days... weeks... (!) looking for good tools. It is most frustrating when I see some hint of a tool that is great... then I try to find it... and it isn't there. It isn't anywhere! Dead links... downloads I can't get... email addresses that bounce.

Finally, I think I may have found something last night... buried deep inside MSDN...

Testing Windows UI’s… Finding Accessibility…

So, I found a hint somewhere that the Accessibility interface can be automated. Hmmmm... Started digging through MSDN and found this at Accessibility :

Windows Presentation Foundation
Accessibility
Microsoft UI Automation is the new accessibility framework for Microsoft Windows. It addresses the needs of assistive technology products and automated test frameworks by providing programmatic access to information about the user interface (UI). In addition, UI Automation enables control and application developers to make their products accessible.

Sounds promising. But, it wouldn't be the first time that documentation sounded promising. More digging ... trying to find examples ... no luck... oh, wait, they call them "samples", not examples. And there are some.

So, I download the "simple provider example" and look over the code. It's for a custom control. They consider a custom control simple? And, I can't see how to make a test out of it. Not useful. Time to see if I can write a really simple example.


UI Testing with Accessibility

So, what is a simple test case for this ? I settled on a dialog box, a button and a text box.

Testing with Accessibility Form1

Testing with Accessibility Form1

When the button is clicked, it puts some text in the text box. My test should click the button and then look for the text box being filled with the correct text value. My test should be callable by a [TestMethod].

Oh, I almost forgot. I'm currently using TeamCity to do test automation. This all has to work under TeamCity, which does not natively handle UI testing.

Here is the code of the form:

public string _sample_text = "This is a test";
private void button1_Click(object sender, EventArgs e) {
  textBox1.Text = _sample_text;
}

Now, I add a way to click on the button and look at the textbox contents:

public bool _Tester_Method( ) {
  var _result = false;

  // Click the button
  var _Target = button1.AccessibilityObject;
  _Target.DoDefaultAction(  );

  // get the text box contents
  var _T = textBox1.AccessibilityObject;
  var _Text = _T.Value;

  // compare the text with the expected value
  if ( _sample_text == _Text ){
    _result = true;
  }
  return _result;
}

The AccessibilityObject is what gives me the ability to send a click to the button. It also gives me the ability to get the contents of the text box.
The build fails until I add a reference to Accessibility in Solution Explorer / References / Add Reference / .Net / Accessibility .

Then, I add a Test Project to the solution, and add the test code to the Test Method...

[TestMethod]
public void TestMethod1() {
  var _result = false;
  var F = new Form1();

  _result = F._Tester_Method();

  F.Dispose(  );
  Assert.AreEqual( _result, true );
}

The Test Method needs a reference to the code it is testing:

Add Project Reference

Add Project Reference

This built fine, and TeamCity was even able to run the tests.

Success! Or, at least the first step to it...

UI Accessibility Testing with Private Accessors

Well, that all went great.
I created the test, sent it through to TeamCity and the test ran successfully.

But, how to test private methods ?
In the earlier test, I used what is essentially a public wrapper to a private method.
That adds a lot of code.
Is there a way to do it without adding a lot of code to the program under test ?
It turns out there is.

VS will create something for me called a "Private Accessor".
It's basically an interface layer that allows a test project to call private methods.
VS will even generate test stubs for all my methods.
Will this work under TeamCity ? I'm certainly going to see...

Keystrokes: Test / Unit Test / Unit Test Wizard.

Select the form. Click Settings. Adjust the names. I changed it from "[Class]Test.cs" to "[Class]_Test.cs", because I find it easier for me to read.

VS Test unit Test Wizard

VS Test unit Test Wizard

I only selected "Form1" on the wizard. So, it only created one .CS file: /TestProject1/Form1_Test.cs . Inside Form1_Test.cs were a bunch of skeleton wrapper methods; one for each method in Form1, including the public test wrapper I had created. So, after a bunch of cleaning and coding, I had this test method:

///
///A test for button1_Click
///
[TestMethod()]
[DeploymentItem("AccessibilityTest.exe")]
public void button1_Click_Test() {
  Form1_Accessor target = new Form1_Accessor();
  object         sender = null;
  EventArgs      e      = null;

  target.button1_Click(sender, e);

  // get the text box contents
  var _Text = target.textBox1.Text;

  // compare the text with the expected value
  Assert.IsTrue( target._sample_text == _Text );
}

I checked it in, and TeamCity ran the test just fine. This is starting to look like success. Now, to try this technique out on a real project...

Almost… But, not quite...

So, I built this into my real project and it ran fine in Debug mode.  Then I created a Release mode project and the CI didn't work.  For now, this is close enough for the project this is targeted for.  There's a bug posted against it at TW-11148 importData java.io.FileNotFoundException when Configuration: Release and UnitTestFramework Accessor

Comments (0) Trackbacks (0)

No comments yet.


Leave a comment

You must be logged in to post a comment.

No trackbacks yet.