testing – Omar AL Zabir Blog

WatinN to automate browser and test sophisticated ASP.NET AJAX sites

WatiN is a great .NET library for writing automated browser based tests that uses real browser to go to websites, perform actions and check for browser output. Combined with a unit test library like xUnit, you can use WatiN to perform automated regression tests on your websites and save many hours of manual testing every release. Moreover, WatiN can be used to stress test Javascripts on the page as it can push the browser to perform operations repeatedly and measure how long it takes for Javascripts to run. Thus you can test your Javascripts for performance, rendering speed of your website and ensure the overall presentation is fast and smooth for users.

Read this article for details:

http://www.codeproject.com/KB/aspnet/watinajaxtest.aspx

I have written some extension methods for WatiN to help facilitate AJAX related tests, especially with ASP.NET UpdatePanel, jQuery and dealing with element positions. I will show you how to use these libraries to test sophisticated AJAX websites, like the one I have built – Dropthings, which is a widget powered ASP.NET AJAX portal using ASP.NET UpdatePanel, jQuery to create a Web 2.0 presentation. You can simulate and test UpdatePanel updates, AJAX calls and UI update and even drag & drop of widgets!

You can see the implementation of automated tests in my open source project codebase.

Tests written using WatiN replaces the need for human driven tests and thus sheds significant time off your regular regression test suite. Moreover, it empowers developers with a way to quickly run regression tests whenever they need to, without waiting for human QA resource’s availability. When you hook it with xUnit like test frameworks and integrate with your continuous build, you can run UI tests automatically to test all the UI scenarios overnight after your nightly build and generate reports without requiring any manual intervention.

User story is worthless, Behavior is what we need

User Story is suitable for describing what user needs but not what user does and how system reacts to user actions within different contexts. It basically gives product team a way to quantify their output and let their boss know that they are doing their job. As a developer, you can’t write code from user stories because you have no clue on what what is the sequence of user actions and system reactions, what are the validations, what APIs to call and so on. As a QA, you can’t test the software from user stories because it does not capture the context, the sequence of events, all possible system reactions. User stories add little value to dev lifecycle. It only helps product team understand how much work they have to do eventually and it helps finance team get a view on how much money people are talking about. But to UI designers, solution designers, developers, they are nothing but blobs of highly imprecise statements that leave room for hundreds of questions to be answered. The absence of “Context” and “Cause and Effect”, and the imprecise way of saying “As a…I want… so that…” leaves room for so many misinterpretations that there’s no way development team can produce software from just user stories without spending significant time all over again analysing the user stories. Software, and the universe eventually, is all about Cause and Effect. The Cause and Effect is not described in a user story.

Unlike user stories, the “Behavior” suggested by Behavior Driven Development (BDD) is a much better approach because the format of a behavior (Givencontext, When event, Then outcome), when used correctly, lets you think in terms of sequence of events, where the context, event and outcome are captured for each and every action user or system does, and thus works as a definite spec for designing the UI and architecture. It follows the Cause and Effect model, thus can explain how the world (or your software) works. It can be so precise that sometimes a behavior work as guideline for a developer to write a single function! Not just the develoeprs, even the QA team can clearly capture what action they need to perform and how the system should respond. However, to get the real fruit out of behaviors, you need to to write them properly, following the right format. So, let me give you some examples on how you can write good behaviors for UI, business layer, services and even functions and thus eliminate repeated requirement analysis that usually happens throughout the user-story driven development lifecycle.

Read more about how user stories suck and if behavior is used throughout the development lifecycle, it can greatly reduce repeated requirement analysis effort and can make the communication between product, design, development and QA team much more effective:

http://www.codeproject.com/KB/architecture/userstorysucks.aspx

If you like it, vote for it!

Step by Step screencasts to do Behavior Driven Development on WCF and UI using xUnit

For those who have missed my presentation, I am trying to encourage my team to get into Behavior Driven Development (BDD). So, I made two quick video tutorials to show how BDD can be done from early requirement collection stage to late integration tests. It explains breaking user stories into behaviors, and then developers and test engineers taking the behavior specs and writing a WCF service and unit test for it, in parallel, and then eventually integrating the WCF service and doing the integration tests. It introduces how mocking is done using the Moq library. Moreover, it shows a way how you can write test once and do both unit and integration tests at the flip of a config setting.

Watch the screencast here:

The next video tutorial is about doing BDD to do automated UI tests. It shows how test engineers can take behaviors and then write tests that tests a prototype UI in isolation (just like Service Contract) in order to ensure the prototype conforms to the expected behaviors, while developers can write the real code and build the real product in parallel. When the real stuff is done, the same test can test the real stuff and ensure the agreed behaviors are satisfied. I have used WatiN to automate UI and test UI for expected behaviors.

Hope you like it!

Rescue overdue offshore projects and convince management to use automated tests

I have published two articles on codeproject recently. One is a story where an offshore project was two months overdue, my friend who runs it was paying the team from his own pocket and he was drowning in ever increasing number of change requests and how we brainstormed together to come out of that situation.

Tips and Tricks to rescue overdue projects

Next one is about convincing management to go for automated test and give developers extra time per sprint, at the cost of reduced productivity for couple of sprints. It’s hard to negotiate this with even dev leads, let alone managers. Whenever you tell them – there’s going to be less features/bug fixes delivered for next 3 or 4 sprints because we want to automate the tests and reduce manual QA effort; everyone gets furious and kicks you out of the meeting. Especially in a startup where every sprint is jam packed with new features and priority bug fixes to satisfy various stakeholders, including the VCs, it’s very hard to communicate the benefits of automated tests across the board. Let me tell you of a story of one of my startups where I had the pleasure to argue on this and came out victorious.

How to convince developers and management to use automated test instead of manual test

If you like these, please vote for me!

Unit Testing and Integration Testing in real projects

I am yet to find a proper sample on how to do realistic Test
Driven Development (TDD) and how to write proper unit tests for
complex business applications, that gives you enough confidence to
stop doing manual tests anymore. Generally
the samples show you how to test a Stack or a LinkedList, which
is far simpler than testing a typical N-tier application,
especially if you are using Entity Framework or Linq to SQL or some
ORM in data access layer, and doing logging, validation, caching,
error handling at middle tier. There are many
articles, blog
posts, video tutorials on
how to write unit tests, which I believe are all very good starting
points. But all these examples show you basic tests, not good
enough to let your QA team go. So, let me try to show you some
realistic unit and integration test examples which should help you
write tests that gives you confidence and helps you gradually move
towards TDD.

I will show you tests done on my open source project Dropthings, which is a Web
2.0 AJAX portal built using jQuery, ASP.NET 3.5, Linq to SQL,
Dependency Injection using Unity, caching using Microsoft
Enterprise Library, Velocity and so on. Basically all the hot techs
you can grasp in one shot. The project is a typical N-tier
application where there’s a web layer, a business layer and a
data access layer. Writing unit tests, integration tests and load
tests for this project was challenging, and thus interesting to
share so that you can see how you can implement Unit Testing and
Integration Testing in a real project and gradually get into Test
Driven Development.

Read this codeproject article of mine to learn how I did
Integration Tests and Unit Tests using Behavior Driven Development
approach:

Unit Testing and Integration Testing in business
applications

http://www.codeproject.com/KB/testing/realtesting.aspx

If you like it, please vote for me.

ASP.NET AJAX testing made easy using Visual Studio 2008 Web Test

Visual Studio 2008 comes with rich Web Testing support, but
it’s not rich enough to test highly dynamic AJAX websites
where the page content is generated dynamically from database and
the same page output changes very frequently based on some external
data source e.g. RSS feed. Although you can use the Web Test Record
feature to record some browser actions by running a real browser
and then play it back. But if the page that you are testing changes
everytime you visit the page, then your recorded tests no longer
work as expected. The problem with recorded Web Test is that it
stores the generated ASP.NET Control ID, Form field names inside
the test. If the page is no longer producing the same ASP.NET
Control ID or same Form fields, then the recorded test no longer
works. A very simple example is in VS Web Test, you can say
“click the button with ID
ctrl00_UpdatePanel003_SubmitButton002”, but you cannot
say “click the 2nd Submit button inside the third
UpdatePanel”. Another key limitation is in Web Tests, you
cannot address Controls using the Server side Control ID like
“SubmitButton”. You have to always use the generated
Client ID which is something weird like
“ctrl_00_SomeControl001_SubmitButton”. Moreover, if you
are making AJAX calls where certain call returns some JSON or
updates some UpdatePanel and then based on the server
returned response, you want to make further AJAX calls or post the
refreshed UpdatePanel, then recorded tests don’t work
properly. You *do* have the option to write the tests hand coded
and write code to handle such scenario but it’s pretty
difficult to write hand coded tests when you are using
UpdatePanels because you have to keep track of the page
viewstates, form hidden variables etc across async post backs. So,
I have built a library that makes it significantly easier to test
dynamic AJAX websites and UpdatePanel rich web pages. There
are several ExtractionRule and ValidationRule
available in the library which makes testing Cookies, Response
Headers, JSON output, discovering all UpdatePanel in a page,
finding controls in the response body, finding controls inside some
UpdatePanel all very easy.

First, let me give you an example of what can be tested using
this library. My open source project Dropthings produces a Web
2.0 Start Page where the page is composed of widgets.

Each widget is composed of two UpdatePanel. There’s
a header area in each widget which is one UpdatePanel and
the body area is another UpdatePanel. Each widget is
rendered from database using the unique ID of the widget row, which
is an INT IDENTITY. Every page has unique widgets, with unique
ASP.NET Control ID. As a result, there’s no way you can
record a test and play it back because none of the ASP.NET Control
IDs are ever same for the same page on different visits. This is
where my library comes to the rescue.

See the web test I did:

This test simulates an anonymous user visit. When anonymous user
visits Dropthings for the first time, two pages are created with
some default widgets. You can also add new widgets on the page, you
can drag & drop widgets, you can delete a widget that you
don’t like.

This Web Test simulates these behaviors automatically:

Visit the homepage
Show the widget list which is an UpdatePanel. It checks
if the UpdatePanel contains the BBC World widget.
Then it clicks on the “Edit” link of the “How
to of the day” widget which brings up some options
dynamically inside an UpdatePanel. Then it tries to change
the Dropdown value inside the UpdatePanel to 10.
Adds a new widget from the Widget List. Ensures that the
UpdatePanel postback successfully renders the new
widget.
Deletes the newly added widget and ensures the widget is
gone.
Logs user out.

If you want to learn details about the project, read my
codeproject article:

http://www.codeproject.com/KB/aspnet/aspnetajaxtesting.aspx

Please vote if you find this useful.