Posts Tagged ‘Software Quality’

Specs and Selenium Together

August 25, 2009 1 comment

I recently had the chance to dive into a new project, this one with a rich web interface. In order to create acceptance tests around the (large and mostly untested) existing code, we’ve started writing specs acceptance tests.

Once we have our specs written to express what the existing functionality is, we can refactor and work on the codebase in more safety, our tests acting as a “motion detector” to let us know if we’ve broken something, while we write more detailed low-level tests (unit tests) to allow easier refactoring of smaller pieces of the application.

What’s interesting about our latest batch of specs is that they are written to express behaviours as experienced through a web browser – e.g. “when a user goes to this link and clicks this button on the page, he sees something happen”. In order to make this work we’ve paired up specs with Selenium, a well-known web testing framework.

By abstracting out the connection to Selenium into a parent Scala object, we can build a DSL-ish testing language that lets us say things like this:

object AUserChangesLanguages extends BaseSpecification {

  "a public user who visits the site" should beAbleTo {
    "Change their language to French" in {
      select("languageSelect", "value=fr")
      location must include("/fr/")
    "Change their language to German" in {
      select("languageSelect", "value=de")
      location must include("/de/")
    "Change their language to Polish" in {
      select("languageSelect", "value=pl")
      location must include("/pl/")

This code simply expresses that as a user selects a language from a drop-down of languages, the page should refresh (via some Javascript on the page) and redirect them to a new URL. The new URL contains the language code, so we can tell we’ve arrived at the right page by the “location must include…” line.

Simple and expressive, these tests can be run with any of your choice of browsers (e.g. Firefox, Safari, or, if you insist, Internet Explorer).

Of course, there’s lots more to testing web pages, and we’re fleshing out our DSL day by day as it needs to express more sophisticated interactions with the application.

We can get elements of the page (via Xpath), make assertions about their values, click on things, type things into fields and submit forms, basically all the operations a user might want to do with a web application.

There are some frustrations, of course. The Xpath implementation on different browsers works a bit differently – well, ok, to be fair, it works on all browsers except Internet Exploder, where it fails in various frustrating ways. We’re working on ways to overcome this that don’t involve having any “if browser == ” kind of logic.

It’s also necessary to start the Selenium RC server before running the specs, but a bit of Ant magic fixes this.

We’ve got these specs running on our TeamCity continuous integration server, using the TeamCity runner supplied with Specs, where we get nicely formatted reports as to what’s pending (e.g. not finished being written yet), what’s passing, and what’s failing.

The specs written with Selenium this way are also a bit slow, as they must actually wait in some cases for the browser (and the underlying app!) to catch up. When run with IE as the browser, they’re more than just a bit slow, in fact…

They are, however, gratifyingly black-box, as they don’t have any connection to the code of the running application at all. For that matter, the application under test can be written in any language at all, and in this case is a combination of J2EE/JSP and some .NET.

There’s a lot of promise in this type of testing, even with it’s occasional frustrations and limitations, and I suspect we’ll be doing a lot more of it.

By: Mike Nash


Day Three at BSCE

June 11, 2009 Comments off

Another day eventful day has come to a close here in Vegas. For those that have missed out I’ll give you a summary of a day here at the Better Software Conference & Expo.

The morning begins at 7:30 with a light breakfast that one of the lovely hosts refers to as, “crumbs and juice.” Quite good and they had my favourite, blackberries! Then time enough to go to the wifi lounge to check email.

The day’s festivities begin with a keynote address by Tim Lister discussing Some Not-So-Crazy Ways to Do More with Less. A thoughtful dissertation about learning to make do has the tendency to create some innovative solutions to problems. Disruption can cause turmoil, but in the end we often tend to be better off – check out the Satir Change Model.

My first lecture of the day was a provocative discussion In Defense of Waterfall by Ken Katz. This was a very lively debate where Ken was actually a proponent of the Agile Manifesto but warned that there is no panacea. It is up to us to find what works best, and then improve upon it; heh, sounds very Lean to me:)

By this point in the day it is time for lunch and networking. Lunch is a fantastic Mexican-ish buffet. In the same space is a small exposition so an hour for lunch is nowhere near enough time to talk to our peers and all of the vendors. In fact, I lost track of time and missed my next presentation, oops, oh well I had a great chat with the folks from ThoughtWorks.

I then ran off to the Agile PMP: Teaching an Old Dog New Tricks. I’ve done some courses based on the PMBOK but never carried through completion on attaining the PMP certification – I wasn’t sure that “best practises” would remain relevent in the software world. Michael Cottmeyer did a good job of showing how an AgilePMP is relevent.

The highlight of my day was learning Andy Kaufman’s Dirty Little Secret of Business. You want the secret, too? Relationships, nothing scandalous (what happens in Vegas, does not stay in Vegas), just that building relationships is extremely important – even for those of us who would prefer to spend quality time with our favourite Mac.

17:30, the day is done and I am drained… but wait, there’s more. There’s a reception, I am tired but the talk of free snacks and beer lures me in. And it gives me a chance to talk to Andy Kaufman a bit (BTW, I did not talk to Latka, I didn’t drink that much). Thanks for the pep talk, Andy. All right, now I’m ready to network.

As for Dave; I haven’t seen him since he entered into the high stakes poker tournement.  I guess he’s networking, too.

By: Kevin Bitinsky

Software Quality

March 13, 2009 2 comments

A number of the sessions I’ve attended here at SD West 2009 cover the same theme: software quality. There are a number of practices that can ensure quality; most of them involve the same thing – feedback.

  1. Requirements analysts can build prototypes to get rapid feedback from usability tests before building any code – time frame: hours or days
  2. Developers can write a unit test before each bit of production code to get quick feedback – time frame: minutes
  3. Business analysts can pair with QA roles to develop acceptance criteria / acceptance tests for each user story – time frame: hours or days
  4. Developers can run continuous intergration test suites to get feedback on whether they broke existing functionality – time frame: seconds or minutes
  5. Executives and product owners can communicate their vision for the product or feature, so the team can work toward common goals. Team members then can constantly validate what they are working on, and make design choices based on shared understanding of priorities – time frame: continuous
  6. QA teams can do exploratory testing on working code to find subtler bugs, usability problems, spelling and grammar mistakes – time frame: hours or days
  7. Developers can peer review each others code to find code smells and spot potential bugs early – time frame: hours
  8. Developers in some languages can use strong typing systems to their advantage. Strongly typing parameters and return values to validate input and output – time frame: instant
  9. Developers can use static analysis tools that help find places your code is likely to do something other than what you intended. Some of these tools are shipped with your compiler or IDE, but you can always get more detailed ones, and you can write your own to enforce local coing standards. These tools can be part of your continuous integration – time frame: seconds or minutes
  10. Agile teams can estimate task sizes to identify risks as early as possible – time frame: 1 hour

One thing that was empasized over and over, was that developer and tester are two very different roles. Developers want to write code that works; testers want to break it. Developers will always need that foil, trying to break their code using just as creative techniques as developers are using to write the code in the first place.

TDD is a method of development and code design, not a method of testing. People often get this mixed up because of the name. Even if TDD reduces your bug count by 95%, your QA team will have plenty of work to fill their time, doing what should have been their role in the first place – exploratory testing: testing the unknown-unknowns (everyone who said this, first felt the need to apologize for the Rumsfeldian phrase – you know they must really mean what they are saying, because nobody’s going around quoting Donald Rumsfeld just for the fun of it).

By: Todd Sturgeon