Our team has started on a fairly large greenfield project for one of our clients, so we have the opportunity to decide what is good enough before getting too far along.  We chose Selenium as our browser testing automation framework because it is widely used and has a large support community.  That it’s free also doesn’t hurt, but sometimes that fact doesn’t help either.

We chose to use Selenium Server over WebDriver because it allows our tester, who has very little development background, to develop his tests using Selenium IDE, save the tests as Selenese HTML, and move on.  We started off exporting the test cases to C# and dealing with it that way, but it has some big issues that caused us to move away from it.  In no particular order:

  • the extra step was easy to forget
  • it was difficult to know if you were running the most up to date version of the test
  • there was no incentive to keep the original Selenese files around
  • the conversion was one-way
  • any helpers that we wrote in C# couldn’t be linked up during the export process
  • Most Importantly: there was no way to run the conversion process outside of the Selenium IDE
Add these all up and it’s just not worth the trouble.  So we’ve been running raw Selenese test cases with some custom user extensions against Selenium Server for the past month or so.  Here is a dump of what we’ve discovered regarding the current version of Selenium Standalone (2.28) and Internet Explorer 9.

Focus Is King

Selenium exhibits some strange quirks when executed inside Internet Explorer, most of which have to do with the execution of Javascript events on page elements.  The two I’ve encountered so far are ‘the annoying key press bug’ and ‘the annoying button click bug’.

The Annoying Key Press Bug

We all know Google’s auto-complete search box.  Here is a simple example that illustrates the problem by typing something into Google’s search textbox:

simpletest
open /
type q hello world
waitForElementPresent rcnt
verifyTextPresent Lady Antebellum
When run against Chrome or Firefox this test will work just fine, but run it against Internet Explorer (9 at least) and you’ll get a timeout:
simpletest
open /
type q hello world
waitForElementPresent rcnt Timed out
after
30000ms
verifyTextPresent Lady Antebellum false
Maddeningly no combination of keyDown, keyUp, keyPressed, typeKeys, sendKeys, etcetera, would make it work.

The Annoying Button Click Bug

Expanding on our last example, what happens if we click the “Google Search” button after typing into the search box?  Surely that will get us something, right?
simpletest
open /
type q hello world
click name=btnK
waitForElementPresent rcnt
verifyTextPresent Lady Antebellum
Wrong.
simpletest
open /
type q hello world
click name=btnK
waitForElementPresent rcnt Timed out
after
30000ms
verifyTextPresent Lady Antebellum false

So What Is Going On?

After fighting with this and discovering several workarounds I’ve come to the conclusion that the problem is Internet Explorer being overly paranoid.  Basically, Internet Explorer refuses to fire Javascript events on elements that do not currently have focus.  I don’t know if this is a security feature to protect against XSS or something else entirely, but there it is.  Going back to the first test case, let’s give the text box focus before we type into it:

simpletest
open /
focus q
type q hello world
waitForElementPresent rcnt
verifyTextPresent Lady Antebellum
Now the test succeeds against Internet Explorer!  Note that performing a click on a text box does not appear to be sufficient to give it focus, so you actually need to call the focus command.
simpletest
open /
focus q
type q hello world
waitForElementPresent rcnt
verifyTextPresent Lady Antebellum

And what about the click test?

simpletest
open /
type q hello world
focus name=btnK
click name=btnK
waitForElementPresent rcnt
verifyTextPresent Lady Antebellum

Not quite.

simpletest
open /
type q hello world
focus name=btnK
click name=btnK
waitForElementPresent rcnt Timed out
after
30000ms
verifyTextPresent Lady Antebellum false

So why not?  Turns out that I removed focus from the Internet Explorer window while the test was running. So if we do this:

simpletest
open /
type q hello world
windowFocus
focus name=btnK
click name=btnK
waitForElementPresent rcnt
verifyTextPresent Lady Antebellum

Victory!

simpletest
open /
type q hello world
windowFocus
focus name=btnK
click name=btnK
waitForElementPresent rcnt
verifyTextPresent Lady Antebellum

So how can we solve this without requiring 3 calls for each click/type we want to do in our system?  I added a user extension that deals with both the click and typing focus problems.
Any place you encounter this issue in Internet Explorer just replace type with compatibleType and click with compatibleClick and it should be good to go.  Now that I look at this again, I could probably completely override the built-in type and click functions in Selenium so no changes to the test scripts are necessary.  I’ll update the Gist and this post if I go that route.

Enhanced by Zemanta