3 Usability Testing Tools – Quick Comparison

Over the last 12 months or so there have been a number of relatively inexpensive web-based usability testing applications launched. These testing tools have been gaining in popularity, as usability testing has often been seen as a time consuming and expensive exercise, seemingly only possible with the right equipment and personnel involved. We take a look at 3 different usability testing applications.

As many usability experts have been saying for quite a while now, any usability testing is better than no usability testing and the online community has responded by providing us with an excellent choice of potentially enlightening applications for our website or web application.

I have gathered 29 of these usability testing tools together and linked to each within my testing tools directory.

It crossed my mind to provide a bit more of a comparison between some of the usability applications to give you an idea of the features of each, what they provide and how much they cost.

UserTesting.com

UserTesting.com has a network of users that they pay to carry out tests based in the US, Canada and UK. These users are real people who can complete tasks set to them and think out loud as they browse to tell us what they are doing, what they are noticing and what is confusing them. This is recorded and given to you as a flash video.

How It Works

When you signup for UserTesting.com you specify certain things about your target audience including their demographic profile (including their gender, age, country, household income and web expertise), how many users you want and what tasks you want them to perform. You can specify whether you want to ensure that users completing the test have a particular technology requirement such as having a Facebook account. Users then self select if they match that requirement.

Once you have signed up to the test UserTesting notifies relevant users who respond to the test and start using your website, completing the tasks you have set them and recording their screen and voice whilst they browse.

What You Get

You receive a flash video of what they did, including their keystrokes, mouse movements, clicks and their comments. You also receive a written summary, which includes what they liked, what they didn’t like about your website and what may have caused them to leave your site.

What It Costs

Costs are on a per user testing basis, as follows:

1 user – $39
2 users – $78
3 users – $87 (equates to $29 each)
4 users – $126
5 users – $165
10 users – $360
20 users – $750

So it’s actually $30 cheaper to conduct 2 tests of 5 users than it is to test 1 set of 10 users.

I’ll be conducting a detailed review of UserTesting.com shortly and will post my findings here as soon as I can.

Whatusersdo

Whatusersdo focuses on UK users and works to similar principles as UserTesting.com. The testers are very also similar to UserTesting’s in that they have pre-qualified as being able to speak their thoughts aloud whilst they complete a task and have their screens recorded.

How It Works

With Whatusersdo you first of all select from one of three possible tests, either a usability test, a benchmarking test or SEO research.

The usability test is the one that will focus on a particular website, i.e. your website, and so would be comparable to UserTesting.com’s offering.

The benchmarking test means that the user completing the test will complete the same tasks for two different websites. In this way you could compare how your site performs against your competitors.

The SEO research allows you to set a task and then see what the user does and which sites they choose to visit to complete that task.

Once you have chosen your test, you input your instructions, choose the number of testers, their age and gender and submit your payment.

What You Get

The main item that you get is a video showing the user’s screen and mouse clicks with audio of them thinking about the task out loud. You can mark points of interest in the video by using tags plus also share or download the videos. For the usability tests you also get a usability score out of 100, which uses the System Usability Scale (SUS) or Website Usability Scale. For the benchmarking tests the user submits their preferred website.

What It Costs

The usability and SEO research tests cost £25 per user, which is about $40 or €30. There is a slight reduction at 5 users in that they cost £115. The benchmarking test is slightly more at £30 per user or around $49 or €35 although with this test 5 users are offered at no reduction (£150).

OpenHallway

The main difference between OpenHallway and the other 2 usability testing tools above is that OpenHallway does not provide testers, you need to get them together yourself. What OpenHallway does do is to provide the web-based tools for you to create your own test scenarios and then invite users to carry out those tests either remotely or locally, which OpenHallway records.

How It Works

You create the test scenarios with your instructions and then share the recording link with your participants. In this way, you could conduct usability testing with all users in a room or distribute the test for users to complete remotely. The users screen and voice are recorded in much the way way as UserTesting.com and Whatusersdo and you can view the recording in your account.

What You Get

OpenHallway provides you with the tools to create scenarios and record users screens and audio of them completing the tasks. The recordings are then stored in your account for you to watch. What you obviously need to do is find your own users.

What It Costs

OpenHallway provides plans based on an amount of space and so the pricing is a little different to UserTesting.com and Whatusersdo, as you are providing the users yourself.

Plans start at $19 per month and all plans allow you to create an unlimited number of tests within your plans storage limit.

Starter – $19 per month gives you 500Mb storage limit, which is around 90 minutes of video.
Basic – $49 per month gives you 1Gb storage limit, which is around 3 hours of video.
Plus – $99 per month gives you 3Gb storage limit, which is around 10 hours of video. You can also download your test results as MP4 video.
Premium – $199 per month gives you 9Gb storage limit, which is around 30 hours of video. You can also download your test results as MP4 video and gain additional security and API access.

Conclusion

UserTesting.com and Whatusersdo are very similar in price and what they offer with UserTesting having a more US bias whereas Whatusersdo is more UK focused. They both offer a very quick, inexpensive and easy to use service that will get users testing your site within hours from signup. Whatusersdo has a couple of interesting alternatives to the main usability test with the benchmarking and SEO research options.

OpenHallway approaches usability testing only slightly differently and is a good option if you have some users already or know where to get some. If you are conducting usability testing locally and just need inexpensive recording tools then OpenHallway would be the ideal choice. I can see OpenHallway also being the option when you want to test against certain demographics or types of users that are not available with either UserTesting.com or Whatusersdo.

These are just 3 examples of good usability testing tools. I’ll be conducting a review of UserTesting.com shortly and hopefully will also be spending time looking at some of the others in more detail. There are 29 usability testing applications contained in the testing tools directory, ranging from inexpensive web-based applications such as these we’ve looked at today going up to much more expensive usability testing software so please check some of them out.

Whichever you choose, basic and simple usability testing is now certainly a lot more straightforward, cheaper and easier to conduct now than it ever was before. So please consider trying one or more with your next project, or giving them a whirl now on an existing site.