Web-based usability testing tools are starting to become more prevalent these days with the likes of Clixpy, Loop11 and UserTesting all appearing recently.

We shall have a look at Loop11 first of all, which has been in private beta for around 4 months now but is soon to be released to [...]]]>

Loop11 - User Experience Tool

Web-based usability testing tools are starting to become more prevalent these days with the likes of Clixpy, Loop11 and UserTesting all appearing recently.

We shall have a look at Loop11 first of all, which has been in private beta for around 4 months now but is soon to be released to the general public.

According to their web site – Loop11 is a web-based user-experience testing tool, allowing companies to conduct online, unmoderated user testing on any kind of digital interface. Loop11 is not a survey or web analytics tool, but a user experience tool… helping you to understand user behaviour.

Sounds good so far, so how do you get started?

First of all, once you log in, click on the big ‘Create New User Test’ button in bright orange so you can’t miss it. You will be presented with a form with the following fields:

Public Title – this will be seen by your test participants
Working Title – used to identify your test for you only
Copy User Test From – so you can duplicate an existing test or just start with a blank one.

Loop11 - Create User Test

Once you have filled those fields in you enter into a 5 step process to complete the creation of your user test.

Step 1 – User Test Details
This includes the ability to set your language and Loop11 supports over 40 languages.

Step 2 – Tasks & Questions
You have the ability to add tasks or questions to your user test and Loop11 provides some help if you are unsure what to do in this respect. A task would be something that you would like the user to perform such as find a particular piece of information on your web site and to add one you give it a name, a scenario that tells the user what it is you want them to do, the start URL where they commence the task and the success URL, which the user has to navigate to in order to complete the task.

I created a task to ask the user to find step 6 from the article ‘How To Test Web Forms in 7 Steps’. When the user is carrying out this task they can either abandon the task if they could not accomplish it or mark it as complete.

To add a question, you choose the question type from a list including multiple choice, rating scale and open ended. So you could ask ‘How Usable In This Web Site?’ and ask the user to rate on a scale from 1 to 10 or using words such as Poor, Average, Good, Excellent.

I entered the question, ‘How easy is it to use the search facility on this web site?’ and gave the possible answers as being Poor, Fair, Good and Excellent (I copied them from Loop11’s help text) plus I made the question mandatory.

If you want the user to type in an answer such as correctly answering how much one of your services costs then choose one of the open ended questions.

Each time you had a task or a question you are presented with the Step 2 – Task & Questions screen helpfully showing all the items you have added with the ability to either edit, delete, duplicate, preview or move them up or down in the list. Once you are happy with what you have set you click ’save and continue’ to move onto step 3.

Loop11 - Tasks & Questions confirmation

Step 3 – User Test Options
This is where you set the number of participants that you would like to complete the user test, from 1 to 999 and specify the thank you text the user sees on completion of the site. There are some other useful settings here such as being able to block or allow specific IP addresses.

Step 4 – Invite Participants
So now you get to invite the lucky participants to take part in your user test. Here you have 3 options:

i) Create a link to your user test – Loop11 generates a link for you that you can distribute any way you like.
ii) Create a popup invitation that can reside on your own web site – Loop11 gives you the code for a popup window to be placed on your web site.
iii) Purchase participants – Loop11 gives you the names of 6 companies that you can potentially purchase participants from with a link to direct users to your user test. This seems to essentially be the same as choosing item 1 although you get the details of 6 companies that you can buy users from.

Step 5 – Launch
Once you have sorted out how you are going to invite your participants you end up on the final step, to launch your user test. This gives you a summary of what you have selected and allows you to preview your test before you launch it. You should be careful because once you have launched your test you are not able to go back and edit it unfortunately.

If, like me, you realise that the questions you have set make little sense to someone seeing your web site for the first time then you should go back and edit them now before you finalise your user test.

Preview – when I clicked on the Preview button when still on step 1 I got a message telling me there was an internal server error and that someone had been notified of it. Once I moved onto step 2 the Preview button did seem to work fine and showed me the first couple of pages that a user will see when participating in the web site evaluation.

One of the tasks I had prepared was for a visitor to submit an enquiry form because I wanted to test that visitors could use the enquiry form including field validation. When previewing the test, I submitted an enquiry but my enquiry form kept telling me that I had not input the validation code that I have to prevent automated software from filling it in (even though I had input the code). This is possibly due to Loop11 conflicting with the validation method of my enquiry form so I removed that task from my test. I will raise this separately with Loop11 because many tests could require this type of input and so the preview does need to be able to handle that aspect.

Launch – once you click on Launch you will be presented with a summary page showing the details of your test plus the URL that will link to the test that you can distribute. If you elected to choose that you wanted the invitation for the test to appear in a popup window then the details of the JavaScript that you need to add to your web site will be given to you.

Loop11 - Launch screen

My Projects

Once you have launched your test you can view the results from within the My Projects tab. For each project you can see the date it was launched, how many responses received, its current status and then you can view the design of the project (what tasks, questions and other settings you selected when creating the project) and its report.

View Report

Once your test starts gaining participants you can view your report, which gives you access to the following screens:

Dashboard – first of all this shows the average task completion rate for your test presented as a pie chart. As I only had one task I can see that the average completion rate was about two thirds whilst one third abandoned the task. Underneath is presented the task results overview which shows the average page views and average time taken to reach a resolution for each task. You also get a bar chart showing the task completion rate for each task.

You are able to export the results of the report at any time in either CSV, Excel XML or PDF formats, which I think is pretty useful for any presentations that you are putting together and/or any followup actions required as a result of this testing.

Loop11 - Dashboard screen

Tasks – this page shows more detail for each task including the completion rate taken from the dashboard but also further aspects such as page views to complete task, time to complete task, most common success page, most common abandon page, most common first click and most common navigation path.

There is also a function called Participant path analysis, which allows you to go through each participant and what they clicked on when completing the task.

These reports give you invaluable information for how users approach each task that you have set them, what they click on first (possibly giving you the most eye opening results), how long it takes them to complete the task and more.

Questions – this report goes through each of the questions that you posed for your users and the results or answers that they gave. For the question in my test, ‘How easy is it to use the search facility on this web site?’, two thirds of respondents felt it was good and one third said it was excellent. However, for my open ended second question, ‘How much does the monthly link check service cost?’ I got a variety of answers where I think respondents did not really understand the question I was asking.

Not only are you testing how users experience your web site, you are also testing your ability to set good tasks and questions that your users will understand.

Participants – the participants section gives you detailed information on each person completing the test including the time they spent on the test, average time taken per task, average page views per task, their IP address, their user agent or browser and the date they completed the test.

Loop11 - Participants screen

Overall I think the results that you get are pretty comprehensive and will allow some very worthwhile followup actions to be completed to improve the user experience of your web site.

The pricing of Loop11 has still to be announced and, if the pricing is not too steep and indications that I have received from Loop11 are that it will be a reasonable price per test, my belief is that you will get a lot of great feedback that will easily pay for the cost of the test as long as you think carefully about the tasks and questions that you set for any participants.

Update (26th August) – I have been informed that pricing for Loop11 will be set at $350 USD per project, which includes unlimited tasks and questions and up to 1,000 participants. This pricing will be in place once the private beta concludes on 1st September.

LinkAlarm is a web-based link checking application that has been around for over 10 years and scans your web site checking each and every link on each web page. You can have LinkAlarm check your links once a week, once a month or you can run the scan manually whenever [...]]]>

LinkAlarm - Link Checking software

LinkAlarm is a web-based link checking application that has been around for over 10 years and scans your web site checking each and every link on each web page. You can have LinkAlarm check your links once a week, once a month or you can run the scan manually whenever you like.

So the principle of LinkAlarm is straightforward enough, let’s see how it performs.

Signing Up

You can register for a free account, which is a 14 day trial that gives you enough credit (more on credit later) to check links on up to 100 pages.

Registering is easy enough, a few details about you (your name, email address and how you found LinkAlarm) and your web site (the URL, how often you would like to check – once only, weekly, every 2 weeks or monthly and the category of your web site).

Once you have registered you are presented with a screen with a welcome message, your site details and a further message saying that your site will be checked in 5 minutes and notification of the report will be emailed once finished.

LinkAlarm - Welcome Message

I waited 5 minutes and nothing happened, no email. So I went back to the screen and read it again. It appears I had ignored some text that said ‘Please continue by confirming the entry of your site’ but there was no confirm site button and I had got confused. I clicked on the ‘Site Control’ link so see what that did and was presented with a page where it said ‘Checking’ in flashing green text. Good enough for me, I’ll leave it a little while and see what happens.

LinkAlarm - Checking Site

So, whilst that is running let me explain how the pricing structure of LinkAlarm works.

You pay for credits to be added to your account and then every time you run a report (either manually or automatically) LinkAlarm deducts 1 cent for each page of your web site. So if you have a 20 page web site you will pay 20 cents every time a report runs.

In my book that is pretty good value but is obviously more expensive for larger web sites and how often you want LinkAlarm to scan your site.

About half an hour later I received an email from LinkAlarm with the results and links to access more detailed reports online.

The Results

Within the email that LinkAlarm sends to you there is a section displaying your overall link check results. So my Testing Web Sites blog has 18 pages (still pretty small but growing) with a score of 64 and a failure of 3.7% (oh dear, really?). I had spotted that the category average for my web site is 2.5% so a failure percentage of 3.7% is a bit embarrassing and worrying. Let’s have a look at that in some more detail.

The email also gives you a number of statements (I won’t list them all here), which give you a quick impression of how well your web site performs in terms of internal links, external links and an overall score.

So, here are a couple of statements from the scan completed for www.testing-web-sites.co.uk, the comments are my comments relating to each statement:

Of the 18 pages checked, 18 (or 100%) were found to have one or more alarms.

Comment – so there is something broken on every page of my web site, great.

LinkAlarm checked 53 internal links and found 1 (or 2%) need attention.
Internal links are URLs that point to files inside your site.

Comment – there is 1 link that has a problem and it is probably on multiple pages, which is why I am getting such a high failure percentage.

LinkAlarm also checked 29 external links and found 2 (or 7%) need attention.
External links are URLs that point to files outside your site.

Comment – again, just 2 links that need attention but a high percentage because of the relatively low number of external links I have in place at present.

The LinkAlarm Score for this site (64) is an average of the page and alarm
ratios above. A score of 100 indicates all links checked reported no alarms.

Comment – I am guessing at the moment that a score of 64 is not great.

The link failure rate for this site (3.7%) is worse than
the benchmark link failure rate of 2.5% for the category – Computers & Internet.

Comment – I had spotted that too so will look into those failing links now to see what is wrong.

What the email from LinkAlarm also gives you is an understanding of what it would cost to run this report regularly on your web site plus links to their order form to order some credits. The costs are based on the number of pages and how often the report is run, for example, once each week. As I explained above the costs are based on 1 cent per page so if my web site stays at 18 pages then it would cost 18 cents to run each report or just under $10 each week for a year, not exactly big bucks.

It is helpful to know how much it is likely to cost to keep LinkAlarm checking my site but the only problem I have with this costing model is that I am aiming to grow my web site a lot over the next few months and so my costs to run LinkAlarm will continue to grow as the number of pages in my site increases.

So let’s have a look at the report now.

The Report

A screenshot of the LinkAlarm report summary

As mentioned in the email, the report gives you a summary first and the average for your category.

Then comes an Alarm Summary that shows the type of link failure for internal links, external links and total links. As you can see one of my internal links has a 405 error, which is Method Not Allowed. If you don’t know what this is then there is some handy help text available to give you an explanation.

405 Method Not Allowed means:
Common Alarm. Most commonly seen in ACTION links on FORMs where the server has not been configured to allow POST operations.

What you can do: Enable the ACTION for the correct location in your server configuration.

Clicking on the number of links shows me the pages that LinkAlarm found with the Method Not Allowed issue. This problem is being picked up on a PHP script relating to the posting of comments and is found on 7 of my 18 pages. At first glance this looks like an issue to do with my Wordpress blog or perhaps the theme that I am using, which will need further investigation and is unfortunately outside the scope of this review.

Now let’s have a look at the external links, the 2 errors are showing as 403 Forbidden. Again, some help text gives us a description of what that type of error is.

403 Forbidden means:
Common Alarm. The server will not provide the requested file. Commonly seen where content from an intranet has been made publicy available on the public internet or when your link points to a directory on a server and that server is configured to not provide directory listsings.

What you can do: Remove protection or provide and index file in the directory if necessary.

Both issues relate to links in the footer of each page of my blog, linking to the theme’s author. Clicking on those links manually does not result in a 403 Forbidden message but obviously LinkAlarm was getting that response.

So how does this happen? My view is that my theme’s author’s web site has some checking in place that works out if an automated bot is trying to access its pages and maybe responds with a different response code (403 in this case) than if my browser were trying to access the page.

This gives me a bit of a problem in trying to tidy up my LinkAlarm report because the software is going to think there are issues every time the report runs whereas in actual fact there does not appear to be any issues with the quality of my external links. This problem then is also not going to be restricted to just LinkAlarm but will extend to other link checking software too.

LinkAlarm, however, allows you to ignore certain pages or URLs and so you can maintain a list of any links such as the ones above. After ignoring the 2 external links that were producing incorrect 403 errors I ran the report again. This time I got a much more satisfactory result of 1.3% failure rate and an overall score of 86. There is now just that 1 internal link to investigate and deal with.

Other Features

It is worth detailing some of the other main features that you get as part of your LinkAlarm account, as follows:

History – shows you the history of the reports run with a graphic and summary information. Useful to see the improvement or worsening of your link check reports over time.

Download – you can download reports as zip files so you can keep copies of reports for later use.

Ignore URLS – as mentioned above.

Checking Limits – you can limit the number of pages that LinkAlarm checks if you want to. May be useful for larger sites that could stretch into hundreds or even thousands of pages.

Password Access – LinkAlarm can check password protected sites so useful for Intranets or web sites still in development.

Sites – the ability to add multiple sites to LinkAlarm so you can have it regularly checking all of your web sites using the same overall account. A good feature for agencies or larger companies who maintain a number of web sites.

LinkAlarm screenshot showing sites feature

Conclusion

Regularly link checking is a good thing and means that you are able to pick up on any link quality or linkrot problems at an early stage. Just the report run today on Testing Web Sites has raised some interesting items.

There is, however, only so much that any automated link checking software can do and this highlights the point about manually checking each and every link when you first complete an update to your web site or add a new page or blog post.

LinkAlarm is easy to set up and its free account enables you to start checking links very quickly. The ongoing costs are not great for smaller web sites but I wonder how LinkAlarm would perform for large, dynamic web sites such as an ecommerce site where they are potentially thousands of individual pages (although you can limit the number of pages or check specific pages if you want to).

The additional features that you get mean that LinkAlarm is worth considering for a wide variety of different companies and I will be continuing to use it unless I can find a better alternative.

Overall, I would recommend LinkAlarm for static web sites and possibly some further testing would be needed for larger and database driven web sites.