Presenting part 2 of 2 blog posts describing usability testing methods of the LIS Website team (as promised in the Usabilla post).
The Team presented the results of our findings at a meeting with Area Directors and since the presentation itself does a good job of providing an overview of the other tools we used, here it is: Web Team Recommendations. We will be passing the torch to a new iteration of the LIS Website team soon. They will be charged with following up on the status of these recommendations (among other tasks). In addition, we’ll be sharing these recommendations directly with the people in charge of the specific areas of the site.
The presentation is a collection of highlights from our testing data (not an exhaustive list of all the results). There are links to additional information and fuller descriptions of the issues and recommendations, within the presentation itself, but they can be seen here as well.
(The observational testing data was collected with the help of 18 student, faculty and staff testers and the Silverback testing tool, which created videos that we analyzed to get our results. Just a note of explanation about Silverback: to get this information we recorded our testers performing task-based scenarios and we drew direct conclusions about specific tasks and generalized conclusions based on overall interaction with the website. We owe an extra special thank you! to all of the testers who generously volunteered their time, braved the webcam, and who, across the board, gave us wonderful insights and feedback to help improve our site!)
We continue to invite additional feedback on all parts of the LIS Website and the College site as a whole. Even though the current installment of the LIS Website team is wrapping up its work, there are many people involved with continued improvement and refinement of the website, and we’ll do our best to make sure feedback is directed to the right people.
This post describes the usability testing that the LIS Website Team has done with one testing method. Stay tuned for a later post that summarizes our finding from direct feedback, surveys, observational testing, and this method.
The LIS Website Team used a service called Usabilla, which allows you to quickly design usability tests for web pages based on questions asking respondents to click on the page in order to answer the question. The application is described in this video:
The information gathered with this tool is highly subjective when there are few responses. The LIS Website Team has made recommendations for action based on each of the questions, but we leave it up to the people responsible for each area of the site to decide whether to implement these recommendations based on the number of people who took the survey. We also expect that the experts in each area of the site will draw their own conclusions from the data.
Here are the number of respondents for each test:
Curricular Technology: 15
LIS Homepage: 38
To see a heatmap of clicks for each question click the link for the question. Our recommendations based on the response are included below the question.
There is some confusion about where to find contact information. Both the Directory and About LIS links on the LIS homepage will answer this question, but the information is more easily found in the About LIS section and better organized. We recommend removing the Directory link from the LIS homepage and moving it as the last link in the About LIS section. We are considering adding profiles for all LIS staff to this section, but that will require significant work and organization. This work should be deferred until conpletion of the Identity Management project so we can use the extended information that tool will provide.
There were many different areas where people thought they could find this information: the Library site, the LIS Handbook, and information on visiting our libraries. Each of these places does contain some information about borrowing. They should use the Pages feature on the node editing page so that this information only needs to be updated once or, at the very least, each of these sections needs to link to their counterparts so that if the information about borrowing isn’t found in, for instance, the Handbook, people can click from there into the Library page on borrowing to read more.
The results show that the majority of our audience understands where to find LIS policies, through the appropriately named Policies link on the page. Though this goes to the College Handbook, outside of the LIS site, it should remain in the navigation.
Many people thought that they could find this information in the LIS dropdown menus that appears in the Quick Links section of the page. This menu mirrors the left hand navigation and is redundant. The dropdown menus for the current page should be removed from the Quick Links.
Many people thought that they could find this resource in the Journals A-Z tab of the Library Quick Search, which is not true. We should consider adding a Databases A-Z tab to mirror the section in the Research section to contain links to these resources.
Nearly all people selected the Library Hours right sidebar tab, indicating that, even though this information is on the right side of the page which is less prominent it is still found by most people visiting the site and doesn’t need to be moved. We received some comments that people wanted a dynamic calendar with this information. We’re working with Events Management to make this happen, but don’t know when we’ll be able to make it happen.
People were divided between directly searching for the CD through the MIDCAT search on the Library Quick Search and drilling down through the Library Collections navigation. We should add text to the Catalog search tab to make it more obvious that you can search all of the collections through this interface and do not need to visit the page for a specific collection before searching.
People were divided between the Ask a Librarian form and the Contact Us navigation link. Since this question was ambiguous about the type of help the person is seeking, both of these sections of the page are appropriate, so no changes are needed.
Most people selected the two printing links in the center of the Helpdesk homepage. These links serve different types of printing needs and the question wasn’t specific enough to differentiate. Still, the Helpdesk might consider making the distinction between the information in these two sections more obvious.
Most people thought they could find this in the Software Support section of the Helpdesk site. While there is a link on that page to documentation on Symantec Anti-Virus, there isn’t a link to the local installer. This should be added to the page and we should check the other items on that page to add links to get the software, where possible.
People were evenly split between the “I Need To Find…”, “What can we help you with?”, and “Frequently Asked Questions” links in the navigation, with many also clicking “Software Support” and “Help Portals”. The top-level navigation for the Helpdesk needs to be re-examined to make the distinction between these items more obvious.
Most people thought that they would find this information in the “I Need To Find…” section of the Helpdesk site, but there is no obvious link to that information on the page. A link to the wireless location section of the LIS Wiki should be added to this page or the top-level navigation should be changed to make this information easier to find.
Some people selected the self-help section “Help Portals”, but most chose to click the email link in the Helpdesk contact information section. Both are appropriate, but the Helpdesk should consider advertising the Help Portals section more to get people to go through there before emailing directly.
The majority of people responding found the appropriate direct link to Course Sites in the center paragraphs of the page, indicating that this information is prominent enough without needing to be a main navigation link. However, the Course Sites page does not inform the user hw to create a course site, so this information should be added to that page or this link should point to a page with that information.
Most people clicked the stories link in the navigation and the center section, but some people clicked the Tools link. The Curricular Technology Team should consider adding stories about each of the Tools to their Tools page.
Most people clicked the Help link in the site navigation which goes to a page about contacting the liaisons, the Helpdesk or the Wilson Media Lab. Some people clicked on the direct link to the liaisons on the home page and might have missed the information about contacting the Helpdesk or Media Lab. The Curricular Technology Team should consider changing the link in the center of the page to go to the Help page instead of directly to the liaisons if they think it is important for people interesting in CT to see all of the support options.
The LIS Website team has set up four quick tests to see if we’ve placed links to resources and information in the right place on the page and used the correct labels. For each test, you’ll be asked 5 questions like, “Where would you click to find out when the next Cookie Night will be?” You can click anywhere on the screenshot and can leave multiple clicks for each question. To add a comment to one of your clicks like, “I’d click here, but only because I know to find Cookie Night information on the blog…” you can click the plus (+) sign above and to the right of your placemark.
We’ve created one test for each of the four areas of the LIS Website. Each test has a different set of five questions. A test should only take 1-2 minutes to complete. Thanks for your help!
I’ve compiled the results from the LIS Website Team’s survey of LIS students workers where we asked about familiarity with areas of the LIS website and knowledge of web technologies. This was a shorter survey than the staff version with 20 students responding to the 2009 version and 17 completed 2010 surveys. Before I present the results of the student survey, I want to share my recommendations based on the results of both surveys. These are just my initial thoughts. The LIS Website Team will compile its recommendations based off this data, the comments included with both surveys, and usability testing we’re conducting this month.
[Reminder: If you want to participate in usability testing, reply to the all-campus email that was sent Wednesday, April 7 with the times when you would be available.]
Establish education programs for RSS, surveying and using the research guides. LIS will soon be forming an Education & Training team and we should recommend that this team prioritize training programs on these services, which our survey showed were unfamiliar to a large portion of LIS staff and student workers. Training on surveying could include both how to use the survey tool as well as research best practices, an area where having a Faculty adviser would be very helpful.
Improve visibility of the Emergency Protocols and make sure this information is up-to-date. Both LIS staff and student workers were largely unfamiliar with this documentation. Due to its nature, we need to make sure that these documents are available and understood.
Advertise the website as a primary source for information. LIS staff reported that many people still contact them directly for information that is available on the website. When contacted, we should refer people directly to the website to encourage people to go there first in the future.
Improve the Staff Directory. With so many staffing changes in the last year, and more to come, it’s understandable that we haven’t maintained the staff directory as well as we should. Hopefully, the Identity Management project will give us new ways to automate the staff directory and more information about its structure.
Make LIS event and space information more visible. We’ll be working with the Events Management Office over the summer on additional scheduling tools for the spaces managed by LIS. We need to make sure that this information is prominently featured on the website.
Student Survey Results
Question 2: What features of the new LIS website work or could be improved?
The hours of service for the library and the locations of LIS services both improved with the new site. This is good, since our examination of analytics data showed that these are the most visited sections of the LIS website. Students are still largely unfamiliar with the research guides and the tagging features of our site.
Question 3: Is there information not on the LIS Website that would be useful to include?
There was no change in the LIS student workers’ response to this question between the 2009 and 2010 surveys. Most respondents still feel that the information that needs to be available on the LIS Website is present.
Question 4: What is your level of comfort with the following technologies?
There was increase comfort with all of the technologies on the 2010 survey with the largest improvements in Blogs, Wikis, and Content Management Systems. This was expected as these were the tools used to build the new LIS website and several student workers were involved in that effort. The respondents were less comfortable with RSS and surveying.
Thanks to all the LIS staff who responded to the LIS Website Team’s survey! There were 63 responses to our 2009 survey, prior to the redesign of the LIS Website, and 24 responses to our more recent survey, after the launch of the new site. In this post, I will share some of the results of these surveys and compare the feedback between the two surveys. I’ll discuss the student survey in another post.
Question 3: How important is it for you to reach each of these audiences with your web content?
There was little change between the 2009 and 2010 surveys in which audiences LIS staff see as the primary visitors for their sites. In general, LIS staff now feel that it is less important to reach all of the audiences, but reaching students, faculty and staff remain the primary audiences. Faculty is now the most important audience, where LIS staff had been the most important audience on the 2009 survey.
Question 5: How often do YOU use these LIS website features?
There was little change in the usage of LIS website features between the 2009 and 2010 surveys. There was a slight increase in the use of LIS Blogs and the LIS Wiki, which we anticipated as we’ve focused a large amount of effort to improve both of these platforms in the last year, particularly by adding documentation to our wiki. The only service that saw decreased usage was the LIS staff directory.
Because of the smaller number of responses on the 2010 survey, the increase usage of the LIS Blog and LIS Wiki is not noticable until you examine the responses for each category as a percentage of the total responses. This view for each of the items is shown on the charts below.
Question 6: What features of the LIS website could be improved?
There is a clear trend with each of the responses to this question: many of the staff who felt that each of these features could be improved in 2009 now feel that these features work on our site! However, there remains a large segment of LIS staff who are not aware of these features of the LIS site, suggesting both that LIS staff may not visit the LIS site often and, for those who do, we need to do a better job of promoting these features.
The one exception to this trend is emergency procedures, where fewer respondents in 2010 believe this section of our site works and a larger percentage are unfamiliar with this content. Given the nature of this information, especially its importance to LIS staff who were the subjects of these surveys, we need to focus on improving access to this information. Hours of service and space availability were other areas that saw less improvement than other features of the site, though both were improved between the two survey groups.
Question 7: Do users often contact you for information that is available on the website?
There was little change in the response to this question between 2009 and 2010, which might suggest that we need to improve advertising of the website as a primary source for information, or steer people who contact us to the website to find the information. We should ask people who contact us whether they checked the website to find this information first to determine if the issue is people not using self-service methods to get information or if the information is too difficult to find on the site.
Question 8: Is there information not on the new LIS Website that would be useful to include?
The 2010 survey shows a dramatic improvement in this metric. I believe this is a result of the process we conducted to transfer information to the new website: all information had to be transferred by hand so people doing the work put a lot of though into including only the necessary information. It’s also probable that the permissions system for our new site, which allows areas to add people to work on the site as needed, helps keep the site up-to-date as new staff can be added to work on a site.
Question 9: What is your level of comfort with the following technologies?
Overall, LIS staff are more comfortable with all of the technologies listed than they were in 2009. The largest growths in comfort were with Blogs and Wikis, probably as a result of their emphasized use as part of the new LIS website. The most pronounced growth in the highest level of comfort was with the use of RSS.
Question 11: Please rate your overall experience in navigating the LIS website.
Most LIS staff still only feel ‘OK’ about navigating the website, but far fewer find it frustrating and some people actually love it now.
Here are exports of the data used to create these charts. If you think I’ve misrepresented the information, please feel free to post your own results based on this data. Let us know your thoughts about the survey results in the comments section here. Thanks again to all of the staff who participated!