Tag Archives: Testing

Learning Management System Pilot Information Sessions

The Curricular Technology team is currently evaluating learning management systems (LMS) for use at Middlebury.  To this end the team will work with the Web Application Group to make a number of these platforms available for faculty to pilot over the winter and spring semesters.  For more information about these LMS pilots and how to participant, we encourage faculty and staff to come to one of our LMS pilot information sessions:

  • 3 – 4 pm, Monday, Dec 6th, Library 105
  • 3 – 4 pm, Monday, Dec 7th, Library 105

For more information, see:
Segue from Segue > Learning Management Systems Evaluations

LIS website usability testing

The LIS Website team invites LIS Staff to help us out with our usability testing activities. Many of you have already been involved–either through your work during the building phases of our site, or by sending us feedback about the site–and we thank you for your input. We will be incorporating these observations into our recommendations for changes and adjustments to the site.

As we turn our focus to usability testing in the form of observational testing sessions, we want to provide the opportunity for interested LIS staff to participate. The LIS Website team’s current usability testing plan involves using an audio/video/screencapture tool, and coordinating and conducting testing with student, faculty, and staff testers (along with other methods).

Please note: if you submitted questions to the usability testing form that was previously advertised, you do not need to resubmit them—we have them in hand and will incorporate them into our testing. If you have additional tasks or questions you feel are essential for testing, please send them to liswebteam@middlebury.edu. If you are interested in helping us with the testing, please let us know by emailing the team. Thank you!

There is no "pottery barn rule" on the Web.

You break it. You buy it. The age old rule of retail codified in modern furniture superstore form. Not only does this rule not apply to websites, there’s absolutely no reason to introduce such a thing on the web.

We spend a lot of time making sure that you can’t break it.

So go ahead.



This is a conversation I’ve had several times with my fellow web programmers, always in the context of, “they just don’t get it!” The idea is that we sometimes see people who are reluctant to experiment with the web applications we introduce out of fear that they might break something, or do something wrong, or bring the whole institution to its knees with a single keystroke.

Unfortunately, this prevents people from trying out all the features of the application, clicking on a button to see what it does, or publishing some of their ideas out of fear of reprisal. This is exactly the opposite of the environment we seek to create on the web. The more you experiment, the more things you try, the more you’re willing to push the medium to its extreme, the more you are rewarded.

I’m often asked, “how did you know to do that?” in relation to a feature in a web application. The simple answer is: I didn’t. Nobody was born with a priori knowledge of MediaWiki, WordPress, or any of the myraid content management systems we employ. And I guarantee you that I’ve never read a manual on any of these systems before using them. Instead, I set out trying to do something, pressed any button on the screen that seemed related to what I was trying to accomplish and then judged the response of the system to my interaction.

I’ve never managed to crash any of our servers by doing this.

Well, alright, only a couple times.

But even then, we keep regular backups of everything, we keep a history of all revisions, we engineer error checking and exception handling into our systems, we keep logs to see what went wrong and when it did. We desperately want you to use all of the features and functionality of the services we provide, so please experiment, try things out and test the limits of our systems. We’ll be happy to help you out, and even more happy if you can tell us what you tried and what happened when you tried that.

And if something does go drastically wrong, let me know because I’m sure we can make it work again. And you won’t have to pay for it.

Video on Drupal

I’ve added the Video Filter module to all of our Drupal pre-production instances to help those of you who are testing Drupal or doing preliminary content development on those servers. You can use this module to include video from one of the 15 sources that it accepts out of the box, like YouTube, Google Video, and Vimeo.

This uses the format . For example:

I also extended this by creating the MiddMedia Video Filter module which uses the same syntax as the Video Filter module, but allows you to add videos from our local streaming media server, which supports flv and mp4 video as well as mp3 audio. All the optional parameters work for the videos, but you can’t set them for the audio player. For example:

My Response to the Digital Media Tutor's Search Interface Feedback

Joe Antonioli recently asked the DMTs to evalute the search interface prototype I blogged about in a previous post. The responses I received from the DMTs were extremely thoughtful and helpful. I want to take this opportunity to provide some additional information about the search service and respond to their suggestions. First, there are two things about the new search service that may not have been entirely clear in the assignment.

1. The interface choices (colors, fonts, placement, images) on the prototype were chosen for expedience rather than good design. I reused several elements of past design to make something that would functionally work. Several DMTs responded and said they liked aspects of the design, which I certainly appreciate. However, we’ll be receiving new font, color and framing suggestions from White Whale as part of the Web Redo Project and when we do those choices will overwrite what is currently displayed on the search prototype.

2. The search engine is purposefully stupid and minimal. There will very likely be a box in the upper right hand corner of our new website where you enter search terms, click a search button and then are taken to a page with search results. This field won’t have additional selection boxes or drop downs or anything. That’s not to say that there can’t be an advanced search as well, but that the search needs to do *something* with very little information.

Those two things said, here are the main points I took from each of the DMT’s responses and my feedback on them.

You don’t always want to search all the search engines all the time. If I’m looking for a person, I might just want to search the directories and the website. If I’m looking for a book, I may only want to search the library catalog.

Excellent observation. There were two suggestions for improving this: (1) have the search results from each search engine appear in different tabs or pages so that you only have to look at one at a time, and (2) allow the user to select which search engines they want to use. I definitely want to do (2) and then see if that improves the experience enough or if we need to do (1) as well. We can implement (2) by adding a set of checkboxes under the search box that allows the user to choose which engines they want to use as one of the DMTs suggested.

There was also the suggestion that the labels of these selection checkboxes be hyperlinks that, when clicked, take you to the results from that engine. I would prefer not to do this because the suggested action for clicking on a checkbox label on a page is to check or uncheck the box associated with that label. This would break that design convention for sites. I’d rather add another row of labels under the selection area that act as these hyperlinks, if the navigation column on the left is insufficient to carry this action.

Show more results from Google, show fewer results from the library and the directories.

I agree that the Google results should list at least ten pages. I like the suggestion of limiting the results from the directories and the library by adding pagination to those results. I will not be able to effectively paginate the results from Google because their API only returns a small number of results to me – Google obviously doesn’t want you to be able to replicate their search engine on your site without displaying any of the ads they use to generate revenue. There are other restrictions with using results from Google: I cannot change the order of the results and I cannot intermix results from other search engines with results from Google. I will, however, change the script to show the top 10 results from Google, rather than the top 4.

Show more relevent results from the directory search. If I search from Ian McBride, I don’t want to have to scroll though all the Ians and all the McBrides to find him, I just want to see the information about the person I searched for.

As I said, the search is intentionally stupid. When you search for “Ian McBride” in the directory it doesn’t know if that’s a person’s name, job title, department, building location, or telephone number. So the directory search looks through an index of all those fields and spits back whatever it finds. Ideally, the directory search would list this information in order of relevence, so you’d see my entry at the top of the list, then the rest of the McBrides and then the rest of the Ians. This is really hard to do, but that doesn’t mean we shouldn’t try to do it.

Show fewer results from the library and tell me whether that resource is available or not and the branch location or whether it’s at another library.

The library search is really problematic. In order to search the current catalog system, I have to send a request to the catalog website with the search terms, parse the response looking for bibliography numbers and then send an additional request to the catalog for each bibliography number I found to get its author, title, etc. Our library staff has been looking at next generation library catalog front ends, like Scriblio, with the intention of providing a better search interface to the catalog. When we have a system like this set up that allows me greater programmatic access to the catalog information, I’ll be able to greatly improve the results from that system. I will change the current interface so that it only lists the first ten or so results, rather than the first 50.

Tell me what format the result document is in. If I’m opening a Word document by clicking on a link, I want to know.

Excellent suggestion. This should be handled by icons (with appropriate alt and tooltip text of course) for each file or media type.

Allow me to choose the number of results.

Absolutely, but with the recognition that this will be part of an advanced search interface.

The Back button on my browser doesn’t work with this search engine.

This is a usability issue which I need to correct. Clicking Back should bring you to the list of your last search results.

There were also two questions asked by the DMTs that deserve answers. The first was a compliment from one who liked the graphic of the mountains in the title background of the search result bubbles and wanted to know who came up with that design. The answer is Mark Zelis, the Web Producer for College Communications. Mark developed that design for the News Portal. You might also recognize Mark’s work on the new Institutional Diversity site and many email campaigns for the College.

The other question was, “I’m curious about Search. What is it, exactly? What search feature is in the works?” The answer to this can be found in the Strategic Recommendations Document from White Whale, which can be read by Middlebury community members at the Web Redo Blog.

I want to again thank the Digital Media Tutors for their feedback and encourage them to continue to test our systems and provide feedback on them. The more eyes there are on these interfaces, the more we can improve them and make them easier to use.

New Search Interface

One of the strategic recommendations we’ve heard back from White Whale is that our search interface can be improved by incorporating search results from multiple search sources, rather than just the GSA’s internal index. Additionally, they’ve recommended that we use Google.com’s search results scoped to our site, rather than the on-campus tool. Joe Antonioli asked me to spend some time this week building out a rough prototype of how this system could work. The first version of the prototype is available (on-campus or via VPN only) at: http://chisel.middlebury.edu/search/

Please try out this prototype and let me know how it could be improved. Don’t worry about the look-and-feel right now since those changes will be delivered by White Whale as part of the design specification. Instead, try some different searches, examine the results, and think of things the service could do to make the results more understandable to you. If the results of your search are way off-base for the search terms you used, let me know about that too. And, of course, this is such a rough prototype that there are bound to be bugs, which I’d be happy to fix.

Here are some things I already know (but if you agree or disagree, let me know):

  1. The Directory search is *really* slow. This is because we’re doing a simple search, rather than the advanced search you see on the general Directory interface. I don’t know if you’re searching for a name, a phone number, or a department, so I search all the fields for all the terms. As you might imagine, this isn’t a very fast search algorithm. What might we do to improve this?
  2. You only get 4 Google search results. I’m using Google’s RESTful search API, rather than the AJAX search API. I wanted to avoid use of JavaScript on this interface so that it would be as accessible as possible for all users. The RESTful API is much less flexible in terms of what results you can receive. They’re only giving me four at a time and I suspect this is because its not really in Google’s interest to let us create a version of their search results page without any of their advertising. I could improve this by using the AJAX version, which would require using JavaScript. But if I start using JS for this, I could use it for all the searches and do them asynchronously, which would improve response time on the page (you wouldn’t need to wait for the Directory searches to load to see the library catalog results, for example). What are people’s thoughts on this?
  3. I want to extend this to include other search sources. GO is the next up, but it doesn’t have a search interface yet, so I have to write that before I can include it here. What other services might we include in this search interface?

You can comment on this at my blog in the comments field for this entry (http://chisel.middlebury.edu/wordpress/imcbride/2009/06/02/new-search-interfacenew-search-interface/) or by emailing me directly. I’d prefer comments to be posted to the blog so that we can have a public discussion about this, but I’ll respect the confidence of anyone who would prefer not to have their thoughts published.

Testing Our Services

One of the updates that will (hopefully) happen over the holiday break is a hardware and software upgrade to the primary web service. This should increase system performance and also applies some updates to the central system we use for managing all of the content of the website (Microsoft Content Management Server). To perform this upgrade, we needed to ensure that all of the services currently in use on the primary web server would remain working once we switched over to the new machine. I just completed that testing, begun several weeks ago, this afternoon with the production of a script we can follow in the future for testing other upgrades. Just to give a sense of the scope of the services that are going to potentially be affected, here’s a list of all the stuff we tested in preparing for this update.

1	Custom PHP Applications

1.1		Campus Map

1.2		Change Log

1.3		Dining Menu

1.4		Middlebury Directory

1.5		Monterey Directory

1.6		Forms (Non-TouchNet)

1.7		GO

1.8		MAuthRA

1.9		Economics REPEC Repository

1.10		Ride Board

1.11		Schools Abroad PreDeparture

1.12		TouchNet Forms

1.13		Monterey Store

2	Custom ASP Applications

2.1		Banner Web Scout

2.2		List Subscribe / Unsubscribe

2.3		List Segmentation

3	Microsoft Content Management Server

3.1		Rendering Scripts

3.1.1			FAQ

3.1.2			Front

3.1.3			Gateway

3.1.4			List

3.1.5			ListNewsPostings

3.1.6			ShowNewsPostings

3.1.7			Academics				ListExperts				ListProfiles				PrintProfiles				ShowProfilesPermissions

3.1.8			Admin				GetByTool				SearchTool

3.1.9			Athletics				EventsManagement				GeneralManagement				GetRoster				ListCaptains				ListProfiles				TeamScoreboard

3.1.10			HR			ListJobDescriptions

3.1.11			SEO			Babysitters			JobPostings			OffCampusJobPostings

3.1.12			Tools			GetFeed			GetFooter			GetHeader			GetImage			GetLeftCol			GetRightCol

3.2		Templates

3.2.1			Contacts

3.2.2			FAQ

3.2.3			Gallery

3.2.4			Main

3.2.5			MME

3.2.6			News

3.2.7			Redirect

3.2.8			RedirectBlurb

3.2.9			Academics				Expert				Profile				Student

3.2.10			Athletics			AsstCoaches			Profile			Roster

3.2.11			Errors			Error			SSLRedirect

3.2.12			HR			JobDescription

3.2.13			SEO			Babysitter			Base			GreenCard			JobDescription			JobPosting			StudentApp

3.3		The Edit Console

3.3.1			Live and Edit Modes

3.3.2			Production Manager

3.3.3			Approval Assistant

3.3.4			Resource Manager

3.3.5			Create a New Page

3.3.6			Connected Postings

3.3.7			Opening the Edit Window

3.3.8			Saving Pages

3.3.9			Deleting Pages

3.3.10			Moving Pages

3.3.11			Copying Pages

3.3.12			Page Properties

3.3.13			Revision History

3.3.14			View Revisions By Date

3.3.15			Create Channel

3.3.16			Delete Channel

3.3.17			Sorting Items in a Channel

3.4		The r.a.d. Editor

3.4.1			Basic Functions				Copy				Paste				Find & Replace				Print				Spell Check

3.4.2			Custom Links

3.4.3			Formatting				Alignment				New Paragraph				Text Formatting

3.4.4			Listing				Horizontal Rule				Indenting				Ordered List				Unordered List

3.4.5			Non-Text Functions				Docuement Manager				Image Manager				Tables

3.4.6			Other Text Functions				Format Stripper				Hyperlinking				Insert Symbol

3.4.7			Styles				Apply CSS Class				Headings

3.4.8			Undo & Redo