“The Future is mobile, is your library ready?”
May 20, 2010 1 – 4pm

The future of information services and mobile technology is tightly intertwined. That’s why OCLC and Library Journal have come together to present a free online symposium on the future of mobile.

• How will better connection speeds affect services and functions?
• What will the rise of the smartphone mean to personal computing?
• How will upcoming mobile trends impact your library, your users, and our culture?

Join our panel of mobile industry experts and librarians and find out.

Register today at: http://www.oclc.org/innovation

Nope.

The Strategy

After “the font is too small” and “it’s too hard to edit”, search results were the top complaint about our old site during the web makeover’s requirements gathering phase. We heard that people got better results about our site from Google.com than they did from the GSA. The designers we worked with to build the new site proposed a solution in three parts:

1. For some searches, you want to craft a hand-written response. If someone searches for “natatorium hours”, tell them “The pool is open right now! Here’s the full schedule…”. This also includes ambiguous searches like “summer”. We have a lot going on in the summer: Language Schools, two Bread Loaf programs, etc., so one Key Match isn’t going to cut it. We need to show a list of the top things having to do with “summer” at Middlebury.
2. For other searches, there’s no need to display a search results page. If you search for “webmail”, you probably do want to read articles about webmail being upgraded last year, you just want to check your email on the web. For these, we should deliver the user directly to the page.
3. If the search doesn’t fall into either of these categories, we should show a list of search results, but if people say that the search results from Google.com are better than those from the GSA, then why not just show them the results from Google.com? Also, we should provide some results from other databases like our Directory or Course Catalog.

Fortunately, these recommendations were easy to implement. For the first class, the custom search result pages, I developed a template that can be used like any other theme on our site for a page. If a page is using this theme, then it will be the search result for any search of its URL. For example, there is both a men’s and women’s hockey team at Middlebury, so if you search for “hockey” it’s not always clear what you want. The custom search result page for “hockey” lists the scores for both teams, links to the team pages, a link to order tickets, a link to the page about our hockey rink, and a link to the intramural team. Barbara has put together several of these custom search result pages based on data we’ve gathered about the most popular searches on our site.

The next class of search results, the automatic redirects, were also easy to manage. We’ve compiled a large list of URLs and quick terms referring to those URLs over the last couple years: the GO database. If you search for a GO shortcut, you’ll be automatically taken to the page for that GO shortcut. For the large majority of GO shortcuts, this works very well. If you search for “bannerweb”, you’ll be taken to go/bannerweb, searching for “eres” brings you to the e-reserves site. There are a minority of searches where this doesn’t work as well: “german” takes you to the German department’s site, but you might have been looking for the German language school or several other possibilities. I’ll describe how we’ve addressed this issue in a bit.

Google.com and the 404 page

The last category of search results got us into some trouble. When we first launched out new site, the standard search results were coming from Google.com, but Google hadn’t updated its search index to reflect the contents or structure of our new site. I had thought, based on experience with the MIIS site, that it would take Google 2-3 days to index our site and search would mostly be normalized after that. This actually did happen. Google’s index of our new site was generally complete in that timeframe. However, the new pages were listed much lower in the results than links to the old pages. Since most people click the first few links in results, they wer only seeing the 404 page, getting frustrated and leaving search before finding the working links further down.

I overlooked two differences between Middlebury and MIIS that made a big difference here.

1. Middlebury’s website is linked to from many more pages than MIIS’s site, both internal links (we have many more pages) and external links (peer institutions, etc.). Google’s search algorithm is weighted to push up pages that are linked to more frequently. Since other sites haven’t updated their links to Middlebury, Google assumes that those are the right links since there’s a lot more of them and pushes them up in the results. This was less of a factor for MIIS because MIIS is linked to less frequently.
2. We have kept around paths to sites at Middlebury for over 10 years. All of the old /~department_name, /offices/department_name, /depts/department_name, paths on cat.middlebury.edu, etc. from 1997-2009 still worked in January, 2010. These paths were created before Google even existed.

   Google.com in 1998

   Google’s index has never really been updated to reflect changes in our information architecture. We wanted to move away from this practice because:

• It produces multiple results listings in search. If you searched for the Bread Loaf Writer’s Conference, you’d get a result with a link to its homepage at /~blwc, then another with a link to its homepage at /academics/blwc, then another with a link to its homepage at /depts/blwc, and so on. These all go to the same page and push other relevant results for that search further down the page. Ideally, the homepage should be the first result and other pages related to the program following it. By removing the old IA, we minimize the number of duplicate results.
• We are now allowing you to control the IA of the site, which we didn’t do before. On MCMS, I had a slick rewrite rule that allowed me to direct requests for academic departments sites because we required that they be named the same as they were in the old IA:RewriteRule ^/~?(depts/)?((?:alc|art|bio|chem|chinese|classic|cs|dance|econ|english|es|filmvid|french|geog|geol|german
  |haa|hist|ipe|is|italian|japanese|math|mbb|music|neuro|philo|physics|portuguese|ps|psych|rel
  |russian|soca|spanish|teach|theatre|ws)(?:[/\\\?].*)?)$ /academics/ump/majors/$2 [R]So if you went to /depts/filmvid you were taken to /academics/ump/majors/filmvid. I can’t do stuff like this anymore because the departments can now change the path to their sites without alerting me to the change. It gets even harrier for sub-pages of those sites. It would be a logistical nightmare to maintain automated redirects for all the variations. I think allowing departments to add pages to their site without submitting a Helpdesk ticket is a fair tradeoff here.
• Some portions of our new IA overlap ports of our old IA, like /offices and /admissions. There were going to be broken links in these areas no matter what I did.
• A really nit-picky point, but reducing the number of paths to a site improves the responsiveness of the site. Every time the server redirects you, a full request-response chain is generated. It’s faster to go right to the final URL than bounce between all these alternatives. We’re talking about milliseconds of difference here, but hey, every bit counts.

What I should have done was begin to phase out the old URLs last year. Starting with the /~department_name addresses and workign forward in the IA timeline. This would have reduced the shock on launch day and sped up conversion to the new IA. This is a lesson I’ll take to future projects of this nature.

That’s what happened, now here’s what we did.

Solving the 404 issues

Indexing Speed

Google offers a service named Google Webmaster Tools where you can see information about your site and control some of the ways that Google interacts with the site. The first thing we did was to double the speed at which Google crawls the site. Google finds out about information our your site by automating typical user interaction on the site: a program they run will request your homepage, then request every page linked to from your homepage, and so on. The faster Google indexed this content, the faster information about our new site would be available in their index. While we were still having performance issues with the site, we needed to decrease this indexing speed, but were able to increase it again as we solved those issues.

Sitemap Files

Our next step was to create a sitemap file. Since Google’s crawler only looks at pages that are linked to from other pages, it might miss some content that isn’t linked to from anywhere, or very few places. A sitemap file is a really simple text document that tells search engines about every page on your site so that they have a base to check their index against. Again, this was done to make sure that the search engines had as much information about our new site as we could provide. At the same time, Adam and Chris worked to block search engines from looking at our old site or portions of our new site that we don’t want indexed by their engine and making those entries in our robots.txt file which tells search engines which paths they should ignore.

Removing Links from the Index

Google also offers you the option of requesting that a URL is removed from its search index for six months, after which we can assume that the index will have updated to reflect that the page is permanently gone. We were able to retrieve a list of the broken URLs (about 100,000 of them) from Google’s Webmaster Tools, and started to run through the list. The problem with the URL removal tool that Google offers is that it only lets you request one page removal at a time. A developer at another college noticed this problem too and wrote an application that fills out the removal request form for you over and over again to remove the tedium from the process.

I started using this to remove some of the URLs from Google’s results and noticed that I was only able to submit 1000 URLs per day from an account. It also took about a day for the new requests to be processed. For a time, I was submitting multiple thousands of broken URLs through this tool using multiple accounts, but that too stopped working, I guess because someone at Google noticed what I was up to. I now take a more targeted approach to the situation.

Each morning I run the following script, inspired by this Drupal blog post, on each of our front-end webservers:

gawk ‘{ print $9,$7,$11,$4 }’ /var/log/httpd/www.middlebury.edu-access_log  | grep ^404 | grep google.com/search > 404.txt

This produces a report of all of the requests coming from Google searches that result in a 404 page. I then combine all of the reports and submit the non-duplicate pages from it through the URL removal tool. This turns out to only be a hundred unique pages per day, since we have eliminated most of the top level pages and are working on the “long tail” of search results.

I can then use this command to find out how many requests from Google.com to our site result in a 404 page each day:

grep ” 404 ” /var/log/httpd/www.middlebury.edu-access_log* | grep google.com/search | grep “09/Mar/2010″ | wc -l

I’ve been recording the results for a couple weeks now and have noticed that this method appears to be working very well at reducing the number of bad landing pages on our site.

I noticed one more thing about 404 pages in the logs from this morning. I had already submitted, and processed, the removal of the old address for the Bread Loaf School of English several times, but it kept appearing as broken in these logs. Looking at the search results page for BLSE I noticed that there is a Maps result. This isn’t part of the normal Google search index so the broken link wasn’t being removed by my request.

To update this, you need to change the record in the Google Local Business Listing administration interface. This is actually a pretty neat tool. It lets you list the location, phone number, email address, and other information about your business to add to the Google Maps interface. You can also upload images and videos about your business. I added all of the information I could about the BLSE except for the screen where it asked if we had any current coupons, though that’s not a bad idea – 10% off your Masters perhaps? Google called the BLSE office and gave them a PIN, which I entered into the interface and now their listing in Google’s search results is better than ever.

Also, since it is integrated with Google Maps, we get some interesting information about the people who search for the BLSE. For instance, people from Springfield, MA need more help getting to the campus:

Where driving directions requests come from:

Top Internal Searches

We’ve also been looking at the list of top searches on our internal search interface and adding GO shortcuts for all the items where there’s only one page you’d want for that search or custom search results page for the more ambiguous items. These results come from one month of Google Analytics information on our site.

Interface Improvements

Using GO for Search Results

I knew that automatically forwarding people to the page of a GO shortcut if they searched for one would be controversial. Everyone agreed with the concept of forwarding certain searches to their final destination, like “bannerweb” or “menu”, but people were alarmed at the extent to which I suggested we take this feature. However, after looking at the list of internal searches, it became clear to me that our top search terms were already GO shortcuts and were shortcuts for which there was only one logical destination.

Still, I am sympathetic to the issue I raised before about certain searches, like “german” going to a department page when there are a lot of other relevant pages for that term. Ideally, these searches would have a custom search result page, and we will likely build one for each of the terms, but those take a while to develop. Instead, we now use a really old feature of HTML, frames, to show a banner at the top of a page you’ve been forwarded to so that you can click back to the full search results if you didn’t find what you were looking for. My original idea was to just have this display in our Drupal site using the themes native to that platform. Adam suggested extending it to any site using frames.

What you see if you search for "banner" on our site.

Frames are a bit of a controversial feature of HTML. Few people consider using them any more as layout based on Cascading Style Sheets has replaced both tables and HTML frames as the preferred method for laying out a web page. Still, they do have some valid uses and I’d contend that this is one of them. What frame do is split your browser window into multiple windows. A classic example is the Java API documentation where you’re looking at three windows in one.

For the GO search results, we use two frames: one on top that links you back to the full search result and one on the bottom that shows you the search results page.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml">
<?php if ($go_url): ?>
 <head><title>Search Results</title></head>
 <frameset rows="30px,*">
 <frame marginheight="0" marginwidth="0" scrolling="no" noresize="noresize"
        src="<?php print $base_path . drupal_get_path('theme', 'midd-search'); ?>/searchbar.php?search=
        <?php print $q2; ?>" />
 <frame id="contentFrame" src="<?php print $go_url; ?>" />
 </frameset>
</html>
<?php else:  // print the full search results ?>
<?php endif; ?>

The searchbar.php script itself is a really simple page that just displays a message and a link:

<?php $search = str_replace("+", " ", $_GET['search']); ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <style type="text/css">
 #searchbar { margin:0px; padding:0px 18px; width: 100%; height:30px; line-height:30px; font-size: 1em;
     font-family:Verdana,"Lucida Grande",Lucida,sans-serif; background-color: #BEDA90; color:#003468; }
 .closeBar {position:absolute; right:0;}
 .closeBar a {padding-right: 18px; text-decoration:none;}
 </style>
 <script type="text/javascript">
 var mainloc = parent.document.getElementById('contentFrame').src
 function closeFrame() { window.top.document.location = mainloc; }
 </script>
 </head>
 <body>
 <div id="searchbar"><span><b><a href="javascript:closeFrame()">X</a></b></span>We think this is the
   right page for your search of <b><?php print $search; ?></b>, but if it's not, you can <b>
   <a href="http://www.middlebury.edu/search?q2=<?php print $_GET['search']; ?>&nocustom=true"
   target="_top">view all the results</a></b>.</div>
 </body>
</html>

You can click on the “X” in the upper right to close the frame and keep browsing. One of the limitations of this approach is that frames can’t actually communicate with each other for security reasons. If they could, a site could create a really small frame with malicious code and then a really big frame with any site on the internet, then have the small frame execute its malicious code on the previously secure big frame. Since they can’t communicate with each other, when you close the top frame it will take you to the location that the bottom frame was at when you first saw it. So if you browse around for a bit in the bottom frame, then close the top frame, you’ll be taken back to your original search result page.

GO terms in the Search drop-down

Right before our site launched we noticed that the constituent landing pages like Current Students and Faculty & Staff had these search boxes on them with the label “go” in front of them. The idea was to let people search the database of GO shortcuts. We didn’t have any way to do this at the time, so Adam developed a little module for Drupal that made a request to the GO database to conduct searches of terms and used the jQuery autocomplete plugin to make it so that the results were returned to the user in real-time.

function go_fetch_url ($name) {
 if (!is_string($name))
 throw new InvalidArgumentException('$name must be a string.');

 if (!strlen($name))
 return array();

 $pdo = go_pdo();

 if ($inst = variable_get('go_scope_institution', '')) {
 $stmt = $pdo->prepare("SELECT code.url FROM code LEFT JOIN alias ON (code.name = alias.code)
   WHERE (code.name=:name1 AND code.institution=:inst1)
   OR (alias.name=:name2 AND alias.institution=:inst2)");
 $stmt->bindValue(":inst1", $inst);
 $stmt->bindValue(":inst2", $inst);
 } else {
 $stmt = $pdo->prepare("SELECT code.url FROM code LEFT JOIN alias ON (code.name = alias.code)
   WHERE code.name=:name1 OR alias.name=:name2");
 }
 $stmt->bindValue(":name1", $name);
 $stmt->bindValue(":name2", $name);
 $stmt->execute();

 $row = $stmt->fetch(PDO::FETCH_ASSOC);
 if (!$row)
 throw new Exception('No result matches.');

 return $row['url'];
}

I took this and applied it to the search boxes throughout the site, making a couple modifications. The GO boxes on the constituent page assumed you only wanted to search GO and would complete your search term for you. On the site-wide search, we know that not every search will be covered by a GO shortcut and left that out. I also added “go/” to the beginning of each of the results so that people were more aware of what they meant using the autocomplete plugin’s formatItem option:

$('.go_query').autocomplete(
 url,
 { max: 30,
 width: 200,
 autoFill: false,
 selectFirst: false,
 formatItem: function(row) {
 return "go/"+row[0];
 }
});

Results from the Google Search Appliance

Though we had initially wanted to move away from using the Google Search Appliance (GSA) for search results, because so many of the links to our site on Google.com were broken, we reindexed the new site using the GSA and added its results to the search results page. This involved requesting results from the GSA in a way that we hadn’t done before. We used to just have the GSA serve as the search front end using the XSLT style sheet interface that the server provides, but doing that would bypass all of the GO shortcut and custom search result page work that we’d done, as well as leave out results from the Directory and Course Catalog.

Instead, I found in the GSA documentation that you can make a request to the service and have it return an XML document of results, using this URL for our search engine:

http://search.middlebury.edu/cluster?q=SEARCH_QUERY&site=SEARCH_COLLECTION&coutput=xml&btnG=Google+Search&access=p&entqr=0&ud=1&sort=date%3AD%3AL%3Ad1&output=xml_no_dtd&oe=UTF-8&ie=UTF-8&client=default_frontend&%20proxystylesheet=default_frontend

For this to work, you need to replace SEARCH_QUERY with your search and SEARCH_COLLECTION with one of the collections that we maintain to segment search results. For example, there is a search collection named “Middlebury” that has all of our sites, but also one named “Blogs” that has only pages on our WordPress instance. Here is an example of what is returned by a search for “Google” on our Blogs server.

I then parse these results and display them on the search results page. Not wanting to get rid of work that was already there, and because I know we will want to switch back to using Google.com for our primary search results once the issue of 404 pages has been resolved, I added a tabbed interface on the search results page that lets you alternate between the two search collections. Just click on the tabs to see results from the other service.

Asynchronous Results

The search results page was one of the slowest loading pages on our site, taking between 6-15 seconds to load. The reason for this is that it needs to make requests to a lot of services to get all of the information:

1. Check to see if there are custom search pages
2. Check to see if there is a GO shortcut
3. Get the results from Google.com
4. Get the results from the GSA
5. Get the results from the Directory
6. Get the results from the Course Catalog

Steps 1 & 2 still happen before the page loads, since we might need to redirect you based on their results, but steps 3-6 now happen after the page loads. While you’re viewing the page, we’re requesting results from all of those services in the background. When the results come in, the page displays them using JavaScript. You might see the results from the GSA immediately and then results from the Course Catalog a second or two later. This gives the illusion of the page loading faster than it actually is and, if the thing you’re looking for appears early on, allows you to skip loading results from services you don’t need.

Upcoming Improvements

Improving search has been our secondary focus (after site performance) since launching our new site and a very important part of our work. We really want to get this right, so we’ll be adding in more and more of these types of improvements around the search results as time goes on. We’ll next be looking at statistics on how well our strategy of using GO shortcuts to deliver people directly to result pages works based on click patterns and, once we solve the 404 issue, how well Google.com does at proving basic search for our site.

Faceted Search

The next area of work is to figure out segmented search. We have a number of collections of highly structured content like HR Job Descriptions or Undergraduate Research Opportunities. We want to be able to build search interfaces for these collections so that people can search for, say, all of the jobs on campus that have a job level of Specialist 3 or all of the Research Opportunities in East Asia.

To do this, we’re setting up a local copy of the Apache Solr search engine. There is a Drupal module for this search engine that allows it to build filters based on content types. Job descriptions and research opportunities are content types and each of their fields could then be used as a filter in the faceted search results. I’m still in the preliminary stages of setting up this service, but am hoping to have a rough prototype done in April of how this will work.

Search My Site

Another use for the Apache Solr search engine would be to provide URL filtering for search results. We can do this by setting up collections in the GSA, but we don’t necessarily want to create a collection for every sub-site or maintain all of those filtering rules. Instead, we want to use Apache Solr’s flexible query syntax to let us find documents whose URL paths match patterns like “http://www.middlebury.edu/academics/lib” by passing that as a parameter to the search engine that we can alter if needed. This will also help us to add search to areas of the site like news archives.

Excluding GO Addresses from Search

There are some times when a custom search page is not appropriate for a search term and we don’t want to go directly to the URL for the GO shortcut when you search for that item. For example, Adam had set up “go/adam” to go to his personal blog, but there are a lot of people at Middlebury named Adam and people might be looking for a different Adam. We’ll add an option in the GO administration interface to allow you to exclude a GO shortcut from being used in search results, if you don’t want it to be.

User Preferred Search Engine

We’ve already got results from Google.com and the GSA and are adding the Apache Solr search engine to our site. Why just get results from one of these? Why not all of them? Why not Yahoo and Bing as well? Why not let people pick which one they want to use. We’ll add a field to the user profile pages in Drupal so that you can set your own preferred search engine and have it provide the default results for all your searches. This will also help us to look at which search tools you prefer and gauge which are giving better results.

What else?

Do you have other ideas for things we could do? Is there something that I glossed over that you have more questions about? Please let us know by adding your comments to this post.

What is MiddLab?

MiddLab will be a new section of our website that helps push information about scholarly and service work up to the top. We know that there are a lot of great academic resources built by people at Middlebury and many ongoing projects and activities that not everyone hears about or gets to see. The purpose of MiddLab is to:

• Tell people about this great work.
• See how it all ties together.
• Let people contribute to the discussion.

What do you want to know?

We’re very interested in hearing about the project work you’re doing this semester. This includes research projects, service learning projects, student organization work, entrepreneurial ventures, presentations, conferences, seminars and symposiums. This can be work done for any Middlebury or MIIS program or an independent project conducted by a member of one of those institutions.

Because we want to tell people about this work on the web, the project should produce some type of artifact that can be shown on the web: papers, posters, slideshows, videos, a website, a database, a blog. Don’t worry if your project doesn’t have many of these. We’re happy to do the work of converting what you have for the web.

If you don’t have a full project, but do maintain an existing online database or resource, let us know about that too. We’ll aggregate the information into MiddLab to increase the use and visibility or your resource.

What do I need to do?

We don’t want to add to your work, especially in the middle of a semester. To get involved, just submit the online form, or send an email to middlab@middlebury.edu. We’ll need to meet to discuss your project a couple times and you’ll need to help me a bit in handing off materials and checking the final product. More involvement is welcome, but we want this to be available to everyone, from the professor who lives online to the student who thinks a whiteboard is a bit too much new technology for the classroom.

What’s the timeline?

We’d like to hear about your projects this month and get together a couple times during March and April with the intention of having most of the information about the project compiled by the end of the semester. Over the summer, we will work on MiddLab to build out the site with all of the projects collected in this initial phase and host an “open beta” of the site where you can see it coming together. During the next fall semester, we hope to have MiddLab fully operational and allow direct submission and build-out of projects through the site, while continuing this organic process as well.

Thanks for your attention. We’re looking forward to hearing about your work.

This situation echos previous instances when content owners have threatened our institutions with litigation for infringement, for example the various institutions whom the American Association of Publishers approached regarding e-reserves and course management systems a few years ago. It differs from the peer to peer aspect of copyright significantly because, apart now from HEOA compliance, our colleges and universities did not have liability as conduit service providers, i.e. the allegedly infringing material was not being served from our servers, we acted only as I.S.P.s. Thus, this current matter is serious. Not only does it threaten exorbitant legal expenses and damages in both dollars and reputation, by touching instruction it threatens the exercise of our missions.

Steve Worona, on EDUCAUSE’s behalf, has begun a blog to educate people about this matter and stimulate discussion in the community. http://www.educause.edu/blog/sworona/UCLAVideoStreamingDamnedDammed/197444

Please take a moment to learn more about this matter, as we are learning about it, and most especially talk with your colleagues at home and around the community. It may be that higher education must approach the issue from a range of positions (standing firm on fair use, understanding better the opportunities and limitations of the Teach Act, proactively and collectively arranging for licensing are some examples that jump quickly to mind) but what is absolutely critical is that we do so as a community, working to help each other to preserve our missions.

Implementation (Dec. 10 to 20): Which Federal agencies are good
candidates to adopt Public Access policies? What variables (field of
science, proportion of research funded by public or private entities,
etc.) should affect how public access is implemented at various
agencies, including the maximum length of time between publication and
public release?

Features and Technology (Dec. 21 to Dec 31): In what format should the
data be submitted in order to make it easy to search and retrieve
information, and to make it easy for others to link to it? Are there
existing digital standards for archiving and interoperability to
maximize public benefit? How are these anticipated to change.

Management (Jan. 1 to Jan. 7): What are the best mechanisms to ensure
compliance? What would be the best metrics of success? What are the
best examples of usability in the private sector (both domestic and
international)? Should those who access papers be given the
opportunity to comment or provide feedback?

Please do comment at the OSTP site (you’ll need to register first).

http://blog.ostp.gov/wp-login.php?action=register

“In the next 10 years, the way people interact with computers will wildly change…”

http://edition.cnn.com/2009/TECH/12/01/craig.mundie.microsoft/index.html

Drupal

We selected Drupal as the platform for these sites because of its reputation of having a solid foundation, allowing large flexibility through modules, and having a huge developer and user community. There were also members of our team who had worked with Drupal on other site projects and could bring their experience with the platform to bear here. The fact that it’s free certainly didn’t hurt either!

But in looking at the features and functionality of Drupal, we encountered a problem. There are several ways to build out a site hierarchy and assign user permissions at each level of the hierarchy, but they don’t scale particularly well. Perhaps our biggest frustration with our current CMS, as site administrators, is how much time we devote to setting editing permissions. We wanted to simplify this task and put it in the hands of our content managers.

Monster Menus

Amherst College noted this problem as well when they moved their website into Drupal several years ago. Fortunately for us, they developed a solution named Monster Menus which adds both a way to build a site hierarchy and to assign permissions at each level. We’re hugely in debt to their work on this feature, and to the assistance of Dan Wilga who helped get us set up with a version of it.

The screencast linked to above does a much better job of explaining what Monster Menus is and how it works than I could hope to do in this space. In addition to the permissions features demonstrated there, Monster Menus and its related modules offer several other key features when dealing with a website where you know you’ll have thousands of pages and potentially hundreds of content editors:

• Lets you define the node types (think: page types like FAQ, Profile, News Article) that are available at any level of the site. This helps guide the content editors so that News stories aren’t showing up in the Course Catalog.
• Defines a content treatment for “sidebars” that automatically cascade down your site hierarchy. This way the Giving site can have a sidebar button to the Give Now form, create it on their site homepage and have it display on the right side of any giving page.
• A way to do specific scoped searches of content across your site.
• A content type for displaying an automatic list of sub-pages of any page on your site.
• Through the Media module, a file browser that lets you easily upload and include attachments, organizing them within your site hierarchy.
• A content type to display RSS feeds from any source, including feeds from pages or tags within the site.

Amherst also developed a full profile management system, named Amherst Profile, that is connected to their administrative database system and offers a sort of portal for members of their community. At this time, we decided not to implement this feature for our sites both due to time constraints and because we’re answering different questions with our sites than they answered using Amherst Profile.

Again, I want to thank Amherst for sharing this module with us. I don’t know how we would have been able to implement all the features it offers within our project timeline and have the interface come out as clean and easy to use as it is in Monster Menus. We’ll be working with the development staff at Amherst to do continuous improvements to this module, though much of that work will need to wait until after the Middlebury site launch.

The MIIS Home Page

I want to take you through each of the page designs for the MIIS site and explain how they work. This begins with the Home Page. This page has three main content areas: Be the Solution, News & Events, and the MIIS Spotlight. Be the Solution is a list of terms from a custom defined taxonomy. I used a function to fetch the taxonomy dynamically so that any additional terms will be immediately reflected on the home page. The links to the term pages are then based on the term names. MIIS can choose to grow or reduce the themes their institution highlights on the main page. During site development, their Web Strategy Team came up with an initial list of 17 themes (which they considered incomplete), then reduce the number to 5, and currently display 12.

The News & Events and MIIS Spotlight features are two places where we used the Views module for Drupal to display site content. MIIS Spotlight shows a block view that selects randomly from the story archive any news item specified as an MIIS Spotlight. I used a separate field on the News Story content type to allow editors to select whether their node was a Spotlight or not. This could also have been done as a separate content type, but that would have been a duplicate of the News Story content type and less efficient to maintain. It could also have been handled using a custom taxonomy, though the MIIS site already uses several of these and the editing form for News Stories is quite long as a result.

The other areas of the page, such as the list of languages, links to social networks, address in the footer, links below each of the features (such as “Expore academics as MIIS”), and quick links in the footer were all set using Advanced theme settings. These allow site administrators, or others with the Administer Themes permission, to make updates to these areas through the site administration interface without having to contact a developer to update the page template. The user interface for doing this isn’t particularly user friendly, but the settings need to be changed rarely enough, and by people familiar enough with the site to figure this out, that it’s much less painful than the alternatives. Some of these, such as the Quick Links, could have been done as a custom block in much the same way, but for others like the News Feed URL it was more easier done with theme settings and so I decided to keep both settings in the same interface.

Theme, or “Be the Solution” Pages

Examples: Business, Environment, Translations & Interpretation

We originally called this the “Theme Template”, but as you can imagine this got really confusing when we starting throwing around the word Theme in its normal web development context. The thinking on each of these pages was that they would be an aggregator of content related to each theme. Since we used a custom taxonomy to build the list of themes, we could tag various nodes in the site with each of the terms and they would appear on these pages in some sort of custom defined columns.

The problem was that we didn’t have a good way to get those columns. Some of them, like Degrees & Programs, indicated that the content nodes would be in the Academics portion of the site, probably nodes with the Page content type, which we would need to distinguish from nodes in the Careers section of the site using the same content type, tagged with the same term. At the time, Monster Menus didn’t have a way to select nodes tagged with a term in a particular portion of the site tree, so we worked with developers at Amherst to add that feature. I’m hoping later to add functionality like this to the Monster Menus Views integration, so that a view can be passed an argument to select nodes within a portion of the site tree.

This worked well for a time; MIIS editors could enter paths to areas of the site, we would fetch that page’s title and use it as the column title, then select a certain number of nodes from that area of the site tagged with the term for that theme. This works really well if you’re selecting news items, or a photo stream, or other content this is refreshed periodically because the theme page will always show the latest nodes tagged with that term. It works really poorly when used to select a list of degree programs where the institution needs to place the items in a defined order. To solve this, I split out the Degrees & Programs column into its own field using the Link module for CCK. We can now enter the links to degrees in whatever order is required and then specify any number of additional dynamic columns.

More to Come

This is getting to be quite lengthy and there’s a lot to discuss about the MIIS site. I’ll end the post here for now and make this a series of posts on the topic. Check back shortly (or subscribe to this blog’s RSS feed!) for a new post on the section pages, course and faculty views, and our struggle to come up with the perfect sub-page list bucketing algorithm.

In the mean time, if you have any questions about the MIIS site and its technical implementation, leave a comment.