Tags » Web Application Development

 
 
 

New WordPress Themes Available

August 20, 2010 by
Categories: Midd Blogosphere

We’ve made several changes to the WordPress platform, known on-campus as sites.middlebury.edu. Hey! You’re there right now!

New Header

Soon, we will update the design of the header so that a small bar appears across every blog we host with some useful links. This design is modeled on blogger.com and its purpose is to foster a sense of community amongst the many blogs we host on the site. You can read more about this design in the original White Whale strategic recommendations document on the Web Redo blog, but here is the recommendation that led us to make this change:

Once some Midd-specific WordPress themes are created, Middlebury’s blogs should be linked together via a unifying header or title bar element of some kind. The bar across the top of most Blogger blogs is a good example; it doesn’t interfere with the branding or messaging of the blog itself, but provides quick and consistent links back to the Blogger homepage and other blogs. Once some Midd-specific WordPress themes are created, Middlebury’s blogs should be linked together via a unifying header or title bar element of some kind. The bar across the top of most Blogger blogs is a good example; it doesn’t interfere with the branding or messaging of the blogitself, but provides quick and consistent links back to the Blogger homepage and other blogs.

The logo at the top left will bring you to the home page of our blogging network. If you’d like posts from your blog to appear there, send an email to website@middlebury.edu and we’ll add you to the list.

New Themes

There are three new themes available on our blogging platform. These are based on designs we received from the people who put together our new site design. We put these together in a way that makes them each to set up. The catch is that there are very few configuration options for these themes. That means these are great for people who want to set up a blog quickly and aren’t interested in doing a lot of customization on the look-and-feel of their blog. Additionally, these themes do not work properly in Internet Explorer 6. As of today, only 3.06% of visitors to our blogs use this browser and we are going to recommend phasing out support.

For themes that offer you a massive variety of customization options, be sure to check out the many theme options Alex Chapin has created for our blogging network.

BLOGS DOT MIDDLEBURY Navy

The new Navy (as in blue) theme offers a straight-forward, even minimalist, design for your blog. There is no background image on this theme, which offers two columns for you to add widgets. The left column only appears on pages with more than one post: the blog home page, search results, and archives. If you are viewing a single post or page on this theme, the left sidebar will disappear, giving the post more space on the page.

BLOGS DOT MIDDLEBURY Pastoral

The Pastoral theme features an image of the Bread Loaf campus as its background. This theme uses the same two-column format as the Navy theme, with the left column only appearing when more than one post is being displayed. The big difference with this theme is that you can change the background image if you like (more on that later).

BLOGS DOT MIDDLEBURY Map

The Map theme uses a professionally done watercolor illustration of the campus as its default background. As with the “Pastoral” theme, you can change the background image if you like. The big difference with this theme is that the left column is on the left of the blog’s content. Because of this positioning, both columns appear on all views of the blog, even when viewing a single post. Use this theme if you really like columns!

MiddLab Blog Theme

We’ve also added a new theme that you can use for a research project that you would like us to feature in MiddLab. Remember to send your MiddLab project ideas to middlab@middlebury.edu and check out the site to discuss the ongoing research projects of your fellow faculty, staff and students.

Setting up one of these themes

To add one of these themes to your blog:

  1. Click the Log in link at the top right of the page and fill in your username and password.
  2. Click the Dashboard link at the top right of the page.
  3. In the Appearance box on the left, click the Themes link.
  4. Click Activate below the picture of the theme you want to use.
  5. In the Appearance box on the left, click the Widgets link.
  6. Drag the widgets you want to use from the boxes in the center to the Left Column or Right Column boxes on the right.
  7. You’re done!

Adding a custom background image

Middlebury’s status as a top school depends on offering the services our students require. Perhaps one day Middlebury will accept penguins as applicants and you’ll be asked to create a blog for the new Office of Penguin Services and you’ll realize that you need a background image that speaks to the students you’re helping. Our themes support this.

This can only be done on the Pastoral or Map themes.

  1. Click the Log in link at the top right of the page and fill in your username and password.
  2. Click the Dashboard link at the top right of the page.
  3. In the Appearance box on the left, click the Custom Header link.
  4. Click the Browse button, select the image you want to use and click OK.
  5. You’ll be asked to crop the image you chose. Select the part of the image to use as the background and click Crop Image.
  6. You’re done!
Tags: , , , , , , ,
Comments Off

More GO Info

June 28, 2010 by
Categories: Midd Blogosphere

Since we moved GO to its new home last week I’ve been busy fixing a number of bugs that have come up, as well as made a few improvements that I hope will be helpful.

Today’s big improvement is that the GOtionary now provides info pages for every shortcut.

The info page will tell you who created and who administers the shortcut so that you know who to contact to when a link is broken. As well, the info page will now serve as the landing page when trying to access a broken GO shortcut, rather than being presented with a blank screen.

Head to the GOtionary to check it out.

Tags: , , , ,
Comments Off

GO is moving to a new server

June 22, 2010 by
Categories: Midd Blogosphere

Over the past few years the GO shortcut and redirection application has become central to the college’s web infrastructure, allowing easy-to-remember permalinks that can be updated as resources are moved.

Tomorrow morning we will be migrating GO from a multi-use Windows server to its own RedHat server. The primary impetus for this move is to resolve a PHP-on-Windows memory leak bug that has taken out GO for several minutes every few months. In addition to this bug fix, migrating GO to its new environment allows a few additional improvements at this time:

  • GO will be on its own server, more isolated from interference from other applications
  • GO will now fail-over to a secondary database should its primary database become unavailable.
  • Improved user-information caching will dramatically speed up the self-service admin screens
  • Redirects will now be re-written internally, requiring one less round-trip to the GO application for every redirect.
  • go/shortcut should now work more reliably on the MIIS network without having to type the full go.miis.edu/shortcut URL in the address bar.

    Note: the full http://go.middlebury.edu/shortcut or http://go.miis.edu/shortcut URL should still be used when putting links in websites or email.

  • The GOtionary will now live under go.middlebury.edu and go.miis.edu, allowing go.miis.edu to have its own logo.

We do not anticipate that this migration process will result in any downtime as the new GO server and the old GO server will both continue to operate at the same time, against the same database. After we switch the DNS records for go.middlebury.edu and go.miis.edu users will slowly move over to the new GO server as their computers look up the address of go.middlebury.edu again. For on-campus users this may happen quickly, while for off-campus users it may take several weeks. After the vast majority of users are accessing the new GO server (likely two weeks or so), we will turn off the old GO server.

Update 1 – June 23rd

We successfully migrated go.middlebury.edu to the new host and haven’t had any problems. We’ll be waiting for a while for go.miis.edu to switch over.

Tags: , , , , , ,
Comments Off

WordPress Update

June 11, 2010 by
Categories: Midd Blogosphere

Earlier this week, we updated WordPress (the platform that powers sites.middlebury.edu).  Along with updating the WordPress codebase, we also updated all the Midd blog themes and a number of plugins.  There were many changes made to the backend of Midd blog themes to make them more flexible and easier to use.

Options have been added for what sidebars get displayed when viewing single post pages, as well as pages for categories, tags, authors and search.  Lots of small design refinements has been made to make make blogs easier to read.  For a full list of changes, see:

WordPress > Themes > Change Log

New blogs will now be created with the Translucence theme and a default set of sidebar widgets.  Documentation has been added for many of plugins, for example see: Geo-mashup

Tags: , , ,
Comments Off

Website Performance: Pressflow, Varnish, Oh-My!

May 17, 2010 by
Categories: Midd Blogosphere

Executive summary:

We’ve migrated from core Drupal-6 to Pressflow, a back-port of Drupal-7 performance features. Using Pressflow allows us to cache anonymous web-requests (about 77% of our traffic) for 5-minutes and return them right from memory. While this vastly improves the amount of traffic we can handle as well as the speed of anonymous page-loads it does mean that anonymous users may not see new versions of content for at most 5 minutes. Traffic for logged-in users will always continue to flow directly through to Drupal/Pressflow and will always be up-to-the-instant-fresh.

Read on for more details about what has change and where we are at with regard to website performance.

Background

When we first launched the new Drupal website back in February we went through some growing pains that necessitated code fixes (Round 1 and Round 2) as well as the addition of an extra web-server host and database changes (Round 2).

These improvements brought our site up to acceptable performance levels, but I was concerned that we might run into performance problems if the college ended up in the news and thousands of people suddenly went to view our site.

At DrupalCon a few weeks ago I attended a Drupal Performance Workshop where I learned a number of techniques that can be used to scale Drupal sites to be able to handle internet-scale traffic — not Facebook or Google-level traffic, but that of The Grammys, Economist, or World Bank.

Since before the launch of the new site we were already making use of optcode-caching via APC to speed code execution and were doing data caching with Memcache to reduce the load on the database. This system-architecture is far more performant than a baseline setup, but we still could only handle a sustained average of 20 requests each second before the web-host started becoming fully loaded. While this double our normal average of 10-requests per second, it is not nearly enough headroom to feel safe from traffic spikes.

Diagram of the execution flow through the web-host using normal Drupal page caching.

Request flow through our Drupal web-host prior to May 13th; using normal Drupal page-caching stored in Memcache. Click for full-size.

Switching to Pressflow

Last week we switched from the standard Drupal-6.16 to Pressflow-6.16.77, a version of Drupal 6 that has had a number of the performance-related improvements from Drupal-7 back-ported to it. Code changes in Pressflow such as dropping legacy PHP4 support and using only MySQL enable Pressflow execute about 27% faster than Drupal, a useful improvement but not enough to make a huge difference were we to get double or triple our normal traffic.

For us, the most important difference between Pressflow and Drupal-6 is that sessions are ‘lazily’ created. This means that rather than creating a new ’session’ on the server to hold user-specific information on the first page each user sees on the website, Pressflow instead only creates the session when the user hits a page (such as the login page) that actually has user-specific data to store. This change makes it very easy to differentiate between anonymous requests (no session cookies) and authenticated requests (that have session cookies) and enables the next change, Varnish page caching.

Varnish Page Caching

Varnish is a reverse-proxy server that runs on our web hosts and can return pages and images from its own in-memory cache so that they don’t have to execute in Drupal/Pressflow every single time. The default rule in Varnish is that if there are any cookies in the request, then the request is for a particular user and should be transparently passed through to the back-end (Drupal/Pressflow). If there are no cookies in the request, then Varnish assumes correctly that it is an anonymous request and tries to respond from its cache without bothering the back-end.

Request flow through our Drupal/Pressflow web-host after May 13th; using the Varnish proxy-server for caching. Click for full-size.

Request flow through our Drupal/Pressflow web-host after May 13th; using the Varnish proxy-server for caching. Click for full-size.

Since about 77% of our traffic is non-authenticated traffic, Varnish only sends about 30% of the total requests through to Apache/PHP/Drupal: all authenticated requests and anonymous requests where the cache hasn’t been refreshed in the past 5 minutes. Were we to have a large spike in anonymous traffic, virtually all of this increase would be served directly from Varnish’s cache, preventing any load-increase on Apache/PHP/Drupal or the back-end MySQL database. In my tests against our home-page varnish was able to easily handle more than 10,000 requests each second with the limiting factor being network speed rather than Varnish.

A histogram of requests to the website. Y-axis is the number of requests, X-axis is the time to return requests, '|' requests were handled by Varnish's cache and '#' were passed through to Drupal. The majority of our requests are being handled quickly by Varnish while a smaller portion are being passed-through to Drupal.

A histogram of requests to the website. Y-axis is the number of requests, X-axis is the time to return requests, '|' requests were handled by Varnish's cache and '#' were passed through to Drupal. The majority of our requests are being handled quickly by Varnish while a smaller portion are being passed-through to Drupal.

MySQL Improvements

During the scheduled downtime this past Sunday, Mark updated our MySQL server and installed the InnoBase InnoDB Plugin, a high-performance storage engine for MySQL that can provide twice the performance of the built-in InnoDB engine in MySQL for the types of queries done by Drupal.

Last week Mark and I also went through our database configuration and verified that the important parameters were tuned correctly.

As the MySQL database is not currently the bottleneck that limits our site performance these improvements will likely have a minor (though wide-spread) effect. Were our authenticated traffic to further increase (due to more people editing for instance) these improvements will be more important.

Where We Are Now

At this point the website should be able to handle at least 20,000 requests/second of anonymous users (10,000 on each of two web-hosts) at the same time that it is handling up to 40 requests/second from authenticated users (20 on each of two web-hosts).

While it is impossible to accurately translate these request rates into the number of users we can support visiting the site, a very rough estimation would be to divide the number of requests/second by 10 (a guess at the average number of requests needed for each page view) to get a number of page-views that can be handled each second. (1)

In addition to how many requests can be handled, how fast the requests are returned is also important. Our current response times for un-cached pages usually falls between 0.5 seconds and 2 seconds. If pages take much longer than 2 seconds, the site can “feel slow”. For anonymous pages cached in Varnish response times range from 0.001 seconds to 0.07 seconds, much faster than Apache/Drupal can do and more than fast enough for anything we need.

The last performance metric that we are concerned with is about the time it takes for the page to be usable by the viewer. Even if they receive all of the files for a page in only 0.02 seconds, it may still take their browser several seconds to parse these files, execute javascript code, and turn them into a displayable graphic. Due to these factors, my testing has shown that most pages on our site take between 1 and 3 seconds for users to feel that our pages are loaded. For authenticated users, this stretches to 2-4 seconds.

Finally please be aware that, anonymous users see pages that may be cached for up to 5 minutes. While this is fine for the vast majority of our content, there are a few cases where we may need to have the content shown be up-to-the-second fresh. We will address these few special cases over the coming months.

Future Performance Directions

Now that we have our caching system in place our system architecture is relatively complete for our current performance needs. While we may do a bit of tuning on various server parameters, our focus now shifts to PHP and Javascript code optimization to further improve server-side and client-side performance respectively.

One big impact on javascript performance (and hence perceived load-time) is that we currently have to include two separate versions of the jQuery Javascript Library due to different parts of the site relying on different versions. Phasing out the older version will reduce almost by half the amount of code that the browser has to parse.

Additional Notes

(1) As people browse the site their browser needs to load the main HTML page as well as make separate requests for Javascript files, style-sheet (CSS) files, and every image. After these have been loaded the first time, [most] browsers will cache these files locally and only request them again after 5 minutes or if the user clears their browser cache. CSS files and images that haven’t been seen before will need to be loaded as new pages are browsed to. For example, the first time someone loads the Athletics page, it requires about 40 requests to the server for a variety of files. A subsequent click on the Arts page would require an additional 13 requests, while a click back to the Athletics page would require on 1 additional request as the images would still be cached in the browser.

Tags: , , , , , , , , ,
Comments Off

MiddLab

May 12, 2010 by
Categories: Midd Blogosphere

http://go.middlebury.edu/middlab

MiddLab is a new section of Middlebury’s website with no precedent: an academic network, uniting all of the… blah, blah blah.

Truth is, MiddLab has been hard for us to explain ever since we heard the idea. A research network featuring discussions and blogs, and linking together disciplinary themes? How does that work? Rather than write a manifesto, here is what we’re trying to accomplish with MiddLab.

Our Goals

  • Make research easy to discover. If you want to know what student and faculty research is going on in a department, you shouldn’t have to know where their papers are published or the address of the project’s web site. Instead, these should be one or two clicks from our home page.
  • Show connections between research. Whether researching the population growth of trees in Biology or the population density of people in Geography, projects share themes and people interested in the topic can easily explore both.
  • Start a discussion. We encourage and recommend that you add comments to the projects on this site. Ask questions, suggest new research, or explain why you disagree with the conclusions. You can add your thoughts to any project page on MiddLab, explore the individual blogs for some projects, or contact the researchers directly.
  • Provide space for research and the sciences on our site. We’ll be expanding this site to feature more presentations from the Spring Research Symposium and research projects in our science departments. Though MiddLab is open to any student, faculty or staff projects, these are areas where we know we’re not offering enough information on our site and would like to use MiddLab to expand.

Your Feedback

We aren’t sure these are the right goals for our site. We’d like to hear from people: what would you like to see in MiddLab? What parts of this site work toward these goals and which don’t? Leave your thoughts by commenting on this page.

Oh, and if you would like us to feature your project in MiddLab, send an email to middlab@middlebury.edu.

Tags: , , , , , , ,
Comments Off

DrupalCon 2010 Trip Report Day 1

April 20, 2010 by
Categories: Midd Blogosphere

Hello from San Francisco! I was waylaid in Chicago and missed the morning presentations, but I wanted to share what I’ve learned so far at DrupalCon. First, a quick bullet point summary for those who don’t want to dive into the details:

  • Drupal now powers over 1% of the total websites, closely tied with Joomla. Wordpress powers about 8.5%.
  • Drupal 7’s forms will allow us to add conditional form fields that appear for the user without requiring a postback to the server. See the (very relevent for us) example here: http://d7.drupalexamples.info/form_example/states
  • Drupal 7’s User Experience (UX) team has made improvements to the interface that on our site is called the “Edit Console”. You can read more about their project at their website: http://www.d7ux.org/content/
  • We can improve our site performance by moving functionality out of the template files and into theme functions. Basically, the way we currently do things, we have to read a file off the server’s disk every time anyone loads anything on the site. By using theme functions instead of template functions we avoid this disk read and dramatically improve performance.
  • You can watch many of today’s presentations at http://sf2010.drupal.org/conference/schedule for free! Many of those without video have their slides up. The presentations from Monday are at the bottom of the page since, at the time I’m posting this, they’ve already happened and aren’t as interesting to the conference attendees.
  • Monster Menus, the module the Amherst developed that lets you add sub-pages and manage permissions is a few weeks away from being refactored to eliminate any Amherst-dependent code. The version we’re currently running assumes that Amherst’s version of Banner exists, which we’ve had to work around. The new version will make this easy for us and open MM up for other schools to use.

All of the sessions I attended today focused on the improvements coming in Drupal 7. Currently, Drupal 7 is in “feature freeze” with 114 critical bugs left to resolve before it is released. At the keynote presentation today, Dries projected that Drupal 7 would likely be released some time between June and October of this year. Even so, and even with the large number of improvements it offers, we will not move to Drupal 7 when it is released. Our challenge is that the system we rely on to provide our site editors the ability to add sub-pages and manage permissions for their site is not part of “core” Drupal – it is provided by a module that is only used by us and Amherst College. I had an opportunity to speak with the developers from Amherst today and our projection is that, at best, we will be able to move to Drupal 7 for the start of Fall Semester 2011.

Even that timeline will be challenging, but I will provide a quick synopsis of each of the sessions I attended below, focusing on how they will impact our site if and when we make the move to Drupal 7.

The State of Drupal

http://sf2010.drupal.org/conference/sessions/state-drupal

The meat of this presentation was defining a framework for thinking about the future growth of Drupal. Right now the project is at a crossroads where it can continue to add features to satisfy the “enterprise” users of large organizations and compete with commercial CMSs like Microsoft Sharepoint, or it can aim to reduce the number of features and compete for the low end of the market with WordPress, aiming at people who just want a simple site with a few pages that is easy to manage. Dries seemed to believe that the Installation Profile system, of which there are now 19 releases, will allow Drupal to target specific low-end markets while core continues to add features to satisfy enterprise needs, but he seemed unsure of his own assertion that Drupal could do both.

This will be an interesting discussion for Middlebury as it continues in the Drupal community. We are one of those places that has an “enterprise” need that is not satisfied by Drupal core: the need to organize our site as a tree that allows our editors to add to that tree and manage fine-grained permissions in that tree. To that extent, Monster Menus is like our own installation profile of Drupal since we know that it imposes limitations on what Drupal can do since modules need to be changed to intergrate with it. We will have to see what features get added for Drupal 8 and how well they align with our needs.

The Rest

I had actually planned on diving into the details of the afternoon presentations I attended, but have run out of time before this morning’s round of sessions. Today, by the way, seems to be offer a lot more for the people currently running Drupal in production. Monday’s sessions were all about all the cool new features in Drupal 7, which is fun, but not something I’ll be working with for over a year. Today I’m attending sessions on search integration, search engine optimization, cloud computing integration and database optimization. These are topics closer to my day-to-day work.

Here are links to the sessions I attended on Monday. If you’re interested in hearing more, be sure to ask in the comments.

AJAX and JavaScript in Drupal 7

D7UX How to integrate the core Drupal 7 usability improvements with your module

Default theme implementations

Monster Menus

After the sessions, I had a good conversation with Dan and Victor from Amherst about the future direction of Monster Menus. Dan is close to being done with the “revamp” branch of Monster Menus that removes the Amherst-specific code from their system. We’ll want to convert to this and try it out when he’s done since there are some features of our implementation that don’t work right now because the module assumes that it will be able to talk to Amherst’s version of Banner on the backend.

Other interesting things from this meeting:

  • They are working on a way to move course sites (which are currently pages in their website – everything Amherst does on the web is in Drupal including their LMS) from one semester to the next while preserving associations like page permissions. This is tricky since you might assign permissions on a page in your course site to one student who won’t be in the course the next time it is taught.
  • We should be able to theme the RSS page content type without much trouble (this solves a request that I currently have with Communications to improve the way news items in the Newsroom are displayed), but we will probably never be able to theme the actual menu in the fashion that Drupal expects because of the processing overhead on generating that menu.
  • Amherst runs the Google Search Appliance to manage their search services. They allow the GSA to crawl their site as an administrative user and have a Drupal module that filters the search results based on the permissions of the currently logged in user. This is a requirement because their site also includes their LMS which they need to search with admin permissions. They are interested in seeing what we do with faceted search, an area they’ve wanted to look into but haven’t yet had the chance.

We had a lot more discussion about the minutiae of various modules and parts of Monster Menus, but those are the major points. I’ll post again this evening with a roundup of today’s sessions and on Wednesday after I meet with the guys from White Whale.

Tags: , , , , ,
Comments Off