At 3:07pm (ET) Google made us aware they they were investigating issues with Google Drive and related services like Docs, Sheets, and Slides. These issues affected all Google Apps customers, not just Middlebury. As of 5:37pm (ET), Google Apps services appear to be back up.
Google’s official Apps status updates can always be found here: http://www.google.com/appsstatus#hl=en&v=status
This post has been updated because the issues have been resolved. Thanks for your patience!
Please note that there was an unexpected disruption to Middlebury’s incoming email services this morning. The issue has since been resolved. Please see below for more details.
Between 8:30 AM and 10:00 AM (ET), Middlebury’s email system was unable to receive incoming email sent from external addresses. Email messages sent to @middlebury.edu and @miis.edu email addresses from external addresses were rejected by Middlebury’s email servers. External senders would have received a “Non-Deliverable Notice” notifying them of the problem. Internal email messages, i.e. messages sent from @middlebury.edu or @miis.edu addresses, were not impacted.
The problem was corrected at approximately 10:00 AM (ET). External senders should be advised to resend messages that were rejected before the problem was resolved.
The site-to-site connection between Middlebury campuses and the Ellucian data center has been restored.
All Ellucian hosted systems are UP and available for your use including: all instance of Banner INB, Banner SSB, Hyperion EPM Reporting and Degree Works.
Tuesday, September 15th
We are currently experiencing a disruption of the site-to-site connection between the Middlebury campuses and the Ellucian data center where our Ellucian Banner, Hyperion Reporting and Degree Works systems are hosted. This disruption began at approximately 12:38pm ET.
The disruption appears to be on the Ellucian side of the network connection. We are currently in contact with the Ellucian Emergency Operations Center (EOC) and they have mobilized their response staff to address this issue.
At this time the following systems are DOWN: Internet Native Banner (INB), Hyperion EPM 11 Reporting and Degree Works.
The Self-Service Banner (SSB) is UP as this connection works across the public internet.
We regret any inconvenience this service interruption is having on your day and will have this issue resolved as soon as possible. I will send out another message once all services are restored.
The campus voice mail system is indicating multiple performance issues at present. This includes choppy/clipped speech on new messages and new greetings recorded since August 21. Currently, the system is not accessible. Our vendor is working on this.
Middlebury College is retiring the Bombay print server on August 3, 2015, as it is incompatible with our current 64-bit operating systems. Bombay has been replaced by a new print server named Walnut that has all of the same print queues on it.
ITS has worked with the majority of people who were connecting to the old server but if you have been away from campus you may experience an interruption in printing services until you move from Bombay to Walnut printers. If needed, refer to the steps below to restore printing. If you have any questions or need further assistance, please contact the Helpdesk at 802.443.2200 or via email to firstname.lastname@example.org.
The VPN connection between Middlebury and Monterey is seeing intermittent issues, dropping in and out. It’s being actively worked both here in Middlebury and in Monterey. This will impact all connections between the two campuses, including email and many Middlebury campus based services.
We will update this post when service has been restored.
UPDATE 4:00 PM:
The VPN has been stable since the last message. There is still a concern with the backup systems, but the active VPN is in good health.
There could be one more brief interruption (~2 minutes) before the end of the business day today. We’ll be working with the vendor over the next day or so to find root cause and bring us back to full redundancy.
~Zach Schuetz on behalf of ITS
Update: We have found the cause and fixed the issue that was casing the outages.
We have received some reports of brief network outages across campus. We are working to find the cause and correct it. Not everyone is affected, but saving your work is a good precaution. We will let you know when the problem is fixed.
Thank you for your patience.