Update: 6/25/2013, 2:00 PM EDT:
As a result of the change required to isolate the failing component of our email infrastructure late last week, we have discovered that legitimate undeliverable bounce back messages have not been forwarded on to the original senders at Middlebury.edu and MIIS.edu but rather remained queued up. We have made the necessary configuration correction and about 300 legitimately undeliverable messages have now been correctly delivered. You may have received several messages all at once and wondered if there was a problem sending email off-campus generally, there is not, these are legitimate undeliverable bounce messages that have been accumulating for the last several days and now have been correctly forwarded on to you. Our sincere apologies for the delay in the delivery of these messages.
Update as of 6:30 PM EDT:
Testing and monitoring continues to be positive and we are now very confident that the intermittent Outlook disconnect incident has been resolved. Thank you for your patience and support during this event.
Update as of 2:10 PM EDT:
At this time we believe we have identified and eliminated the component that was causing the intermittent disconnects from outlook clients to our email Exchange environment. We are monitoring the systems closely and hope that this incident may be behind us.
At this time we are experiencing intermittent and isolated disconnects to our email Exchange environment from Outlook clients. You may have noticed warning messages or other pop-ups from time to time indicating that your Exchange client is attempting to contact Microsoft Exchange.
We are diligently working to solve this issue. We are isolating each component of our Exchange infrastructure including load balancing appliances that front-end the Exchange infrastructure and eliminating each as the cause of the problem. This is a time consuming process as it can take several hours to place individual components into maintenance mode and isolate them as client sessions tend to hold on particular servers they are currently working with.
Please note that email messages are flowing and being stored normally. Additionally there have not be any issues with accessing Exchange via the web interface (Outlook web access available at https://mail.middlebury.edu), so if the disconnection errors or warnings you receive within your Outlook client are too disruptive, please consider using the web interface until we are able to completely isolate and resolve the issue.
We appreciate your patience and apologies for the inconvenience, we recognize how critical email is to your daily work activity and our full resources are focused on resolving this issue as quickly as possible.
This Sunday morning during our established maintenance window between 6 am and 10 am EDT we will be upgrading a number of essential switches in the path between our network core and our Internet service providers. During this period there will be as many 7 Internet connectivity outages each lasting between 5 and 20 minutes.
We will be also be upgrading our wireless network infrastructure this Sunday morning from approximately 7:30 until 9:00 am EDT. During this period each WAP on campus will reboot causing a few minutes service interruption for the area it serves – the WAPs will reboot across campus at different times as their supporting controllers are upgraded.
We appreciate your patience as this essential work is completed.
This Sunday morning during our maintenance window we will be performing the following activities.
(1) At approximately 8 am we will be upgrading our domain name servers (DNS) to support separate on campus and off campus views. While we do not expect any service impact, we are preforming this change during our maintenance window.
(2) At approximately 9 am we will be upgrading our database infrastructure behind our primary web sites (www.middlebury.edu, www.miis.edu). While we do not expect any service impact, we are preforming this change during our maintenance window.
Thanks for your patience as this essential work is completed.
This Sunday morning during our maintenance window we will be performing the following activities:
(1) Upgrade the Go Web Application server, we expect less than a 10 minute interruption of service at approximately 7:00 am.
(2) Upgrade Central Authentication Service (CAS), we do not expect a service impact, this activity will also take place at approximately 7:00 am.
Thanks for your patience as we strive to keep our systems operating optimally.
This Sunday morning during our maintenance window of 6 am until 10 am we will be upgrading a number of network switches that provide connectivity to buildings throughout the Middlebury campus. There will be about a 15 minute interruption of network access as each network switch is upgraded.
The affected locations are:
220 College St
30 College St
BiHall (this building will be completed before 7:30 am)
Public Safety (this building will be completed before 7:30 am)
This Sunday morning during our maintenance window between 6 am 10 am we will be upgrading network switches that provide network connectivity to the following locations on the Middlebury campus. There will be an approximately 15 minute interruption of network services at each location as each switch is upgraded. Thank you for your patience as we perform this essential proactive maintenance.
Bread Loaf (all)
Voter 1st floor
This Sunday morning between our maintenance window of 6 am to 10 am we will be performing the following network activity:
Between 6 am and 7 am we will be replacing networking equipment that serves Proctor. There will be less than a 30 minute network outage during this period to locations within Proctor including the book store, dining facilities and WRMC, this will include the radio transmission of WRMC.
Between 6:30 am and 8:00 am we will upgrade the network equipment that service the Biomass area within the Service Building. There will be a less than 20 minute network outage during this period within the Biomass facility, Locksmith and Custodial Services locations.
Between 7:00 and 8:30 am we will be checking fiber connections that terminate in one of our primary data centers, while we do not expect any network impact, it is possible there may be several less than 1 minute interruptions of network connectivity to many buildings on campus.
Thanks for your patience as we strive to keep our network operating optimally.