Update: 6/25/2013, 2:00 PM EDT:
As a result of the change required to isolate the failing component of our email infrastructure late last week, we have discovered that legitimate undeliverable bounce back messages have not been forwarded on to the original senders at Middlebury.edu and MIIS.edu but rather remained queued up. We have made the necessary configuration correction and about 300 legitimately undeliverable messages have now been correctly delivered. You may have received several messages all at once and wondered if there was a problem sending email off-campus generally, there is not, these are legitimate undeliverable bounce messages that have been accumulating for the last several days and now have been correctly forwarded on to you. Our sincere apologies for the delay in the delivery of these messages.
Update as of 6:30 PM EDT:
Testing and monitoring continues to be positive and we are now very confident that the intermittent Outlook disconnect incident has been resolved. Thank you for your patience and support during this event.
Update as of 2:10 PM EDT:
At this time we believe we have identified and eliminated the component that was causing the intermittent disconnects from outlook clients to our email Exchange environment. We are monitoring the systems closely and hope that this incident may be behind us.
At this time we are experiencing intermittent and isolated disconnects to our email Exchange environment from Outlook clients. You may have noticed warning messages or other pop-ups from time to time indicating that your Exchange client is attempting to contact Microsoft Exchange.
We are diligently working to solve this issue. We are isolating each component of our Exchange infrastructure including load balancing appliances that front-end the Exchange infrastructure and eliminating each as the cause of the problem. This is a time consuming process as it can take several hours to place individual components into maintenance mode and isolate them as client sessions tend to hold on particular servers they are currently working with.
Please note that email messages are flowing and being stored normally. Additionally there have not be any issues with accessing Exchange via the web interface (Outlook web access available at https://mail.middlebury.edu), so if the disconnection errors or warnings you receive within your Outlook client are too disruptive, please consider using the web interface until we are able to completely isolate and resolve the issue.
We appreciate your patience and apologies for the inconvenience, we recognize how critical email is to your daily work activity and our full resources are focused on resolving this issue as quickly as possible.
This Sunday morning during our established maintenance window between 6 am and 10 am EDT we will be upgrading a number of essential switches in the path between our network core and our Internet service providers. During this period there will be as many 7 Internet connectivity outages each lasting between 5 and 20 minutes.
We will be also be upgrading our wireless network infrastructure this Sunday morning from approximately 7:30 until 9:00 am EDT. During this period each WAP on campus will reboot causing a few minutes service interruption for the area it serves – the WAPs will reboot across campus at different times as their supporting controllers are upgraded.
We appreciate your patience as this essential work is completed.
This Sunday morning during our maintenance window of 6 am until 10 am we will be upgrading a number of network switches that provide connectivity to buildings throughout the Middlebury campus. There will be about a 15 minute interruption of network access as each network switch is upgraded.
The affected locations are:
220 College St
30 College St
BiHall (this building will be completed before 7:30 am)
Public Safety (this building will be completed before 7:30 am)
This Sunday morning during our maintenance window between 6 am 10 am we will be upgrading network switches that provide network connectivity to the following locations on the Middlebury campus. There will be an approximately 15 minute interruption of network services at each location as each switch is upgraded. Thank you for your patience as we perform this essential proactive maintenance.
Bread Loaf (all)
Voter 1st floor
All Banner systems ( INB & SSB/web) are currently unavailable. We have Ellucian technicians working on the problem and will update you as we know more and when the systems are back and available for use.
Associate Dean & Director, Enterprise Applications
Library & Information Services, Middlebury College
The following essential network maintenance activity is planned for this Sunday morning that will result in extended service interruptions throughout the campus.
Between 6:30 and 7:15 am the network core’s components will be upgraded. While all of the components of the network core are redundant, not all parts of the network throughout the campus have a path back to both sides of the network core. As a result there will be a 20 – 30 minute network service interruption for the majority of buildings on campus.
Between 7:00 and 8:15 am critical network aggregation switches that connect campus buildings and server farms to the network core will be upgraded. This will result in a complete interruption of network connectivity of approximately 30 minutes throughout the campus. This includes access to the College’s primary web sites, Email, MiddFiles – all systems hosted at Middlebury.
Between 7:45 and 9:00 am our border gateway routers and other components that connect the network core with the Internet will be upgraded. We expect four 3-5 minute periods where Internet routing will be in flux and expect transient loss of Internet connectivity for the Middlebury campus. During these four 3-5 minute periods, the VPNs with MIIS and Ellucian may be interrupted briefly, this would impact reaching Banner SSB and INB.
These network upgrades are essential and must be completed in a timely manner. We appreciate your patience as we strive to keep our network operating optimally.
The Wiley Online Library is non-functional for us and for other libraries around the country. There has not been any official notification from Wiley, so I cannot predict when access will be restored. When I know more, I will add an update in a comment to this post.
We have received a message from Remote-Learner, our Moodle hosting company, that they will be doing some maintenance to improve the security of the service. Expect disruptions in service between 1-6am on Saturday, February 16th.
Dear Valued Client,
In order to install new security upgrades and additional security features, Remote-Learner’s IT technical support division has planned a five-hour maintenance. During the maintenance period internet connectivity may be briefly disrupted several times on February 16th from 1:00 am EST to 6:00 am EST.
The new upgrades and security features will provide additional hosting security for clients. Ultimately, we believe that you will be pleased with the security updates, and we ask in advance that you accept our apologies for any inconvenience you may experience. We welcome your feedback on our efforts to continue to provide quality customer service and we request that you direct any questions and concerns related to this scheduled maintenance to Remote-Learner’s support division.
The Remote-Learner Team