Webmail outage affects 200 users

By Andrew Ta     9/6/12 7:00pm

As the school day wound down on Aug. 21, Steve Cox, a computational and applied mathematics professor, knew something had gone wrong. 

"A colleague said she had sent me two emails," Cox said. "I never received them. She didn't receive any of mine either."

Webmail remained down for about an hour, preventing many faculty, staff and graduate students from accessing their email. 

---

---

Director for Systems, Infrastructure, Architecture and Cloud Strategy Barry Ribbeck said service for a third of Webmail users had stopped when one of the three servers housing Webmail went down.

To mitigate the problem, Information Technology switched over to its secondary, off-site data center, restoring service to nearly all users, according to Ribbeck.

"In the process of analyzing what had occurred, we discovered roughly 200 users had lost emails from within a 20-day time frame," Ribbeck said. "We had successfully restored the emails of the 3,800 others who had felt the service error, but had some problems with a select few [users]."

Despite devoting a straight 35-hour stretch to restoring emails, IT was still in the process of restoration for 200 users as of Aug. 29, although they did manage to salvage the message headers of all emails.

"Affected users know some information about the emails that haven't been restored: who sent them, when and the topic of each email," Ribbeck said.

For future reference, one can always check https://my.rice.edu/ITAppStatus/ for status updates of all Rice IT systems. Ribbeck noted that Rice's uptime proved better than Google's (www.google.com/appsstatus) over the past five years. 

"In 2011, IT had a 99.9941 percent uptime," Ribbeck said. "We strive for 100, but randomness makes it hard."

---