Oblivion / Xanax are both down as of 2 minutes ago (2:52PM CST) due to a customer distribution switch crash. Techs are in route to correct the situation.

UPDATE: 3:00PM CST

Switch is now back online – all services are back up.

Thanks

Reminder that our website: www.serverorigin.com and the billing system will be down due to a maintenance on the server file cluster it is on. Please continue to submit any support tickets via: support@serverorigin.com

We expect the downtime to be completed in 6-8 hours.

Thanks

Reminder to: http://status.serverorigin.com/2007/11/20/wwwserverorigincom-company-website-maintenance/

Currently our datacenter is performing maintenance which may affect users in Chicago Datacenter. We should be back up within 15 minutes.

– Update:

We took a 90 Second outage with the maintenance. We are now back online in Chicago.

Today at around 1PM PST there appear to have been some power issues in downtown Los Angeles. Several carriers noted connectivity loss. Unfortunately our upstream also appears to be affected. In light of all the recent issues we’ve been having with WVFiber we currently have pending contracts with Hurricane Electric, Savvis, Telia, and Cogent.

These extended outages are completely unacceptable.

Everything is now back up. There was a bad fiber cross connect that brought down the entire Awknet datacenter. CRG techs at the datacenter have replaced the cross connect and we are doing some loop wiring to ensure this issue does not occur again in the future.

Thanks for your patience.

At approximately 6:40PM PST our fiber link to WVFiber went down momentarily, returning at 6:42PM PST. A ticket was submitted requesting information regarding the outage. A few moments later, at 6:45PM PST the port again went offline. Tickets were opened with WVFiber, CRG West, and our own techs quickly went to confirm our routers and fiber were functioning correctly.

A rather disturbing and alarming discovery was that neither WVFiber nor CRG (datacenter where our fiber demarks) had any onsite techs available to diagnose the problems until morning. Awknet attempted to get backup connectivity via Cogent, Mzima, and Linkline – however due to company policies none of these carriers were able to provision us service within 48 hours.

WVFiber has now dispatched a contractor that works on their behalf in the LA area to head into the datacenter and confirm if the issue is with the router  or the CGR crossconnect itself. If it is infact a router port issue we will be online shortly after moving to a new port. Sadly, if it is infact a fiber issue we will be forced to wait for CRG techs to arrive and provision a new crossconnect.

I’ll keep you as updated on the progress as possible.

What a night! Problems at 2 different datacenters, at exactly the same time. We are trying to get it fixed now… It seems:

shells(SharkTech – Chicago) + oblivion(AwkNet – Los Angeles) both took an outage tonight.

Shells was brought down for a bad stick of memory that was replaced and the box is now functional. The entire AwkNet datacenter went down tonight due to a fiber issue with the exchange. Teams are working to get it back up and bring it up temporarily on Cogent so that customers are back online until WV is fully repaired.

Due to the reboot on: shells.serverorigin.com (208.98.1.185) you will need to login and restart your processes.
We are terribly sorry due to the inconvenience this has caused and we are working to improve the situation as we speak. Everyone should be back online very soon and we thank you for your patience.

Thanks,

Kevin Hatfield
ServerOrigin.Com

Box rebooted for memory stick replacement that has gone bad. Sorry for any inconvenience. This seems to be an isolated issue with the server or its connection. This is not affecting other servers in the Sharktech datacenter. The Awknet datacenter has been affected by a fiber outage at WV. They are working to bring us up on Cogent until the WV link is repaired.

Kernel Update on SharkTech Servers. Quick reboot is necessary should be right back up.

I’m sure you are aware of some network problems occurring during the past 24 hours. Our core-switch which has been in duty for more than 3 years now suddenly crashed. This system has been in production with 0 downtime and an uptime of one year.

We attempted troubleshooting the problem and pinpointing the exact defect on the system. During that time we offloaded 60% of our network directly to our 2nd router. This is why during the last 18 hours only 40% of our network was affected. Today we were able to find out the problem with the switch and it’s the gigabit uplink module.

A new gigabit uplink module been purchased and should be arriving on Tuesday or Wednesday. During that time we currently created a new network path so all the users on the Cisco core-switch would be back online.

There will be a short downtime on Tuesday or Wednesday of no more than 15 minutes we’ll update you when we have more information.

I sincerely apologize for this inconvenience and hope you understand.