Today at around 1PM PST there appear to have been some power issues in downtown Los Angeles. Several carriers noted connectivity loss. Unfortunately our upstream also appears to be affected. In light of all the recent issues we’ve been having with WVFiber we currently have pending contracts with Hurricane Electric, Savvis, Telia, and Cogent.

These extended outages are completely unacceptable.

It seems we are having some network connectivity issues in the Awknet Datacenter. We are working to get this resolved. I should have an update soon.

Network seems to have gone offline at: 3:14 PM CST

Everything is now back up. There was a bad fiber cross connect that brought down the entire Awknet datacenter. CRG techs at the datacenter have replaced the cross connect and we are doing some loop wiring to ensure this issue does not occur again in the future.

Thanks for your patience.

At approximately 6:40PM PST our fiber link to WVFiber went down momentarily, returning at 6:42PM PST. A ticket was submitted requesting information regarding the outage. A few moments later, at 6:45PM PST the port again went offline. Tickets were opened with WVFiber, CRG West, and our own techs quickly went to confirm our routers and fiber were functioning correctly.

A rather disturbing and alarming discovery was that neither WVFiber nor CRG (datacenter where our fiber demarks) had any onsite techs available to diagnose the problems until morning. Awknet attempted to get backup connectivity via Cogent, Mzima, and Linkline – however due to company policies none of these carriers were able to provision us service within 48 hours.

WVFiber has now dispatched a contractor that works on their behalf in the LA area to head into the datacenter and confirm if the issue is with the router  or the CGR crossconnect itself. If it is infact a router port issue we will be online shortly after moving to a new port. Sadly, if it is infact a fiber issue we will be forced to wait for CRG techs to arrive and provision a new crossconnect.

I’ll keep you as updated on the progress as possible.

What a night! Problems at 2 different datacenters, at exactly the same time. We are trying to get it fixed now… It seems:

shells(SharkTech – Chicago) + oblivion(AwkNet – Los Angeles) both took an outage tonight.

Shells was brought down for a bad stick of memory that was replaced and the box is now functional. The entire AwkNet datacenter went down tonight due to a fiber issue with the exchange. Teams are working to get it back up and bring it up temporarily on Cogent so that customers are back online until WV is fully repaired.

Due to the reboot on: shells.serverorigin.com (208.98.1.185) you will need to login and restart your processes.
We are terribly sorry due to the inconvenience this has caused and we are working to improve the situation as we speak. Everyone should be back online very soon and we thank you for your patience.

Thanks,

Kevin Hatfield
ServerOrigin.Com

Box rebooted for memory stick replacement that has gone bad. Sorry for any inconvenience. This seems to be an isolated issue with the server or its connection. This is not affecting other servers in the Sharktech datacenter. The Awknet datacenter has been affected by a fiber outage at WV. They are working to bring us up on Cogent until the WV link is repaired.