Everything is now back up. There was a bad fiber cross connect that brought down the entire Awknet datacenter. CRG techs at the datacenter have replaced the cross connect and we are doing some loop wiring to ensure this issue does not occur again in the future.

Thanks for your patience.

At approximately 6:40PM PST our fiber link to WVFiber went down momentarily, returning at 6:42PM PST. A ticket was submitted requesting information regarding the outage. A few moments later, at 6:45PM PST the port again went offline. Tickets were opened with WVFiber, CRG West, and our own techs quickly went to confirm our routers and fiber were functioning correctly.

A rather disturbing and alarming discovery was that neither WVFiber nor CRG (datacenter where our fiber demarks) had any onsite techs available to diagnose the problems until morning. Awknet attempted to get backup connectivity via Cogent, Mzima, and Linkline – however due to company policies none of these carriers were able to provision us service within 48 hours.

WVFiber has now dispatched a contractor that works on their behalf in the LA area to head into the datacenter and confirm if the issue is with the router  or the CGR crossconnect itself. If it is infact a router port issue we will be online shortly after moving to a new port. Sadly, if it is infact a fiber issue we will be forced to wait for CRG techs to arrive and provision a new crossconnect.

I’ll keep you as updated on the progress as possible.