08:04AM CDT:

Appears an issue with a power failure at the datacenter. Backup power failed to kick-off for the cabinet we’re awaiting an update from the engineer checking to get it back up. This issue is affecting a small number of users.

Updates will come shortly.

09:00AM CDT:
Issue was resolved. We apologize for the trouble.

Thanks!

–10:53 PM CDT

ethProxy customers in our Staminus datacenter have an outage due to our transport fiber being cut in LA. We’re awaiting an ETA, we’re also configuring new services out of an alternate datacenter to get customers online. We are working diligently to get an ETA of resolution. The entire Staminus datacenter as well as other datacenters in the area are impacted. Please open a ticket if you’d like to be configured on a different datacenter during the interim. We are hoping this will be resolved sooner than we’ll have the other server online. This issue was in no part due to ServerOrigin or something within our control. We are severely impacted and escalating along with many other datacenters to have this resolved as quickly as possible.

Thanks,

ServerOrigin.Com Management

————

-04:22AM CDT:
Datacenter issues have been resolved. Customers that were moved to the alternate location, we will allow 12 hours before we request the move back. This will ensure no issues.

Please let us know if you have any issues. Thanks so much for your patience throughout this issue.

Thanks,

ServerOrigin.Com

Netherlands VPS Customers:
Ecatel appears to be having a datacenter connectivity issue. We have reported and escalated, awaiting word from their support staff on when this issue will be resolved.

Thanks,

ServerOrigin Support

——————————————–

10/30/2009 – 02:53 AM CDT

Ecatel Update: Servers should be up in the next couple hours. We’re awaiting verification the issue is resolved. We will be applying credit for all users affected. Ecatel was showed horrible support in this matter. We will be purchasing new servers in the EvoSwitch datacenter and moving all customers once the servers are up. We apologize for the inconvenience and we thank everyone for your continued patience. This was an outage completely out of our control but this situation was made worse by the lack of support and updates from the datacenter. We provide better services than this for our customers and we will make every effort to be sure this doesn’t occur in the future.

Thanks,

ServerOrigin.Com

———————

10/30/2009 – 08:03 AM CDT

Ecatel issues resolved. Credit applied to all users impacted.

12:28AM CDT:

We are experiencing a few minute outage due to a POP issue in California that direct connects to one of our primary datacenters. This is affecting thousands of customers including our entire datacenter at Staminus.Net. We’re working to get this issue resolved asap. As stated, this is a huge outage and is potentially affecting multiple datacenters on the West coast. We’re awaiting word from the POP to provide us more details on the situation. Thank you for your patience as we resolve this issue. Our website is also affected by this outage, you can still contact support via: support@serverorigin.com. You can also login to our support system at: http://billing.serverorigin.com

Thanks,

Kevin H.

ServerOrigin.Com

—————

-01:35AM CDT:

It appears part of the Gigenet.Com datacenter is down as well during this outage period. Some servers are up while others are down. This appears to be just as wide-spread as we thought. Blogspot.com is appearing to be part of this outage as well, therefore we have word that Google is also taking large impact. We apologize as we’re still awaiting word on further updates. We are escalating but we’re one of thousands escalating. Hopefully this issue will be resolved soon.

-Kevin

————————–

02:03AM CDT

Dear Subscribers,

After the short outage we appear to be back online. This outage appears to have affected several other datacenters as well. We are awaiting verification of a resolution but at this time, servers appear to be back online and responsive. We thank you for your patience with this matter and we hope to have word that the outage is cleared shortly.

Thanks,

Kevin H.

Oblivion / Xanax are both down as of 2 minutes ago (2:52PM CST) due to a customer distribution switch crash. Techs are in route to correct the situation.

UPDATE: 3:00PM CST

Switch is now back online – all services are back up.

Thanks

Xanax users may have seen some degradation due to a DDOS attack that started about 4 minutes ago: 5:58PM CST. (12/10/07)

Issue has been corrected at: 6:03 PM CST.

Thanks

Today at around 1PM PST there appear to have been some power issues in downtown Los Angeles. Several carriers noted connectivity loss. Unfortunately our upstream also appears to be affected. In light of all the recent issues we’ve been having with WVFiber we currently have pending contracts with Hurricane Electric, Savvis, Telia, and Cogent.

These extended outages are completely unacceptable.

Everything is now back up. There was a bad fiber cross connect that brought down the entire Awknet datacenter. CRG techs at the datacenter have replaced the cross connect and we are doing some loop wiring to ensure this issue does not occur again in the future.

Thanks for your patience.

At approximately 6:40PM PST our fiber link to WVFiber went down momentarily, returning at 6:42PM PST. A ticket was submitted requesting information regarding the outage. A few moments later, at 6:45PM PST the port again went offline. Tickets were opened with WVFiber, CRG West, and our own techs quickly went to confirm our routers and fiber were functioning correctly.

A rather disturbing and alarming discovery was that neither WVFiber nor CRG (datacenter where our fiber demarks) had any onsite techs available to diagnose the problems until morning. Awknet attempted to get backup connectivity via Cogent, Mzima, and Linkline – however due to company policies none of these carriers were able to provision us service within 48 hours.

WVFiber has now dispatched a contractor that works on their behalf in the LA area to head into the datacenter and confirm if the issue is with the router  or the CGR crossconnect itself. If it is infact a router port issue we will be online shortly after moving to a new port. Sadly, if it is infact a fiber issue we will be forced to wait for CRG techs to arrive and provision a new crossconnect.

I’ll keep you as updated on the progress as possible.

What a night! Problems at 2 different datacenters, at exactly the same time. We are trying to get it fixed now… It seems:

shells(SharkTech – Chicago) + oblivion(AwkNet – Los Angeles) both took an outage tonight.

Shells was brought down for a bad stick of memory that was replaced and the box is now functional. The entire AwkNet datacenter went down tonight due to a fiber issue with the exchange. Teams are working to get it back up and bring it up temporarily on Cogent so that customers are back online until WV is fully repaired.

Due to the reboot on: shells.serverorigin.com (208.98.1.185) you will need to login and restart your processes.
We are terribly sorry due to the inconvenience this has caused and we are working to improve the situation as we speak. Everyone should be back online very soon and we thank you for your patience.

Thanks,

Kevin Hatfield
ServerOrigin.Com