Partial power outage Equinix AM5 (Status)

« Back

[#581] Partial power outage Equinix AM5 (Status)

Posted: 2018-01-02 01:30

Start: 2018-01-01 23:45:00
End : 2018-01-02 18:10:00

Affects: Power redundancy Equinix AM5

At 23.45 CET we got a notice from our monitoring system that one of the powerfeeds inside Equinix AM5 went down.

We've sent in a trouble ticket for Equinix AM5 staff to check and were notified Equinix had a power incident inside the datacenter.

The following was sent by Equinix staff:
INCIDENT SUMMARY: A power incident was detected on 01-JAN-2018 23:45 at AM5.

INCIDENT DESCRIPTION:

Equinix engineers are evaluating the situation and will provide more details as they become available.

The next update will be when a significant change to the situation occurs.

Our equipment is connected redundantly over 2 feeds, therefore we have only lost a redundancy in power, and do not expect any issues.

For transparency we decided to publish this notification.

Update 02:42:
Equinix engineers have identified a section of busbar that failed for UPS power feed C in Zone 1 of AM5.

The redundant feed remains operational.

Equinix engineers are still working to fully restore the faulty feed.

Affected customers are using the available functioning power feed.

We will keep you posted of the status as more information becomes available.

Thank you for your understanding and patience.

Update 07:00:
We are working on a temporary solution to connect the faulty busbar to provide redundant feed again.

We will keep you posted of the status as more information becomes available.

Thank you for your understanding and patience.


Update 11:30:
Spare parts are on their way to the IBX and will be installed immediately.

We will keep you posted of the status as more information becomes available.

Thank you for your understanding and patience.


Update 12:52:
All materials to create a temporary solution are now on site. We will start to install the temporary solution from 1400 CET onwards. The total work duration is estimated at 3 hours.

We will start to find the final solution and the root cause of this incident after the temporary solution is in place and operational. Our estimation at this moment is that this can take several weeks.

Update 18:10:
The feed that went down is now back up again and all our systems are redundantly connected again.

In the near future Equinix will look for a final solution but as our systems are up and running with redundant power we will close this case on our side.