Currently we have received outage notification from our monitoring system regarding with one of the links we have with Telecom Italia Sparkle.

After inspecting we have noticed no issues at our side.

We have notified Telecom Italia Sparkle about the issue and requested more information on the downtime.

Update 2018-12-03, 10:49:
As of writing this post, we have received notification the connection is restored. However we are still waiting for reason for outage from Telecom Italia Sparkle.

Wave service provider which connects Equinix Amsterdam 7 (EQX-AM7) towards Frankfurt is having an outage.

Their outage reason is "Please note that this outage is caused by mandatory emergency maintenance of the local cable supplier in Germany.
The end time of the maintenance is 21/09/2018 06:00 CEST."

KPN has given this maintenance reference number: 141575

All DE-CIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Currently there is an issue ongoing causing random packetloss at DBA zone 5. We are currently investigating into the issue to find the cause and to resolve this.

At 07.20 we have detected CRC errors, most likely caused by a flapping port on the connection between zone 5 towards the primary core. We have disabled the port for the time being, NOC is working together with an on-site engineer to debug and resolve this issue. In the mean time we are still seeing minor CRC errors on the secondary port, we will debug connection as well when finished with the primary port.

Update 12.30:
Investigation shows that the issue started within several hours after a firmware upgrade of the zone 5 core switch. The firmware upgrade and/or the reload of the switches caused the 40Gbps transceivers to have CRC errors. Changing and cleaning fiberpaths did not resolve the issue in full. Therefore, for the time being, at 11.10 we have switched the uplinks to a different bridge and fiberpath.

At 10:30 CEST a darkfiber between NZS (Nedzone Steenbergen) and EQX-AM7 (Equinix AM7) went down, carrying 200 Gbps traffic.

Currently all traffic is sent over the alternative path on our ring namely NZS - GSA.

We have contacted the darkfiber provider Eurofiber and were notified that currently one of the fiber paths is damaged and that crews are looking for the exact breaking point.

Update 13:25:

Currently the darkfiber is online again. The root cause of this issue is still not known, but as this line has been stable for the last two hours, we will be closing this case.

At 12.19 CEST a darkfiber between NZS (Nedzone Steenbergen) and EQX-AM7 (Equinix AM7) went down, carrying 200 Gbps traffic.

Currently all traffic is send over the alternative path on our ring namely NZS - GSA.

We have contacted the darkfiber provider Eurofiber and were notified that currently one of the fiber paths is damaged and that crews are looking for the exact breaking point.

Update 17:11:
The location of the fiber damage has been found to be construction works in Etten-Leur, repair crews are underway to repair the fibers.

Update 20:38:
The damage to fibers and pipes is that severe that the decision has been made to place new pipes and put new fibers in them, while Eurofiber does not want to give out a solid ETA the estimate is that they will start placing new fibers at 22.00 CEST.

Update 2018-06-30 04.15:
The splicing seems to be finished and connection is re-established.