Darkfiber outage NZS / BNR (Status)

« Back

[#967] Darkfiber outage NZS / BNR (Status)

Posted: 2021-09-04 12:49

Start: 2021-09-04 12:00:00
End : 2021-09-04 21:00:00

Affects: Redundancy BNR / NZS

Around 12:00 we've noticed our connections from GSA towards Nedzone Steenbergen (NZS) and Bytesnet Rotterdam (BNR) went down.

Our darkfiber provider GTT has informed us of a fibercut between Amsterdam and Rotterdam, as both BNR and NZS makes use of this darkfiber they have had a loss of redundancy.

All traffic for NZS is re-routed over our darkfiber with EQX-AM7 / all traffic for BNR is re-routed over DBC.

Wave customers between NZS and GSA are currently down.

Update 1: 2021-09-04 10:23 UTC
Please be advised that the GTT Infra NOC detected an interruption occurring on our network between Amsterdam & Rotterdam at 09:18 UTC. Initial investigation has detected a potential fiber cut occurring. FLM teams have been engaged and are en route with an ETA of 90 mins to site.

Update 2: 2021-09-04 11:29 UTC
FLM teams report that they have now reached site & are beginning their investigations.
Once the OTDR has been shot crews will travel to the break point & ETTR will be made available following a damage assessment.

Update 3: 2021-09-04 12:23 UTC
Crews on site have shot the OTDR & discovered the location of the break to be occurring approx 450 meters outside of Amsterdam. Teams are en route to the break point now with an ETA on site set at roughly 60 minutes.

Update 4: 2021-09-04 14:05 UTC
Crews have now arrived at the break point & discovered the fiber impacted by 3rd party drilling activity on site. A damage assessment & repair plan is being formulated by the FLM crew & civils work will need to be undertaken to excavate the damaged cable. ETTR will become available once complete.

Update 5: 2021-09-04 15:15 UTC
Crews report that civils crews are now on site & are preparing to excavate the damaged fiber. Repair plan will include the introduction of two new handholds along the route of the fiber. ETTR will become available once civils work has been completed & the additional infrastructure deployed to provide a safe route around the ongoing drilling work.

Update 6: 2021-09-04 16:25 UTC
Crews report that excavation work on site is progressing well with splicing to begin at a tentative time of 18:30 UTC.

Update 7: 2021-09-04 17:42 UTC
Crews on site report that excavation work has now completed & they are prepping the fiber for splicing which will commence momentarily.

Final Update: 2021-09-04 18:56 UTC
FLM teams confirm that all fibers have now been spliced & the GTT NOC can confirm that all alarms have cleared with services restoring.
The actions taken by the team on site will constitute a permanent repair & no further disruption to service is anticipated.