#817 Cooling issues Databarn Amsterdam (DBA)

Posted: 2019-07-26 18:28

Start: 2019-07-26 16:00:00
End : 2019-07-26 18:00:00

Affects: Servers inside Databarn Amsterdam (DBA)

At 16.00 CEST we received a notification from our monitoring system that temperatures were rising inside the datacenter, we immediately contacted the datacenter and ask for any issues with the cooling system.

They had an issue with the cooling system which has been repaired at this moment,

#812 Darkfiber outage NZS - EQX-AM7

Posted: 2019-07-02 16:05

Start: 2019-07-02 17:00:00
End : 2019-07-03 00:00:00

Affects: Redundancy NZS

At 15:40 CEST a darkfiber between NZS (Nedzone Steenbergen) and EQX-AM7 (Equinix AM7) went down, carrying 200 Gbps traffic.

Currently all traffic is sent over the alternative path on our ring namely NZS - GSA.

We have contacted the darkfiber provider Eurofiber and are currently awaiting an exact issue description.

Update 19:20 CEST:
The issue in this case is a fiber break because of digging in the area, the break location has been found and engineers are currently on their way to the break location to asses and repair the broken fiber duct.

Update 2019-07-03 0:05 CEST:
The fiber has been repaired by our darkfiber provider and the darkfiber has been returned to full operation. At this time Nedzone is fully redundant again.

#807 Outage Global Switch Amsterdam

Posted: 2019-06-07 14:48

Start: 2019-06-07 14:22:00
End : 2019-06-07 14:38:00

Affects: Routing towards GSA, Internet customers terminated at GSA

We have experienced a router linecard crash at 14:22, causing an outage on our network in Global Switch Amsterdam.

The linecard is back online and we have created a case at our vendor regarding to this crash.

Update 10-06-2019:
Vendor confirmed existing bug which is resolved with the next firmware. We are awaiting on the release of this firmware.

#797 Outage connectivity between GSA and NZS

Posted: 2019-04-28 00:17

Start: 2019-04-27 23:12:00
End : 2019-04-27 23:54:00

Affects: Traffic NZS

At 23:12 we received notification of the connection towards GSA (Global Switch Amsterdam) as well as in-house patch connections going down. After inspection we noticed this issue appears to be an in house issue at Nedzone.

At 23:54, we received notification the connection gone up again.

Currently we are still awaiting more information from Nedzone about the issue.

Update 28-04-2019 4:39:
The datacenter has updated us that the cause of this disruption was caused by a power cut. Our floor was unaffected but did cause equipment on the other floor, where our NZS - GSA Connection is connected on to go down.

#796 Packetloss / networkloss DBC

Posted: 2019-04-25 16:51

Start: 2019-04-25 15:25:00
End : 2019-04-25 16:17:00

Affects: Connectivity DBC

At 15:25 we got warning alerts from our monitoring system that packetloss was noticed on multiple switches throughout the network in DBC.

After debugging we traced this back to one of the linecards in one of the coreswitches, as all ports on this card are setup redundantly we have rebooted this linecard.

After it went back online we first tested with a small number of racks to check the stability before enabling all the ports.

At this moment all ports have been enabled and stability has been resumed, we will keep monitoring this linecard and if the same issue occurs we will replace the linecard in question.