We have noticed a big outage on one of the dutch Internet Exchanges named NL-ix.

On their traffic graphs it is clearly visible an outage is visible https://www.nl-ix.net/network/traffic-stats/

The issue started at 13:15 and is resolved around 13:40.

We will be monitor the situation and if the issue returns, we will temporary disable our sessions with NL-ix.

Update sent by the NLix 15:07:
Investigation about cause is still ongoing, but is seems this is related to the general issue going on in Nikhef.

Around 9:00 we've noticed our connection from EQX-AM7 towards DECIX went down.

Wave service provider which connects Equinix AM7 (EQX-AM7) towards DECIX is investigating the issue.

Update 14:00: The 3rd Party Engineer´s performed OTDR measurements and showing an interruption between Cologne and Dusseldorf.
Fibre team on the way to break location , no ETA/ETR yet.

Final update from our wave service provider: After concluding our investigation, the GTT optical NOC can report back that we observed outage on our DWDM backhaul between Dusseldorf and Linz through which your circuit traverses. Further case was opened with the underlying fibre provider.Our partner immediately started their investigation and dispatched techs to Dusseldorf to troubleshoot the fiber. They performed OTDR test and identified possible fiber cut. Our partner dispatched the engineer and found a fiber cut due to construction work at Bergisch Gladbacher Str., 51063 Cologne. Our partner splices the fiber which restored the services. Restoration work delay due to our partner need to installed approximately 400ft of new cable with 6 Tubes needs to spliced to restore connectivity. Our partner confirmed this is permanent fixed.

At 10.20 we had one of the ring switches in DBC reload itself, where we have 2 switches running in a stack. The moment the second switch came back online we started seeing packetloss throughout DBC.

For now we disabled all ports of the second switch while we investigate into this matter, currently seeing a steady ping again.

Update 12:20:
Half of the ports have been enabled again, after resetting the bridges on the VPLS ring.

Update 16:15:
The other half has been enabled now as well.

We've made a case at the vendor to debug this in more detail.

Due to the on-going issues following in post #936 we are going to do an emergency reload of our router in EQX-AM7.

Internet Customers will experience a loss of internet as well as customers having only one BGP session. Customers having a redundant BGP session with us should not experience any issues.

We are aware it is the middle of the day for most but due to the on-going issues there is little choice left for us to do it at a very short timespan, we will try our best to keep the downtime to an absolute minimum.

Update 15:30:
The router has been reload and is back online.

We've became aware of multiple issues affecting our FTTB customers and are currently investigating.

Once we have an update it will be posted here.

Update 15:42:
After the reload of our router in the previous ticket we see most of the issues resolved, a few remain and we'll continu on those.

Update 17:22:
The last few we had now also report it to be working again.

In case you still notice any issues please let us know.