AMS-IX Port Malfunction at GlobalSwitch (Status)

« Back

[#1020] AMS-IX Port Malfunction at GlobalSwitch (Status)

Posted: 2023-12-29 14:22

Start: 2023-12-29 13:40:00
End : 2024-01-17 16:00:00

Affects: AMS-IX Reduncancy

We've identified traffic issue with the AMS-IX port located at GlobalSwitch. To ensure the stability of our network and to perform necessary diagnostic checks, we've taken the decision to temporarily disable this port.

We have reason to believe that the root cause of the issue is faulty 100GB optic, we will be scheduling a replacement for the optic later today to restore redundancy for AMS-IX

2023-12-30 02:00 - following extensive check in GSA and replacement of the optics, we have understood that our optics are not at fault and seems like this is an issue form the side of AMS-IX, we have opened a case with them and we are awaiting more information about resolution time from their side.

2023-12-31 13:30 - following a debug session with AMS-IX we have identified that the issue seems to be related to their line card acting up. when moving to the backup PE (we would use a different chassis, line card, port and optic, so it would help us check our internal domain. Please let us know if we can proceed to swap the service. The swap is performed by our photonic switches and takes less than 20 ms)
We can currently confirm that CRC errors have stopped and we will be re-enabling the traffic.
once an RFO(Reason For Outage) will arrive from AMS-IX we will provide an update.

2024-01-17 16:00 - after several tests and maintenance on AMS-IX' part, we've restored the connection fully and all peering is operational again.