We have received reports of slow speeds towards Suddenlink Communication (AS19108)

We have denied the path over GTT in the routing tables.

The best selected path is now going over Cogent.

We will be performing a maintenance on our ticketing desk on 24 December 2018, between 9:00 AM and 9:30 AM CET.

Ticketing system will be brought down once to reboot for the configuration and testing of the LACP configuration.

During this time you may experience that you are unable to access https://ticketing.nforce.com/ and mail based support may be delayed.

For urgent issues during this time, you may want to contact us by phone +31 (0)20 6919299

We have received a notification of a maintenance by one of our transit providers, Telecom italia Sparkle (Seabone)

During their maintenance traffic will automatically reroute over other transit providers. We do not expect any issues.

Their maintenance reason is: "Our engineers will perform URGENT activities on TIS Network: rack replacement on the Amsterdam Equipment."

Their reference: TIS IP-GP191218-03

For transparency purposes we decided to publish this notification.

Currently we have received outage notification from our monitoring system regarding with one of the links we have with Telecom Italia Sparkle.

After inspecting we have noticed no issues at our side.

We have notified Telecom Italia Sparkle about the issue and requested more information on the downtime.

Update 2018-12-03, 10:49:
As of writing this post, we have received notification the connection is restored. However we are still waiting for reason for outage from Telecom Italia Sparkle.

Wave service provider which connects Global Switch Amsterdam (GSA) towards FranceIX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing mandatory splicebox works on our fiberoptical network. This planned activity will take place in Belgium. "

Interoute has given this maintenance reference number: 1-4536270031

All FranceIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Update 2018-12-05:
Interoute has rescheduled this maintenance, the start and end times have been adjusted accordingly.

We have received a notification of a maintenance by one of our transit providers, GTT.

During their maintenance traffic will automatically reroute over other transit providers. We do not expect any issues.

Their maintenance reason is: "Improve network stability - software upgrade"

Their reference: 3130079

For transparency purposes we decided to publish this notification.

Wave service provider which connects Equinix AM7 (EQX-AM7) towards LINX is having a planned maintenance.

Their maintenance reason is "Our provider informed us that, in order to guarantee the stability of their network, they need to conduct maintenance activities that will affect the above-mentioned service. "

Interoute has given this maintenance reference number: 94729

All LINX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Wave service provider which connects Equinix AM7 (EQX-AM7) towards LINX is having a planned maintenance.

Their maintenance reason is "Our provider informed us that, in order to guarantee the stability of their network, they need to conduct maintenance activities that will affect the above-mentioned service. Expected outage duration: 5 minutes"

Interoute has given this maintenance reference number: 94620

All LINX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

We will be performing a maintenance on our ticketing desk on 4 December 2018, between 9:00 AM and 10:00 AM CET.

Ticketing system can be brought down a couple of times for upgrades to the Operating System.

During this time you may experience that you are unable to access https://ticketing.nforce.com/ and mail based support may be delayed.

For urgent issues during this time, you may want to contact us by phone +31 (0)20 6919299

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards DECIX is having a planned maintenance.

Their maintenance reason is "As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing cable rerouting works on our fiberoptical network. "

Interoute has given this maintenance reference number: 1-4494276851

All DECIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Update 2018-12-05:
Interoute has rescheduled this maintenance, the start and end times have been adjusted accordingly.

We have received reports of packet loss towards Amazon AWS (AS16509)

We have denied the path over NL-IX in the routing tables.

The best selected path is now going over AMS-IX.

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards LINX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing hardware reconfiguration works on our transmission platform. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4488969141 and 1-4489176771

All LINX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Update 2018-11-16:
This maintenance has been rescheduled by Interoute, we have updated the start and end date of this post.

We have received reports of slow speeds towards Hargray Communications Group, Inc. (AS22646)

We have denied the path over Cogent in the routing tables.

The best selected path is now going over GTT.

We have received a notification of a maintenance by one of our transit providers, Telecom italia Sparkle (Seabone)

During their maintenance traffic will automatically reroute over other transit providers. We do not expect any issues.

Their maintenance reason is: "Our engineers will perform activities on TIS Network: Software Reconfiguration"

Their reference:1-946841830

For transparency purposes we decided to publish this notification.

We planned in a maintenance for upgrading our router Firmware for R2 in DBC(Databarn Capelle) to the latest firmware.

The traffic will be rerouted through R1 of DBC(Databarn Capelle), if you have redundant BGP sessions/links, only one session will go down and the traffic will go through your other session/link.

For those who do not have redundant connection through this router, may experience an outage up to 1 hour.

We planned in a maintenance for upgrading our router firmware for R1 in DBC(Databarn Capelle) to the latest firmware.

The traffic will be rerouted through R2 of DBC(Databarn Capelle), if you have redundant BGP sessions/links, only one session will go down and the traffic will go through your other session/link.

For those who do not have redundant connection through this router, may experience an outage up to 1 hour.

Update 2018-11-23, 7:42 AM CET:
The firmware upgrade is completed successfully.

For the last steps in our DBA core infrastructure upgrade, we will be moving the uplinks of the racks of Zone 2 to the new infrastructure. Also the VPLS uplinks for customers with interconnects between datacenters will be moved to the new infrastructure.

We will plan in the following 4 maintenances:

Monday 19 November 2018, at 1 AM, DBA Zone 2 Row B and VPLS - Completed at 19 November 2018, 6:00 AM CET
Monday 3 December 2018, at 1 AM, DBA Zone 2 Row C - Completed at 3 December 2018, 6:00 AM CET
Monday 10 December 2018, at 1 AM, DBA Zone 2 Row D
Reserved maintenance day:
Monday 17 December 2018, at 1 AM, in reservation in the event of delayed works.

The maintenances will start at 1 AM and ends at 6AM for each day.

During the maintenance, a short interruption or packet loss of up to 30 seconds can be noticed when the rack uplinks are being moved.

Any racks delayed in maintenance due to unforeseen circumstances will be moved to the day of 17 December 2018, We strongly suggest to visit this maintenance post to stay up to date and see if your rack was postponed to the 17th of December 2018.

Darkfiber provider Relined which connects Globalswitch Amsterdam (GSA) towards Equinix AM5 (EQX-AM5) is having a planned maintenance. The connectivity between GSA and EQX-AM5 will automatically reroute over our EQX-AM7 darkfiber during this maintenance.

Their maintenance reason is:
"Due to reconstruction work carried out by the local authorities. Relined is obliged to relocate a section of her network. This action will result in a temporary interruption of the service."

Their reference is: PA-769001

For transparency purposes we decided to publish notification.

Darkfiber provider Eurofiber which connects Nedzone Steenbergen (NZS) towards Equinix AM7 (EQX-AM7) is having a planned maintenance. The connectivity between NZS and EQX-AM7 will automatically reroute over our GSA darkfiber during this maintenance.

Their maintenance reason is "during this maintenance multiple fibers will be moved because of the reconstruction of the N629"

For transparency purposes we decided to publish notification.

Darkfiber provider Eurofiber which connects Nedzone Steenbergen (NZS) towards Equinix AM7 (EQX-AM7) is having a planned maintenance. The connectivity between NZS and EQX-AM7 will automatically reroute over our GSA darkfiber during this maintenance.

Their maintenance reason is "during this maintenance multiple fibers will be moved because of the reconstruction of the N629"

For transparency purposes we decided to publish notification.

We have received a notification of a maintenance by one of our transit providers, Cogent.

During their maintenance traffic will automatically reroute over our Cogent connection at Equinix Amsterdam 5 and over other transit providers. We do not expect any issues.

Their maintenance reason is: "Hardware upgrade on Core router controller cards"

Their reference:NC837-132-

For transparency purposes we decided to publish this notification.

Update 2018-10-30 15:20:
We received a notification from Cogent that this maintenance has been rescheduled to 3 November on the same timeframes.

We have received reports of packet loss towards PLDT (AS9299)

We have denied the path over NTT in the routing tables.

The best selected path is now going over Cogent.

We have received a notification of a maintenance by one of our transit providers, Cogent.

During their maintenance traffic will automatically reroute over our Cogent connection at Global Switch Amsterdam and over other transit providers. We do not expect any issues.

Their maintenance reason is: "Code upgrade of router modules"

Their reference:NC840-113

For transparency purposes we decided to publish this notification.

Update 2018-10-09 12:12: After having contact with cogent having concerns as it looked like both connections would go down, we have found out incorrect ID's and was assured this is only affecting EQX-AM5 connetion.

As part of the infrastructure change that is currently going on in DBA, Now physical connections of Zone4 and Zone5 is completed, we will be moving the remaining VLANS to the new infrastructure.

We have planned in the following maintenance:

Tuesday 23 October, VLAN 200 tm 300 (Completed)
Thursday 25 October, VLAN 301 tm 377 (Completed)
Wednesday 31 October, VLAN 378 tm 454

During this time, network connections per VLAN may be interrupted up to 1 minute.

Update 2018-10-23, 07:04: The first set of vlans (200 to 300) are moved, we have monitored the stability and now marked the maintenance as completed.

Update 2018-10-25, 06:46: The second set of vlans (301 to 377) are moved, we have monitored the stability and now marked the maintenance as completed. We did have noticed on a few vlans there have been longer than expected downtime. This is resolved automatically. The information we have gathered will be used for replication to ensure lowest downtime possible on the next set.

Update 2018-10-31, 10:39: The last set is moved and everything was stable without any issues.

We received a maintenance notification from one of our infrastructure datacenters Equinix-AM5 that they are having their 3 yearly power maintenance.

Their exact maintenance reason is:

Please be advised that Equinix will temporarily switch off one of the redundant power supplies to some or all of your cabinets within AM5 for approximately 60 minutes. The remaining power supply to your cabinets will stay available including UPS and Emergency Generator backup.

We are performing 3 yearly maintenance activities to our main breakers.

Only one feed will be de-energized at any one time to perform the maintenance.

As all our equipment is setup redundantly we expect no issues, this does not affect any server hosting customers.

their reference is 5-166701745337

Update 2018/10/22:
This maintenance has been canceled currently by Equinix.

As a follow up on an earlier issue described in https://noc.nforce.com/notifications/item/693, we have tested and made changed the cable in between the zones. As this path is now stable again, we will be moving the infrastructure back to the old path.

During Monday 8 October 2018, from 6:30 to 7:00, we will be performing the move. This may cause an interruption of your services up to a minute. We will try to keep the downtime as low as possible.

Update 2018-10-08 8:36: After running on the new paths for an hour, everything seems to be stable so far and we have not found any errors. Hereby we close this maintenance.

Wave service provider which connects Equinix Amsterdam 7 (EQX-AM7) towards Frankfurt is having an outage.

Their outage reason is "Please note that this outage is caused by mandatory emergency maintenance of the local cable supplier in Germany.
The end time of the maintenance is 21/09/2018 06:00 CEST."

KPN has given this maintenance reference number: 141575

All DE-CIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards FRANCEIX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing splice box replacement works on our fiberoptical network. This planned activity will take place in Belgium. "

Interoute has given this maintenance reference number: 1-4352791011

All FRANCEIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Darkfiber provider which connects GlobalSwitch Amsterdam (GSA), BNR and NZS (Nedzone Steenbergen) is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing splice box replacement works on our fiberoptical network. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4352918271

Traffic between GSA and BNR will take the path via DBC.
Traffic between GSA and NZS going over BNR will go to AM7.

For transparency purposes we decided to publish this notification.

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards FRANCEIX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing splice box replacement works on our fiberoptical network. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4352918271

All FRANCEIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards FRANCEIX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing splice box replacement works on our fiberoptical network. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4352858701

All FRANCEIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

We have received reports of high latency towards Beltelecom (AS6697)

We have denied the path over Fiord in the routing tables.

Best selected path is now going over DE-CIX.

We have received reports of slow speeds Cogeco Cable

We have denied the path over transit GTT Communications (AS3257) in the routing tables.

Best selected path is now going over Cogent.

We have received reports of slow speeds towards Dataweb Global Group B.V.

We have denied the path over transit GTT Communications (AS3257) in the routing tables.

Best selected path is now going over Cogent.

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards LINX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing capacity optimization works on our transmission platform. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4314764681

All LINX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards LINX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing capacity optimization works on our transmission platform. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4314704321

All LINX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Currently there is an issue ongoing causing random packetloss at DBA zone 5. We are currently investigating into the issue to find the cause and to resolve this.

At 07.20 we have detected CRC errors, most likely caused by a flapping port on the connection between zone 5 towards the primary core. We have disabled the port for the time being, NOC is working together with an on-site engineer to debug and resolve this issue. In the mean time we are still seeing minor CRC errors on the secondary port, we will debug connection as well when finished with the primary port.

Update 12.30:
Investigation shows that the issue started within several hours after a firmware upgrade of the zone 5 core switch. The firmware upgrade and/or the reload of the switches caused the 40Gbps transceivers to have CRC errors. Changing and cleaning fiberpaths did not resolve the issue in full. Therefore, for the time being, at 11.10 we have switched the uplinks to a different bridge and fiberpath.

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards Frankfurt is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing hardware upgrade works on our transmission platform. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4297978822

All DE-CIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

Wave service provider which connects Equinix AM7 towards the LINX peering exchange in London is having a planned maintenance.

All LINX routed traffic is automatically routed over our other LINX peering exchange connection. We therefore do not expect any issues.

Their maintenance reason is "Maintenance needs to take place in our network. These works will impact your service during the above mentioned service window.".

Their reference is: 88510

For transparency purposes we decided to publish notification.

We have received reports of slow transfer rates towards TDC Denmark (AS3292)

We have denied the path over GTT in the routing tables.

Best selected path is now going over Seabone.

We received a maintenance notification from one of our infrastructure datacenters Equinix-AM5 that they are having their 3 yearly power maintenance.

Their exact maintenance reason is:
Please be advised that Equinix will temporarily switch off one of the redundant power supplies to some or all of your cabinets within AM5 for approximately 60 minutes. The remaining power supply to your cabinets will stay available including UPS and Emergency Generator backup.

We are performing 3 yearly maintenance activities to our main breakers.

Only one feed will be de-energized at to any one time to perform the maintenance.

As all our equipment is setup redundantly we expect no issues, this does not affect any server hosting customers.

their reference is 5-165605442492

Update 2018-08-22:
This maintenance has been canceled by Equinix-AM5, when a new maintenance will be planned in we will create a new NOC post on our site.

We have received reports of slow transfer rates towards Google Fiber (AS16591)

We have denied the path over GTT in the routing tables.

Best selected path is now going over Level 3.

For future expansion to 40/100gbit towards the zones in DBA (Databarn Amsterdam), We will be performing preparations to Zone 4 and Zone 5 at 9 August 2018 and 14 August 2018.

We will be performing the following steps:

- At 9 August 2018, 5:30 AM CEST, we will preform preparations on the cores of Zone 4 and Zone 5 at IRF Stack level.
There should be no interruptions expected during this time.

- At 14 August 2018, 5:30 AM CEST, we will be moving physical ports to the new core infrastructure, supporting 40/100G connections between zones. (Postponed)
At this time, you may experience interruption of network connections for a brief moment (between 5 to 10 seconds).

- At 23 August 2018, 3:30 AM CEST, we will be upgrading the firmware of Zone 4 core switch and moving the physical ports to the new core infrastructure. At this time you may experience interruption of network connections for a brief moment (up to 1 minute). We will try to keep the downtime as low as possible. [Postponed]

- At 29 August 2018, 3:30 AM CEST, we will be upgrading the firmware Zone 5 core switch and moving the physical ports to the new core infrastructure. At this time you may experience interruption of network connections for a brief moment (up to 1 minute). We will try to keep the downtime as low as possible. We will also perform firmware upgrade earlier on this day around 2 AM CEST using a different method of upgrading.




Update 14-08-2018 9:44:
The maintenance of 14 August 2018 is postponed due to incompatibility issues with the optics on the older devices. We suspect a firmware upgrade is required.

Update 15-8-2018 9:27:
We have verified that the optics are only compatible with the latest firmware and we have planned in firmware upgrades for both Zone 4 (On 23 August 2018) and Zone 5 (On 29 August 2018).

Update 23-08-2018 8:45:
There was an issue with the firmware upgrade using the ISSU method causing the switch to be stuck in ISSU unable to rollback or continue. Currently the switches are still running the old firmware. Currently we are investigating alternative ways for the upgrading and we will update this post when we are ready with additional options. In the meanwhile the vendor is investigating why ISSU is currently stuck.

Update 29-08-2018 04:15:
The upgrades are completed.

We have received a notification of maintenance by one of our transit providers, GTT.

During their maintenance traffic will automatically reroute over our other transit providers. We do not expect any issues.

Their maintenance reason is "Software upgrade of core device in Amsterdam".

Their reference is: 2747726

For transparency purposes we decided to publish this notification.

At 10:30 CEST a darkfiber between NZS (Nedzone Steenbergen) and EQX-AM7 (Equinix AM7) went down, carrying 200 Gbps traffic.

Currently all traffic is sent over the alternative path on our ring namely NZS - GSA.

We have contacted the darkfiber provider Eurofiber and were notified that currently one of the fiber paths is damaged and that crews are looking for the exact breaking point.

Update 13:25:

Currently the darkfiber is online again. The root cause of this issue is still not known, but as this line has been stable for the last two hours, we will be closing this case.

We have received a notification of a maintenance by one of our transit providers, Cogent.

During their maintenance traffic will automatically reroute over our other Cogent connection as well as over other transit providers. We do not expect any issues.

Their maintenance reason is: "Service migration to a new linecard"

Their reference:EU5171

For transparency purposes we decided to publish this notification.

We have received a notification of a maintenance by one of our transit providers, NTT.

During their maintenance traffic will automatically reroute over our other NTT connection as well as over other transit providers. We do not expect any issues.

Their maintenance reason is "We will be performing a software upgrade affecting the service(s) listed below.
Users connected to this device will experience latency/downtime as the router is reloaded and routing is reestablished."

For transparency purposes we decided to publish this notification.

Darkfiber provider Eurofiber which connects Nedzone Steenbergen (NZS) towards Equinix AM7 (EQX-AM7) is having a planned maintenance. The connectivity between NZS and EQX-AM7 will automatically reroute over our GSA darkfiber during this maintenance.

This maintenance is done in follow up to the issues with the darkfiber between NZS - EQX-AM7 on 2018-06-29

Their reference: PA-876013

For transparancy purposes we decided to publish notification.

We have received reports of high latency towards EXTREME LTD (AS39709)

We have denied the path over Core-Backbone in the routing tables.

Best selected path is now going over DE-CIX.

We have received a notification of a maintenance by one of our transit providers, Zayo.

During their maintenance traffic will automatically reroute over other transit providers.

Their maintenance reason is "Zayo will perform circuit migration for network optimization".

Their reference is:TTN-0002548704

For transparency purposes we decided to publish this notification.