All Notifications

#708 Packetloss towards PLDT (AS9299)

Posted: 2018-10-17 07:45

Start: 2018-10-17 07:42:00
End : 2018-10-17 07:42:00

Affects: Traffic towards PLDT

We have received reports of packet loss towards PLDT (AS9299)

We have denied the path over NTT in the routing tables.

The best selected path is now going over Cogent.

#707 IP transit provider Cogent

Posted: 2018-10-09 08:06

Start: 2018-10-15 00:00:00
End : 2018-10-15 06:00:00

Affects: Traffic over Cogent EQX-AM5

We have received a notification of a maintenance by one of our transit providers, Cogent.

During their maintenance traffic will automatically reroute over our Cogent connection at Global Switch Amsterdam and over other transit providers. We do not expect any issues.

Their maintenance reason is: "Code upgrade of router modules"

Their reference:NC840-113

For transparency purposes we decided to publish this notification.

Update 2018-10-09 12:12: After having contact with cogent having concerns as it looked like both connections would go down, we have found out incorrect ID's and was assured this is only affecting EQX-AM5 connetion.

#706 Maintenance DBA Core infrastructure

Posted: 2018-10-09 08:03

Start: 2018-10-23 06:30:00
End : 2018-10-31 07:00:00

Affects: Traffic of related vlans in DBA

As part of the infrastructure change that is currently going on in DBA, Now physical connections of Zone4 and Zone5 is completed, we will be moving the remaining VLANS to the new infrastructure.

We have planned in the following maintenance:

Tuesday 23 October, VLAN 200 tm 300
Thursday 25 October, VLAN 301 tm 377
Wednesday 31 October, VLAN 378 tm 454

During this time, network connections per VLAN may be interrupted up to 1 minute.

#705 Power maintenance Equinix-AM5

Posted: 2018-10-05 14:30

Start: 2018-11-11 00:01:00
End : 2018-11-11 01:30:00

Affects: Equipment located at Equinix-AM5

We received a maintenance notification from one of our infrastructure datacenters Equinix-AM5 that they are having their 3 yearly power maintenance.

Their exact maintenance reason is:

Please be advised that Equinix will temporarily switch off one of the redundant power supplies to some or all of your cabinets within AM5 for approximately 60 minutes. The remaining power supply to your cabinets will stay available including UPS and Emergency Generator backup.

We are performing 3 yearly maintenance activities to our main breakers.

Only one feed will be de-energized at any one time to perform the maintenance.

As all our equipment is setup redundantly we expect no issues, this does not affect any server hosting customers.

their reference is 5-166701745337

#704 Maintenance Zone 5 Databarn Amsterdam

Posted: 2018-09-29 12:46

Start: 2018-10-08 06:30:00
End : 2018-10-08 07:00:00

Affects: Traffic Zone 5 Databarn Amsterdam

As a follow up on an earlier issue described in https://noc.nforce.com/notifications/item/693, we have tested and made changed the cable in between the zones. As this path is now stable again, we will be moving the infrastructure back to the old path.

During Monday 8 October 2018, from 6:30 to 7:00, we will be performing the move. This may cause an interruption of your services up to a minute. We will try to keep the downtime as low as possible.

Update 2018-10-08 8:36: After running on the new paths for an hour, everything seems to be stable so far and we have not found any errors. Hereby we close this maintenance.

#703 Wave service outage DE-CIX EQX-AM7

Posted: 2018-09-20 22:39

Start: 2018-09-20 22:36:00
End : 2018-09-21 06:00:00

Affects: Routing DE-CIX EQX-AM7

Wave service provider which connects Equinix Amsterdam 7 (EQX-AM7) towards Frankfurt is having an outage.

Their outage reason is "Please note that this outage is caused by mandatory emergency maintenance of the local cable supplier in Germany.
The end time of the maintenance is 21/09/2018 06:00 CEST."

KPN has given this maintenance reference number: 141575

All DE-CIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

#702 Wave service maintenance FRANCEIX GSA

Posted: 2018-09-19 14:05

Start: 2018-10-28 00:00:00
End : 2018-10-28 05:00:00

Affects: Routing FRANCEIX

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards FRANCEIX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing splice box replacement works on our fiberoptical network. This planned activity will take place in Belgium. "

Interoute has given this maintenance reference number: 1-4352791011

All FRANCEIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

#701 Darkfiber maintenance BNR GSA

Posted: 2018-09-19 14:01

Start: 2018-12-01 23:00:00
End : 2018-12-02 05:00:00

Affects: Routing IP transit / FTTB/FTTH customers / Nedzone

Darkfiber provider which connects GlobalSwitch Amsterdam (GSA), BNR and NZS (Nedzone Steenbergen) is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing splice box replacement works on our fiberoptical network. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4352918271

Traffic between GSA and BNR will take the path via DBC.
Traffic between GSA and NZS going over BNR will go to AM7.

For transparency purposes we decided to publish this notification.

#700 Wave service maintenance FRANCEIX GSA

Posted: 2018-09-19 14:00

Start: 2018-12-01 23:00:00
End : 2018-12-02 05:00:00

Affects: Routing FRANCEIX

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards FRANCEIX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing splice box replacement works on our fiberoptical network. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4352918271

All FRANCEIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

#699 Wave service maintenance FRANCEIX GSA

Posted: 2018-09-19 14:00

Start: 2018-11-24 23:00:00
End : 2018-11-25 05:00:00

Affects: Routing FRANCEIX

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards FRANCEIX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing splice box replacement works on our fiberoptical network. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4352858701

All FRANCEIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

#698 High latency towards Beltelecom (AS6697)

Posted: 2018-09-18 11:48

Start: 2018-09-18 11:48:00
End : 2018-09-18 11:49:00

Affects: Traffic towards Beltelecom

We have received reports of high latency towards Beltelecom (AS6697)

We have denied the path over Fiord in the routing tables.

Best selected path is now going over DE-CIX.

#697 Slow speeds towards Cogeco Cable (AS7992)

Posted: 2018-09-07 11:39

Start: 2018-09-07 11:33:00
End : 2018-09-07 11:34:00

Affects: Traffic towards Cogeco Cable (AS7992)

We have received reports of slow speeds Cogeco Cable

We have denied the path over transit GTT Communications (AS3257) in the routing tables.

Best selected path is now going over Cogent.

#696 Slow speeds towards Dataweb Global Group B.V. (AS39572)

Posted: 2018-09-07 09:17

Start: 2018-09-07 09:12:00
End : 2018-09-07 09:13:00

Affects: Traffic towards Dataweb Global Group B.V. (AS39572)

We have received reports of slow speeds towards Dataweb Global Group B.V.

We have denied the path over transit GTT Communications (AS3257) in the routing tables.

Best selected path is now going over Cogent.

#695 Wave service maintenance LINX GSA

Posted: 2018-09-07 07:53

Start: 2018-10-01 00:00:00
End : 2018-10-01 08:00:00

Affects: Routing LINX

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards LINX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing capacity optimization works on our transmission platform. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4314764681

All LINX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

#694 Wave service maintenance LINX GSA

Posted: 2018-09-07 07:53

Start: 2018-09-30 00:00:00
End : 2018-09-30 08:00:00

Affects: Routing LINX

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards LINX is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing capacity optimization works on our transmission platform. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4314704321

All LINX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

#693 Packetloss DBA zone 5

Posted: 2018-08-29 09:36

Start: 2018-08-29 06:30:00
End : 2018-08-29 11:10:00

Affects: DBA zone 5

Currently there is an issue ongoing causing random packetloss at DBA zone 5. We are currently investigating into the issue to find the cause and to resolve this.

At 07.20 we have detected CRC errors, most likely caused by a flapping port on the connection between zone 5 towards the primary core. We have disabled the port for the time being, NOC is working together with an on-site engineer to debug and resolve this issue. In the mean time we are still seeing minor CRC errors on the secondary port, we will debug connection as well when finished with the primary port.

Update 12.30:
Investigation shows that the issue started within several hours after a firmware upgrade of the zone 5 core switch. The firmware upgrade and/or the reload of the switches caused the 40Gbps transceivers to have CRC errors. Changing and cleaning fiberpaths did not resolve the issue in full. Therefore, for the time being, at 11.10 we have switched the uplinks to a different bridge and fiberpath.

#692 Wave service maintenance DECIX GSA

Posted: 2018-08-28 18:46

Start: 2018-09-18 00:00:00
End : 2018-09-18 06:00:00

Affects: Routing DE-CIX

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards Frankfurt is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing hardware upgrade works on our transmission platform. This planned activity will take place in Netherlands. "

Interoute has given this maintenance reference number: 1-4297978822

All DE-CIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparency purposes we decided to publish this notification.

#691 Wave service maintenance LINX EQX-AM7

Posted: 2018-08-28 07:38

Start: 2018-09-13 05:00:00
End : 2018-09-13 07:00:00

Affects: Routing LINX

Wave service provider which connects Equinix AM7 towards the LINX peering exchange in London is having a planned maintenance.

All LINX routed traffic is automatically routed over our other LINX peering exchange connection. We therefore do not expect any issues.

Their maintenance reason is "Maintenance needs to take place in our network. These works will impact your service during the above mentioned service window.".

Their reference is: 88510

For transparency purposes we decided to publish notification.

#690 Slow transfer rate towards TDC Denmark (AS3292)

Posted: 2018-08-23 12:11

Start: 2018-08-23 12:10:00
End : 2018-08-23 12:11:00

Affects: Traffic towards TDC Denmark (AS3292)

We have received reports of slow transfer rates towards TDC Denmark (AS3292)

We have denied the path over GTT in the routing tables.

Best selected path is now going over Seabone.

#689 Power maintenance Equinix-AM5

Posted: 2018-08-15 18:25

Start: 2018-09-10 07:30:00
End : 2018-09-10 09:00:00

Affects: Equipment located at Equinix-AM5

We received a maintenance notification from one of our infrastructure datacenters Equinix-AM5 that they are having their 3 yearly power maintenance.

Their exact maintenance reason is:
Please be advised that Equinix will temporarily switch off one of the redundant power supplies to some or all of your cabinets within AM5 for approximately 60 minutes. The remaining power supply to your cabinets will stay available including UPS and Emergency Generator backup.

We are performing 3 yearly maintenance activities to our main breakers.

Only one feed will be de-energized at to any one time to perform the maintenance.

As all our equipment is setup redundantly we expect no issues, this does not affect any server hosting customers.

their reference is 5-165605442492

Update 2018-08-22:
This maintenance has been canceled by Equinix-AM5, when a new maintenance will be planned in we will create a new NOC post on our site.

#688 Slow transfer rate towards Google Fiber (AS16591)

Posted: 2018-08-14 09:49

Start: 2018-08-14 09:46:00
End : 2018-08-14 09:47:00

Affects: Affects: Traffic towards Google Fiber (AS16591)

We have received reports of slow transfer rates towards Google Fiber (AS16591)

We have denied the path over GTT in the routing tables.

Best selected path is now going over Level 3.

#687 Preparations core switch upgrade

Posted: 2018-08-03 12:34

Start: 2018-08-09 05:30:00
End : 2018-08-29 06:30:00

Affects: DBA Zone 4 and Zone 5

For future expansion to 40/100gbit towards the zones in DBA (Databarn Amsterdam), We will be performing preparations to Zone 4 and Zone 5 at 9 August 2018 and 14 August 2018.

We will be performing the following steps:

- At 9 August 2018, 5:30 AM CEST, we will preform preparations on the cores of Zone 4 and Zone 5 at IRF Stack level.
There should be no interruptions expected during this time.

- At 14 August 2018, 5:30 AM CEST, we will be moving physical ports to the new core infrastructure, supporting 40/100G connections between zones. (Postponed)
At this time, you may experience interruption of network connections for a brief moment (between 5 to 10 seconds).

- At 23 August 2018, 3:30 AM CEST, we will be upgrading the firmware of Zone 4 core switch and moving the physical ports to the new core infrastructure. At this time you may experience interruption of network connections for a brief moment (up to 1 minute). We will try to keep the downtime as low as possible. [Postponed]

- At 29 August 2018, 3:30 AM CEST, we will be upgrading the firmware Zone 5 core switch and moving the physical ports to the new core infrastructure. At this time you may experience interruption of network connections for a brief moment (up to 1 minute). We will try to keep the downtime as low as possible. We will also perform firmware upgrade earlier on this day around 2 AM CEST using a different method of upgrading.




Update 14-08-2018 9:44:
The maintenance of 14 August 2018 is postponed due to incompatibility issues with the optics on the older devices. We suspect a firmware upgrade is required.

Update 15-8-2018 9:27:
We have verified that the optics are only compatible with the latest firmware and we have planned in firmware upgrades for both Zone 4 (On 23 August 2018) and Zone 5 (On 29 August 2018).

Update 23-08-2018 8:45:
There was an issue with the firmware upgrade using the ISSU method causing the switch to be stuck in ISSU unable to rollback or continue. Currently the switches are still running the old firmware. Currently we are investigating alternative ways for the upgrading and we will update this post when we are ready with additional options. In the meanwhile the vendor is investigating why ISSU is currently stuck.

Update 29-08-2018 04:15:
The upgrades are completed.

#686 Transit provider GTT maintenance

Posted: 2018-07-30 13:47

Start: 2018-08-10 02:00:00
End : 2018-08-10 06:00:00

Affects: Routing GTT

We have received a notification of maintenance by one of our transit providers, GTT.

During their maintenance traffic will automatically reroute over our other transit providers. We do not expect any issues.

Their maintenance reason is "Software upgrade of core device in Amsterdam".

Their reference is: 2747726

For transparency purposes we decided to publish this notification.

#685 Darkfiber outage NZS - EQX-AM7

Posted: 2018-07-25 11:14

Start: 2018-07-25 10:30:00
End : 2018-07-25 13:25:00

Affects: Redundancy NZS

At 10:30 CEST a darkfiber between NZS (Nedzone Steenbergen) and EQX-AM7 (Equinix AM7) went down, carrying 200 Gbps traffic.

Currently all traffic is sent over the alternative path on our ring namely NZS - GSA.

We have contacted the darkfiber provider Eurofiber and were notified that currently one of the fiber paths is damaged and that crews are looking for the exact breaking point.

Update 13:25:

Currently the darkfiber is online again. The root cause of this issue is still not known, but as this line has been stable for the last two hours, we will be closing this case.

#683 IP transit provider Cogent Globalswitch Amsterdam

Posted: 2018-07-23 15:51

Start: 2018-07-27 06:00:00
End : 2018-07-27 07:00:00

Affects: Redundancy routing Cogent Globalswitch Amsterdam

We have received a notification of a maintenance by one of our transit providers, Cogent.

During their maintenance traffic will automatically reroute over our other Cogent connection as well as over other transit providers. We do not expect any issues.

Their maintenance reason is: "Service migration to a new linecard"

Their reference:EU5171

For transparency purposes we decided to publish this notification.

#680 IP transit provider NTT

Posted: 2018-07-19 15:53

Start: 2018-08-01 02:00:00
End : 2018-08-01 05:00:00

Affects: Redundancy routing NTT

We have received a notification of a maintenance by one of our transit providers, NTT.

During their maintenance traffic will automatically reroute over our other NTT connection as well as over other transit providers. We do not expect any issues.

Their maintenance reason is "We will be performing a software upgrade affecting the service(s) listed below.
Users connected to this device will experience latency/downtime as the router is reloaded and routing is reestablished."

For transparency purposes we decided to publish this notification.

#679 Darkfiber maintenance NZS - EQX-AM7

Posted: 2018-07-18 10:08

Start: 2018-07-19 04:00:00
End : 2018-07-19 06:00:00

Affects: Redundancy NZS

Darkfiber provider Eurofiber which connects Nedzone Steenbergen (NZS) towards Equinix AM7 (EQX-AM7) is having a planned maintenance. The connectivity between NZS and EQX-AM7 will automatically reroute over our GSA darkfiber during this maintenance.

This maintenance is done in follow up to the issues with the darkfiber between NZS - EQX-AM7 on 2018-06-29

Their reference: PA-876013

For transparancy purposes we decided to publish notification.

#678 High latency towards EXTREME LTD (AS39709)

Posted: 2018-07-16 09:54

Start: 2018-07-16 09:54:43
End : 2018-07-16 09:54:43

Affects: Traffic towards EXTREME LTD (AS39709)

We have received reports of high latency towards EXTREME LTD (AS39709)

We have denied the path over Core-Backbone in the routing tables.

Best selected path is now going over DE-CIX.

#677 IP transit provider Zayo

Posted: 2018-07-13 15:42

Start: 2018-07-27 00:01:00
End : 2018-07-27 02:00:00

Affects: Routing Zayo

We have received a notification of a maintenance by one of our transit providers, Zayo.

During their maintenance traffic will automatically reroute over other transit providers.

Their maintenance reason is "Zayo will perform circuit migration for network optimization".

Their reference is:TTN-0002548704

For transparency purposes we decided to publish this notification.

#676 Slow transfer rate towards Verizon (AS701)

Posted: 2018-07-12 10:38

Start: 2018-07-12 10:38:07
End : 2018-07-12 10:38:07

Affects: Traffic towards Verizon (AS701)

We have received reports of slow transfer rates towards Verizon (AS701)

We have denied the path over Cogent in the routing tables.

Best selected path is now going over NTT.

#675 Global Switch Amsterdam Black Building test

Posted: 2018-07-11 16:31

Start: 2018-10-09 04:00:00
End : 2018-10-09 21:00:00

Affects: Global Switch Amsterdam - at risk

We have received a notification of a maintenance by the Datacenter Global Switch Amsterdam (GSA) as they will be doing their "Annual Black Building test 2018".

They notification we received included the following:
We rescheduled the planned Annual Black Building (BBT) Test 2018 to October 9th which was initially planned to execute on Saturday the 3rd of November as mentioned in the Annual Maintenance Calendar 2018 (v1_1) and in the site letter.
On this day our Grid operator has to do maintenance on the Global Switch Amsterdam Main Feed supply (50KV) and we decided to combine this works with the Black Building test.

The main purpose of this BBT is to test all full automatic functionality of all the emergency backup power systems, electrical distribution, and mechanical cooling systems.

- Before the works start all the technical installations will be checked. When there is a slight possibility for a failure or a deviant value (technically) the works won't start or will be aborted.
- The change level is assessed by GS OPS as a level 2 (Medium) as the supply by the main grid is shut down.

Their reference is: Change Control Form 2018-188

This is only an at risk notification, no outages are expected.

#674 IP transit provider NTT

Posted: 2018-07-09 10:46

Start: 2018-07-19 02:00:00
End : 2018-07-19 05:00:00

Affects: Redundancy routing NTT

We have received a notification of a maintenance by one of our transit providers, NTT.

During their maintenance traffic will automatically reroute over our other NTT connection as well as over other transit providers. We do not expect any issues.

Their maintenance reason is "We will be performing a software upgrade affecting the service(s) listed below.
Users connected to this device will experience latency/downtime as the router is reloaded and routing is reestablished."

For transparancy purposes we decided to publish notification.

#673 Darkfiber outage NZS - EQX-AM7

Posted: 2018-06-29 12:44

Start: 2018-06-29 12:19:00
End : 2018-06-30 04:15:00

Affects: Redundancy NZS

At 12.19 CEST a darkfiber between NZS (Nedzone Steenbergen) and EQX-AM7 (Equinix AM7) went down, carrying 200 Gbps traffic.

Currently all traffic is send over the alternative path on our ring namely NZS - GSA.

We have contacted the darkfiber provider Eurofiber and were notified that currently one of the fiber paths is damaged and that crews are looking for the exact breaking point.

Update 17:11:
The location of the fiber damage has been found to be construction works in Etten-Leur, repair crews are underway to repair the fibers.

Update 20:38:
The damage to fibers and pipes is that severe that the decision has been made to place new pipes and put new fibers in them, while Eurofiber does not want to give out a solid ETA the estimate is that they will start placing new fibers at 22.00 CEST.

Update 2018-06-30 04.15:
The splicing seems to be finished and connection is re-established.

#672 Slow transfer rate towards AT&T Services (AS7018)

Posted: 2018-06-25 11:49

Start: 2018-06-25 11:49:41
End : 2018-06-25 11:49:41

Affects: Traffic towards AT&T Services (AS7018)

We have received reports of slow transfer rates towards AT&T Services (AS7018)

We have denied the path over Cogent in the routing tables.

Best selected path is now going over NTT.

#671 IP transit provider Telecom Italia Sparkle

Posted: 2018-06-22 08:44

Start: 2018-06-26 06:00:00
End : 2018-06-26 08:00:00

Affects: Redundancy routing Telecom Italia Sparkle

We have received a notification of a maintenance by one of our transit providers, Telecom Italia Sparkle.

During their maintenance traffic will automatically reroute over our other Telecom Italia Sparkle connection as well as over other transit providers. We do not expect any issues.

Their maintenance reason is "TIS will perform an Emergency Work. Upgrade Software on router JMX960A-AM_DRT01 in Amsterdam (HOL). *** CIRCUITS WILL BE SERVICE AFFECTING ***"

Their reference is: TIS IP-GL260618-01

For transparancy purposes we decided to publish notification.

#670 Slow transfer rate towards Orange Espagne SA (AS12479)

Posted: 2018-06-14 08:45

Start: 2018-06-14 08:45:36
End : 2018-06-14 08:45:36

Affects: Traffic towards Orange Espagne SA (AS12479)

We have received reports of slow transfer rates towards Orange Espagne SA (AS12479)

We have denied the path over Level3 in the routing tables.

Best selected path is now going over Cogent.

#669 Wave service maintenance LINX GSA

Posted: 2018-06-13 16:35

Start: 2018-07-23 00:00:00
End : 2018-07-23 08:00:00

Affects: Routing LINX via GSA

Wave service provider which connects GSA (Global Switch Amsterdam) towards the LINX peering exchange in London is having a planned maintenance.

As this is one port of the LAG towards LINX, only redundancy is affected.

Their maintenance reason is "Dear Customer, We are undertaking a huge endeavour in an effort to maintain highest possible standards of your services we will be carry out an 12km in-span overlay of the Concerto North cable system between England and the Netherlands. The objective is to increase the overall longevity of the cable and achieve an overall decrease of optical attenuation. This project has been scheduled with our partner operating submarine cable repair vessels to begin on 6th of August 2018 10:00:00 UTC and to finish before 20th of August 2018 20:00:00 UTC, where your service will be experiencing downtime for the full period. Please note that schedule is subject to change due to weather conditions and other incidents which the vessels might be prioritized to. YOUR SERVICE(s) will be MIGRATED to an alternative route via the EuroTunnel in order to avoid downtime for the full period of time, during the project. PLEASE NOTE that during this time your services might experience an increase in latency to some destinations. Also in the case where you purchase diverse services between UK and Europe there will be a risk of geographical overlap. As soon as the return migrations have been scheduled, you will receive the date notification stating when your service(s) will be normalized to its original design. The Interoute team will be making the best we can to minimize downtime and inconvenience to customers with the end goal of delivering the best quality of services. ".

Their reference is: 1-3898503942

For transparency purposes we decided to publish notification.

#668 Wave service maintenance LINX GSA

Posted: 2018-06-13 16:33

Start: 2018-07-21 00:00:00
End : 2018-07-21 08:00:00

Affects: Routing LINX via GSA

Wave service provider which connects GSA (Global Switch Amsterdam) towards the LINX peering exchange in London is having a planned maintenance.

As this is one port of the LAG towards LINX, only redundancy is affected.

Their maintenance reason is "Dear Customer, We are undertaking a huge endeavour in an effort to maintain highest possible standards of your services we will be carry out an 12km in-span overlay of the Concerto North cable system between England and the Netherlands. The objective is to increase the overall longevity of the cable and achieve an overall decrease of optical attenuation. This project has been scheduled with our partner operating submarine cable repair vessels to begin on 6th of August 2018 10:00:00 UTC and to finish before 20th of August 2018 20:00:00 UTC, where your service will be experiencing downtime for the full period. Please note that schedule is subject to change due to weather conditions and other incidents which the vessels might be prioritized to. YOUR SERVICE(s) will be MIGRATED to an alternative route via the EuroTunnel in order to avoid downtime for the full period of time, during the project. PLEASE NOTE that during this time your services might experience an increase in latency to some destinations. Also in the case where you purchase diverse services between UK and Europe there will be a risk of geographical overlap. As soon as the return migrations have been scheduled, you will receive the date notification stating when your service(s) will be normalized to its original design. The Interoute team will be making the best we can to minimize downtime and inconvenience to customers with the end goal of delivering the best quality of services.".

Their reference is: 1-3938453812

For transparency purposes we decided to publish notification.

#665 Ring VPLS Firmware Upgrades

Posted: 2018-06-08 07:56

Start: 2018-08-01 01:00:00
End : 2018-11-03 05:00:00

Affects: (See message for more information)

To improve our network quality and resolve minor bugs in the current firmware, we will be performing firmware upgrades at the following days and times:

Nikhef - 1 Augustus 2018 At 1 AM CEST (Completed)
Affected: Local connected customers. Some peering/transit traffic, which will be redirected over other connections.

EQX-AM5 - 3 Augustus 2018 At 1 AM CEST (Completed)
Affected: Local connected customers. Some peering/transit traffic, which will be redirected over other connections.

BNR - 6 Augustus 2018 At 1 AM CEST (Completed)
Affected: Internet Customers. Local connected customers. Some peering/transit traffic, which will be redirected over other connections.

NZS - Rescheduled to: 4 October 2018 At 4 AM CEST (Completed)
Affected: Network connectivity NZS.

DBA - Rescheduled to: 11 October 2018 At 1 AM CEST (Completed)
Affected: Network connectivity DBA.

DBC - Rescheduled to: 18 October 2018 At 1 AM CEST (Completed)
Affected: Network connectivity DBC.

GSA - Rescheduled to: 27 October 2018 At 1 AM CEST
Affected: Internet Customers. Local connected customers. Some peering/transit traffic, which will be redirected over other connections.

EQX-AM7 - Rescheduled to: 3 November 2018 At 1 AM CET
Affected: Internet Customers. Local connected customers. Some peering/transit traffic, which will be redirected over other connections.


Update 2018-08-03 01.00:
While upgrading the EQX-AM5 VPLS switches all appeared to have been going smooth, but after having both switches upgrades traffic became intermittent on both of leading to random packet loss, engineers are currently on-site debugging this issue.

Update 2018-08-03 03.51:
We have reverted the switch back to the previous firmware which restored all connectivity, we will send our debug logs to the vendor for investigation of why this failed.

Update 2018-08-03 09.16:
Due to a possible firmware bug, which caused the firmware upgrade of EQX-AM5 to cause intermittent packet loss we have decided to postpone major locations. We have not postponed BNR at this time as part of required debugging. As NIKHEF is completed and we are not experiencing any issues here, it is currently not planned in for a downgrade.

We will be awaiting our vendor for more details before we continue on with the firmware upgrades at our major locations. After our vendor have investigated the issue, we will plan in the new firmware upgrades and update this post.

Update 2018-08-09 11.23:
We will continue with the firmware upgrade of EQX-AM5 and debug the issue we are seeing with the vendor on Monday 13 August 2018. All traffic will be balanced over our other transit providers and peers. The firmware upgrade and debug sessions will be during the day.

We do not expect that the maintenance will cause disruptions as traffic will be offloaded before maintenance.

Update 2018-08-13 17.32:
We were able to debug and resolve the issues encountered previously. The firmware of EQX-AM5 has been updated. We will monitor this site closely the coming days. We will replan this maintenance window for the rest of the switches, and sent an update once available.

Update 2018-08-15 12.15:
We have re-scheduled the firmware upgrades for our Ring infrastructure, for more information on the date, please refer to the initial post above.

Update 2018-09-22 02.20:
While flashing the switches in EQX-AM7 after the first switch was flashed with the new firmware it got stuck in a boot loop, after trying to reboot it twice it booted with the new firmware causing packetloss on ports on that same switch, after flashing it back to the old firmware and rebooting it, the stack got stable again. We've created diagnostics on these switches which will be sent to our vendor for debugging.

Update 2018-09-25 13.50:
We have changed the time for NZS firmware upgrade to preform debug operations and tests in case we experience the same issue as with EQX-AM7. We will keep any downtime as low as possible.

Update 2018-10-04 15.00:
Upgrade at NZS has been finished last night.

Update 2018-10-05 8.50:
As the alternative firmware upgrade method is working as expected, we will be performing this method on the remaining firmware upgrades. As EQX-AM7 was delayed, we have now planned the maintenance on the 3rd of November 2018 at 1 AM CET.

Update 2018-10-11 03.38:
The upgrade at DBA has been finished.

Update 2018-10-18 02.10:
Firmware upgrades finished at DBC. We see packetloss at connectivity towards DBC routers.

Update 02.30:
2x 100 Gbps ports (total 4) connecting the ring to each router are disabled. Packetloss issue is resolved. Further debugging problematic ports.

Update 03.55:
Debugging has been finished, a hard reset of the 400 Gbps bridge seems to have resolved the issue. This is currently done for ring - R2. We are awaiting traffic and bgp sessions to restore to its full state. Once done we will offload traffic of R1 to R2 and do the same for ring - R1.

Update 04.25:
The 400 Gbps bridge for ring - R1 has been hard reset as well.

Update 04.35:
Maintenance for DBC is hereby completed.

#664 Slow speeds towards China Telecom (AS4134)

Posted: 2018-06-03 21:34

Start: 2018-06-03 21:34:25
End : 2018-06-03 21:34:25

Affects: Traffic towards China Telecom (AS4134)

We have received reports of slow speeds towards China Telecom.

We have denied the path over Cogent in the routing tables.

Best selected path is now going over NTT.

#663 FTTB outage Stolwijk / Bergambacht / Nieuwerkerk aan de Ijssel

Posted: 2018-06-03 11:46

Start: 2018-06-03 10:22:00
End : 2018-06-03 13:31:00

Affects: Stolwijk / Bergambacht / Nieuwerkerk

*This does not affect any Hosting or Transit Customers and is only valid for Customers located in one of the affected areas.*

Due to a power outage in Waddinxveen the PoP of our transport provider Eurofiber is also without power.

Because of this our connections to Stolwijk / Bergambacht and Nieuwerkerk aan de Ijssel are currently not working.

According to Eurofiber the current ETA for solving the power disruption is 4 PM CEST.
They've also sent out engineers with a power generator to feed the PoP for which the ETA is set to 1 PM CEST.

Our apologies for the incovenience caused.

#662 Wave service maintenance LINX EQX-AM7

Posted: 2018-05-31 09:53

Start: 2018-06-27 00:00:00
End : 2018-06-27 07:00:00

Affects: Routing LINX

Wave service provider which connects EQX-AM7 (Equinix AM7 Amsterdam) towards the LINX peering exchange in London is having a planned maintenance.

Their maintenance reason is "To ensure an optimal service we need to perform Planned Maintenance activities that will affect one of your network connections."

In case of a full outage, all LINX routed traffic is automatically routed over our other LINX peering exchange connection. We therefore do not expect any issues.

Their reference is: 85684 and 85686

For transparency purposes we decided to publish notification.

Update from KPN 2018-06-26:
Please note due to administrative error, the maintenance did not took place but will take place tonight. Our apologies for the inconvenience.

#661 Wave service maintenance LINX EQX-AM7

Posted: 2018-05-30 16:43

Start: 2018-06-01 00:00:00
End : 2018-06-01 06:00:00

Affects: Routing LINX

Wave service provider which connects EQX-AM7 (Equinix AM7 Amsterdam) towards the LINX peering exchange in London is having a planned maintenance.

Their maintenance reason is "We would like to clean the fibers on this service. Our second line engineer informs us that there is to much attenuation on this service and would like to send an engineer to clean the fibers."

In case of a full outage, all LINX routed traffic is automatically routed over our other LINX peering exchange connection. We therefore do not expect any issues.

Their reference is: 132226

For transparency purposes we decided to publish notification.

#660 Wave service maintenance LINX GSA

Posted: 2018-05-29 09:01

Start: 2018-07-23 00:00:00
End : 2018-07-23 08:00:00

Affects: Traffic LINX

Wave service provider which connects GSA (Global Switch Amsterdam) towards the LINX peering exchange in London is having a planned maintenance.

All LINX routed traffic is automatically routed over our other LINX peering exchange connection. We therefore do not expect any issues.

Their maintenance reason is "As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing hardware upgrade works on our transmission platform. This planned activity will take place in Netherlands.".

Their reference is: 1-3898503942

For transparency purposes we decided to publish notification.

#656 Updates to our Privacy Policy

Posted: 2018-05-23 16:30

Start: 2018-05-25 00:00:00
End : 0000-00-00 00:00:00

Affects: N/A

On the 25th of May 2018, the European Union (EU) will bring into force the General Data Protection Regulation (GDPR). GDPR guarantees for its citizens many rights to their digital privacy. In order to be compliant with this regulation we are updating our privacy policy. This will provide even more transparency for our services.

The updated versions of our agreements, policies and guidelines are available at our legal page, which you can find at https://www.nforce.com/legal - will provide you with more insight over what we do with your personal data.

Your security, as well as your privacy, has always been important to us. We will continue to stay committed to protecting your data.

#655 Wave service maintenance LINX EQX-AM7

Posted: 2018-05-22 07:50

Start: 2018-05-29 05:00:00
End : 2018-05-29 07:00:00

Affects: Routing LINX

Wave service provider which connects EQX-AM7 (Equinix AM7 Amsterdam) towards the LINX peering exchange in London is having a planned maintenance.

In case of a full outage, all LINX routed traffic is automatically routed over our other LINX peering exchange connection. We therefore do not expect any issues.

Their maintenance reason is "Your service is protected but a switch-hit might occur twice. We need to switch the service back to the Ulysses cable.".

Their reference is: 85406

For transparency purposes we decided to publish notification.

#654 Packetloss towards Bani Networks LTD (AS134084)

Posted: 2018-05-20 18:52

Start: 2018-05-20 18:52:55
End : 2018-05-20 18:52:55

Affects: Traffic towards Bani Networks LTD (AS134084)

We have received reports of packet loss towards Bani Networks LTD.

We have denied the path over peer Bharti Airtel LTD (AS9498) in the routing tables.

Best selected path is now going over Cogent.

#653 Router firmware upgrades

Posted: 2018-05-17 12:19

Start: 2018-05-24 01:00:00
End : 2018-05-30 07:00:00

Affects: (See message for more information)

During these times we will be performing firmware upgrades of our routers, to solve a part of the issues we have been seeing in the past which is explained at our NOC post: https://noc.nforce.com/notifications/item/649

Our time schedule is as follows:

24 May 2018 at 1AM CEST we will be performing firmware upgrade on the router at Global Switch Amsterdam (GSA)
Affected: IP Transit customers, Redundancy network.

25 May 2018 at 11 AM CEST we will be performing firmware upgrade on the router at Equinix-AM5 (AM5)
Affected: Peer British Telecom

26 May 2018 at 1AM CEST we will be performing firmware upgrade on the router at Equinix-AM7 (AM7)
Affected: IP Transit customers, Internet customers, Redundancy network.

29 May 2018 at 1AM CEST we will be performing firmware upgrade on the router 2 at Nedzone (NZS).
Affected: Redundancy network NZS.

29 May 2018 at 5AM CEST we will be performing firmware upgrade on the router 2 at Databarn Capelle (DBC).
Affected: Redundancy network DBC.

30 May 2018 at 1AM CEST we will be performing firmware upgrade on the router 1 at Nedzone (NZS).
Affected: Redundancy network NZS.

30 May 2018 at 5AM CEST we will be performing firmware upgrade on the router 1 at Databarn Capelle (DBC).
Affected: Redundancy network DBC.

#652 Wave service maintenance FranceIX

Posted: 2018-05-16 22:58

Start: 2018-06-09 01:00:00
End : 2018-06-09 06:00:00

Affects: Routing FranceIX

Wave service provider which connects Globalswitch towards the FranceIX peering exchange in Paris is having a planned maintenance.

All FranceIX routed traffic is automatically routed over our other peering exchanges or IP transit connections.

Their maintenance reason is "As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing splice box replacement works on our fiberoptical network. This planned activity will take place in Belgium."

Their reference: 1-3787161641

For transparancy purposes we decided to publish this notification.


Update 2018-06-07 09.55:
Interoute moved the starttime one hour ahead.

#651 Wave service maintenance DECIX GSA

Posted: 2018-05-16 09:06

Start: 2018-08-05 00:00:00
End : 2018-08-05 06:00:00

Affects: Routing DE-CIX

Wave service provider which connects GlobalSwitch Amsterdam (GSA) towards Frankfurt is having a planned maintenance.

Their maintenance reason is " As part of our commitment to continually improve the quality of service we provide to our customers, we wish to inform you INTEROUTE will be performing mandatory cable works on our fiberoptical network. This planned activity will take place in Germany. "

Interoute has given this maintenance reference number:1-3794823251

All DE-CIX routed traffic is automatically routed over our other exchanges and IP transits.

For transparancy purposes we decided to publish this notification.


Update 2018-06-02:
Interoute has prosponed the maintenance to the 5th of August 2018.