Past Incidents

Affected Components: SysEleven Network, SysEleven API, all regions potentially affected**

Incident Start: 19th Nov. 2022/07:35 pm

Description:

We are currently experiencing issues within our network. Should you be affected, please contact us via our emergency hotline. Opening a ticket prior to calling us, helps us to evaluate, rectify and resolve issues that may persist with your setup.

Update 09:10 pm: All issues have been resolved. As far as we know at the moment, one of the couters at our Berlin Luetzowstraße datacenter was unfunctional and needed to be resetted. These routers are redundant, however, there was impact seen shortly after the failure of the router until all connections successfully moved to the redundant system.

Affected Components: SysEleven Stack, region CBK

Scheduled Start: 17. Nov. 2022/10:00 pm Scheduled End: 18. Nov. 2022/01:00 am

State: ** DONE**

Description:

• Scheduled maintenance of our SysEleven Stack in Region CBK

Customer Impact during the Maintenance:

• network availability will be affected

Customer Actions:

• No customer actions needed
• Please inform us if you notice any irregularities

Affected Components: SysEleven Stack, region DBL

Scheduled Start: 15. Nov. 2022/10:00 pm Scheduled End: 16. Nov. 2022/01:00 am

State: DONE

Description:

• Scheduled maintenance of our SysEleven Stack in Region DBL

Customer Impact during the Maintenance:

• network availability will be affected

Customer Actions:

• No customer actions needed
• Please inform us if you notice any irregularities

Affected Components: Monitoring Icinga2 problems, region BKI/BLU

Incident Start: 14th Sept. 2022/03:30 am

Incident End: 14th Sept. 2022/5:45 am

Description:

• Problem with icinga monitoring system
• expected false positive alarms

Customer Impact:

• false positive alarms
• potential missed alarms

Affected Components: Hardware node failure, region BLU1

Incident Start: 10th Nov. 2022/10:17 am Incident End: 10th Nov. 2022/10:35 am

Description:

• Malfunction of a hardware node
• Restart of the hardware node was necessary

Customer Impact:

• During the period of restart, there was a short interruption in the availability of the systems.
• Affected customers were notified via E-Mail.
• Please check the affected systems for their full functionality.

Affected Components: SysEleven CloudObserver

Scheduled Start: 08. Nov. 2022/01:45 pm Scheduled End: 11. Nov. 2022/06:00 pm

State: SCHEDULED

Description:

• We are migrating various components of the SysEleven CloudObserver from Cortex to Mimir.

Customer Impact during the Maintenance:

• No customer impact expected

Customer Actions:

• No customer actions needed
• Please inform us if you notice any irregularities

Affected Components: Core Routers, region BKI

Scheduled Start: 7. Nov. 2022/10:00 pm

Scheduled End: 8. Nov. 2022/03:00 am

State: SCHEDULED

Description:

to ensure highest levels of stability and availability, we will perform planned maintenance on our core-router(s) at Berlin Kitzingstraße between 2022-11-07 22:00 and 22-11-08 03:00. We will perform software updates and exchange configuration. Customers will experience service interruptions and re-routings during this maintenance window. In the case of redundant services, the corresponding traffic is moved before the maintenance in order to avoid service impact. We apologize for any inconvenience this maintenance work may cause.

Customer Impact during the Maintenance:

• PVC: reduced redundancy
• CBK: reduced redundancy
• Fortigate Firewall / VPN: reduced redundancy
• DDoS Scrubbing: loss of protection, but no service impact

Customer Actions:

• No customer actions needed
• Please inform us if you notice any irregularities

Update:

01:45 AM: The planned maintenance on our core-routers has been completed successfully. According to our monitoring systems, all services are up and running.

Affected Components: MetaKube Core API, Dashboard

Scheduled Start: 03.11.2022 08:00PM Scheduled End: 03.11.2022 11:00PM

State: CANCELED

Description:

We're updating the management systems of MetaKube Core to keep compatibility with future Kubernetes versions.

Customer Impact during the Maintenance:

• MetaKube API will be unavailable
• MetaKube Dashboard will be unavailable

The control planes or nodes of existing clusters will not be affected.

Affected Components: internal DNS infrastructure

Scheduled Start: 01. Nov. 2022/06:00 am

Scheduled End: 01. Nov. 2022/08:00 am

State: FIXED

Description:

Starting at 6:00 am on November 1st 2022, we will migrate part of our internal DNS infrastructure to an up-to-date platform. This Maintenance will last over the course of the morning, but will have no impact on the operation of your customer systems. DNS queries will continue to be answered throughout.

However, if you observe any problems contrary to expectations, we will be happy to answer any queries about this via our helpdesk or by e-mail to support@syseleven.de.

Customer Impact during the Maintenance:

• No customer impact expected

Affected Components: Monitoring System

Scheduled Start: 26. Oct. 2022/11:00 am Scheduled End: 26. Oct. 2022/13:00 am

State: DONE

Description:

On 26.10.2022, we will switch our monitoring system to operate on a more current platform during the course of the day. This changeover will not affect the operation of customer systems, but it cannot be ruled out that the recording of faults and incidents will take a bit longer than usual during the changeover. We do not expect any false alarms during the changeover, but if these should occur contrary to expectations, we will be happy to answer any questions via our helpdesk or by e-mail to support@syseleven.de.

Customer Impact during the Maintenance:

• No expected customer impact