Past Incidents

Affected Components: SysEleven Stack Networking in region FES

Scheduled Start: 16 May 2023, 22:00 CEST

Scheduled End: 17 May 2023, 01:00 CEST

State: SCHEDULED

Description:

We will perform maintenance on networking gateways.

Customer Impact during the Maintenance:

• Two expected interruptions of external connectivity of virtual machines of up to 20 seconds
• VPN connections could be interrupted twice for up to 20 seconds

Customer Actions:

• no customer actions needed
• please inform us if you notice any irregularities

Affected Components: SysEleven Stack in region DBL

Scheduled Start: 9 May 2023, 22:00 CEST

Scheduled End: 10 May 2023, 01:00 CEST

State: SCHEDULED

Description:

We will update some components of the SysEleven Stack.

Customer Impact during the Maintenance:

• During the update the SysEleven Stack API is unavailable
• Object Storage API is not affected

Customer Actions:

• no customer actions needed
• please inform us if you notice any irregularities

Affected Components: MetaKube Platform Load Balancers

Scheduled Start: 3rd May 2023 / 19:00 CEST

Scheduled End: 3rd May 2023 / 20:30 CEST

State: FINISHED

Description:

Scheduled maintenance of the MetaKube load balancers in all regions.

Customer Impact during the Maintenance:

• Short timouts to the Kubernetes Control Plane possible
• Running workloads will not be affected
• Network availability will not be affected
• Storage availability will not be affected

Customer Actions:

• No customer actions needed
• Please inform us if you notice any irregularities

Affected Components: SysEleven Stack in region CBK

Scheduled Start: 2 May 2023, 22:00 CEST

Scheduled End: 3 May 2023, 01:00 CEST

State: SCHEDULED

Description:

We will update some components of the SysEleven Stack.

Customer Impact during the Maintenance:

• During the update the SysEleven Stack API is unavailable
• Object Storage API is not affected

Customer Actions:

• no customer actions needed
• please inform us if you notice any irregularities

Affected Components: SysEleven Stack Networking in region FES

Scheduled Start: 25 Apr 2023, 22:00 CEST

Scheduled End: 26 Apr 2023, 01:00 CEST

State: COMPLETED

Description:

We will perform maintenance on networking gateways.

Customer Impact during the Maintenance:

• Interruption of external connectivity of virtual machines of up to 2 minutes
• VPN connections could be interrupted for up to 2 minutes

Customer Actions:

• no customer actions needed
• please inform us if you notice any irregularities

Affected Components: SysEleven Stack, region FES

Incident Start: 21st Apr. 2023/07:00 am Incident End: 21st Apr. 2023/17:50 pm

Description:

• There is an increased packet loss up to 8% for the IPv6 connections. IPv4 connections are not affected.

Customer Impact:

• Network connectivity has some issues for IPv6.

Affected Components: SysEleven Stack API, region CBK

Incident Start: 13th Apr. 2023/02:12 pm

Description:

• Performance of the SysEleven Stack API is degraded.
• We are investigating the situation currently.

Customer Impact:

• Spawning new virtual machines (VMs) or changing existing resources can be slower.

Update:02:32 pm

The situation is worse than a performance degradation, correcting the status towards a partial outage, we are further investigating

Update:03.20 pm

We pinned down the issue around 02:56 and are now observing the situation. In the end not only API but also Compute and storage (Block+Object) storage were affected. Changes to the monitoring and alerting are being planned.

Affected Components: CloudObserver/Grafana

Incident Start: 5th April 2023/19:00 CEST
Incident Start: 5th April 2023/22:00 CEST

Description:

• We are currently experiencing issues with the CloudObserver, causing some missing trending data. We identified the problem and are working to resolve the issue.

Customer Impact:

• Metrics from dashboards in your Grafana might be missing

Update 21:30

• We identified and fixed the issue, we were also able to recover most metrics. In some rare cases you might experience a gap between ~19:00 and ~20:00
• We are still closely monitoring the system

Update 22:00

• All systems are back and fully operational

Affected Components: SysEleven Stack API, region CBK

Incident Start: 4th April. 2023/15:30 pm Incident End: 4th April. 2023/15:40 pm

Description:

• Accessibility of the SysEleven Stack API is not ensured.

Customer Impact:

• Spawning new virtual machines (VMs) or changing existing resources is not possible.

Update: 15:42 pm The problem has been solved. The API is available as before.