Past Incidents

Affected Components: SysEleven Stack, region FES

Incident Start: 18th Oct. 2023/00:40

Incident End: 18th Oct. 2023/03:56

Description:

• Occurring errors are currently being investigated.
• During the maintenance the SDN cluster started to cause issues, we are investigating
• API is also not available during recovery

Customer Impact:

• Connectivity is restricted

Update: 01:48

Recovery is still ongoing, network connectivity could be restored partially

Update: 02:45

Recovery is mostly through, we are still fixing octavia loadbalancers

Update: 03:56

Last recoveries of octavia loadalancers are being finished up

Affected Components: MetaKube Platform, all regions

Scheduled Start: 17. Oct. 2023 / 18:00 CEST

Scheduled End: 17. Oct. 2023 / 21:00 CEST

State: DONE

Description:

• Scheduled maintenance of the MetaKube platform in all regions

Customer Impact during the Maintenance:

It is possible that various functions of the control plane are unavailable for brief periods.

Customer Actions:

• No customer actions needed
• Please inform us if you notice any irregularities

Affected Components: SysEleven Stack API, all regions

Incident Start: 2023-10-06 12:09

Incident End: 2023-10-06 12:25

Description:

• Accessibility of the SysEleven Stack API is not ensured.

Customer Impact:

• Spawning new virtual machines (VMs) or changing existing resources is not possible.

Update: 12:25

The API is generally working again (with a concurring incident of smaller scope still persisting, see separate entry).

Affected Components: SysEleven Stack Networking in region DBL

Scheduled Start: 2 Oct 2023, 22:00 CEST

Scheduled End: 3 Oct 2023, 00:00 CEST

State: COMPLETED

Description:

We will perform maintenance on networking gateways.

Customer Impact during the Maintenance:

• Interruption of connectivity of virtual machines of up to 2 minutes
• VPN connections could be interrupted for up to 2 minutes

Customer Actions:

• no customer actions needed
• please inform us if you notice any irregularities

Affected Components: SysEleven Stack API, region DBL

Incident Start: 27th Sept. 2023/03:23 pm

Incident End: 27th Sept. 2023/03:58 pm

Description:

• Accessibility of the SysEleven Stack API is not ensured.

Customer Impact:

• Managing (creating/deleting/attaching/detaching) block storage (cinder) volumes may fail or is not possible at all. Volumes in continuous use are not affected.
• Creating new instances may also fail.

Update: 03:58 pm: Services have been restarted, we are checking functionality.

Update: 04:03 pm: Affected services are restored, we are cleaning up some side-effects that occured during the incident. All newly triggered actions should succeed again.

Update: 04:16 pm: All services are verified to be fully operational again.