Past Incidents

Affected Components: SysEleven Stack LBaaS API (Octavia), region DBL

Incident Start: 9 Aug 2023 15:00

Description:

• Creating octavia load balancers failed in region DBL. Updating existing load balancers brought them into an immutable state ("PENDING_UPDATE") after the update failed.

Customer Impact:

• Creating or updating octavia load balancers failed

Update: 18:30

Creating and updating load balancers is possible again. Some load balancers that became stuck in the PENDING_UPDATE state will be repaired now.

Update: 19:45

The load balancers that were stuck in PENDING_UPDATE have been repaired. The octavia LBaaS service is now fully functional again.

Affected Components: SysEleven Stack, region CBK

Incident Start: 08 Aug. 2023 19:45

Description:

• Occurring errors are currently being investigated.

Customer Impact:

• Connectivity is restricted

Update: 20:00

The SDN of the CBK region had a sudden load spike, it was a short term disruption. We are investigating the root cause.

Affected Components: F5 Load-Balancer Site BKI

Scheduled Start: 26. Jul. 2023 / 04:00 am

Scheduled End: 26. Jul. 2023 / 05:00 am

State: COMPLETED

Description:

• Scheduled fail-over to our new F5 load balancer Hardware

Customer Impact during the Maintenance:

• Loss of all active connections, which should re-connect instantly.

Affected Components: SysEleven Stack, region CBK

Incident Start: **13th July. 2023/08:00

Description:

• We see problems with spawning VMs using localstorage flavors.
• Occurring errors are currently being investigated.

Customer Impact:

• Spawning VMs with localstorage (l1.*) flavors can currently fail.

Update:

• 15:00 : After a long debugging session, we identified the root cause and directly fixed it in place. We are extending our monitoring to cover up the newly identified problems.