Past Incidents

Affected Components: SysEleven Stack, region CBK

Incident Start: 14th Sept. 2023/14:48

Incident End: 14th Sept. 2023/15:05

Description:

• Occurring errors are currently being investigated.

Customer Impact:

• Connectivity is restricted

Update:15:08

A SDN component on one of the network gateways of the CBK region crashed and did not restart correctly. This caused further network issues in the whole CBK cluster. Another restart of the SDN component was triggered manually to bring back normal service.

Update:15:09 We consider the incident as resolved. Please contact us if you still experience problems.

Affected Components: SysEleven Stack API, region DBL

Incident Start: 31th Aug. 2023/14:15

Incident End: 31th Aug. 2023/15:00

Description:

• We are seeing problems with the Cinder Volume services as well as Nova API services. The Accessibility of the SysEleven Stack API is not ensured.

Customer Impact:

• Spawning new virtual machines (VMs) or changing existing resources is not possible.

Update:15:00

We brought back the failed services, control over resources is back, we are observing the situation and search for the root cause

Incident Start: 24th Aug. 2023/10:10 am

We saw a problem with the network gateway for our cloud region cbk. We are still monitoring and evaluating what caused the issues.

Update: 24th Aug. 2023/10:21 pm

We consider the incident as resolved. Please contact us if you still experience problems.

Incident Start: 20th Aug. 2023/05:58 pm

We saw a problem with the network gateway for our cloud region cbk. We are still monitoring and evaluating what caused the issues.

Update: 20th Aug. 2023/06:13 pm

We consider the incident as resolved. Please contact us if you still experience problems.

[We have edited the wording for clarity on August 22nd (Service Management)]

Incident Start: 19th Aug. 2023/01:20 pm

We saw a problem with the network gateway for our cloud region cbk. We are still monitoring and evaluating what caused the issues.

Update: 19th Aug. 2023/01:30 pm

We consider the incident as resolved. Please contact us if you still experience problems.

[We have edited the wording for clarity on August 22nd (Service Management)]

Affected Components: SysEleven Stack, region CBK

Incident Start: 16th Aug. 2023/23:37

Description:

• Occurring errors are currently being investigated.

Customer Impact:

• Connectivity is restricted

Update: 17th Aug. 2023/07:25 am

We identified a problem with the gateway and right now no external traffic is possible, we are still working on the restoration

Update: 17th Aug. 2023/07:55 am

We have been observing network problems in the Cloud Region CBK in relation with the underlying SDN (Midonet). So far, we have not been able to restore the functionality of the Cloud Region.

Systems in the CBK cloud region are currently not accessible from public networks, which means that it is also not possible to transfer data to and from the systems. Authentication to MetaKube clusters and individual other systems (SysEleven Login, Open ID Connect, code.syseleven.de) is currently restricted. The systems in the CBK region are basically running, so we do not currently expect any data loss.

We are working at full speed to resolve the issues.

Update: 17th Aug. 2023/08:35 am

Cluster creation in all regions is currently taking longer than usual.

Update: 17th Aug. 2023/08:45 am

Network Connectivity in region CBK is slowly recovering

Update: 17th Aug. 2023/09:10 am

The cause of the Cloud Region CBK network issues has been identified and resolved, Cloud Region CBK functionality has been restored to most systems, or should be shortly.

If you continue to observe problems, please contact us.

Update: 17th Aug. 2023/09:30 am

We are running scripts to automatically redeploy loadbalancers in the cloud region CBK. If you experience further network related problems this might be related and should be fixed shortly.

Update: 17th Aug. 2023/10:40 am

Some loadbalancers are still causing issues, we are looking into the issues, while most loadbalancers should be back online and functional.

Update: 17th Aug. 2023/11:10 am

We consider the incident as resolved. Please contact us if you still experience problems.

Update: 21th Aug. 2023/08:15 pm

Reason for Outage (RfO)