All systems are operational

About This Site

Timezone: Europe/Berlin (CET - UTC +1/CEST - UTC +2)

We provide current information about infrastructure and service availability below. If you experience service impacts or performance issues please contact our helpdesk or our Service Desk.

Past Incidents

Tuesday 14th December 2021

SysEleven Stack Identity API SysEleven Stack API issues in all Regions

Currently we are facing issues with the SysEleven Stack API. Authentication requests may be rejected wrongly. We are working on a solution and keep you informed. The outage only affects spawning new virtual machines or changing existing resources.

We apologise for any inconveniences.

Monday 13th December 2021

MetaKube Dashboard / API Metakube API disruption for FES

Metakube API is failing for the FES region. No clusters can be created or managed currently.

Update (17:45): A change could not propagate to the FES infrastructure for Metakube because of a race condition. We were able to manually fix the problem. All systems are operational again.

Sunday 12th December 2021

No incidents reported

Saturday 11th December 2021

No incidents reported

Friday 10th December 2021

[Sicherheitshinweis/Security notice] Apache Log4j2 Version >=2.0 & <2.15

Affected Systems: Customer Applications

Incident Start: 10.12.2021 17:00

Incident End: 13.12.2021 11:00


Description:

On 10/12/2021 a security alert was published, which can be found at CVE-2021-44228.

The risk is that a Remote Code Execution (RCE) Format String Lookup Vulnerability can be exploited when using Apache Log4j2 in a version >=2.0 and <2.15.

We have contacted all customers individually who, to our knowledge, are urgently affected by the vulnerability. We also cannot exclude the possibility that a corresponding version is running in your user context due to your own deployment and cannot be identified by us.

**We therefore ask every customer to check their own application for the possibility of being affected by the aforementioned security vulnerability.

We are happy to assist you within the scope of our possibilities. Please open a ticket in our helpdesk (https://helpdesk.syseleven.de). Outside our service hours, please contact our on-call service by phone in case of emergency.

See also our HelpDesk Article.


Customer Impact:

Possible: Remote Code Execution (RCE) Format String Lookup Vulnerability


Update: 13.12.2021 11:00

We close this notification Issue. To handle the incident individuelly, please create a Ticket.

SysEleven Stack API issues in region CBK

Currently, we are facing issues with the SysEleven Stack API. We are working on a solution and keep you informed. The outage only affects spawning new virtual machines or changing existing resources.

We apologize for any inconvenience.

Update: There was a problem with the MySQL database in the Octavia Loadbalancer service. Everything should come back to normal now.

Thursday 9th December 2021

No incidents reported

Wednesday 8th December 2021

No incidents reported

Tuesday 7th December 2021

No incidents reported

Monday 6th December 2021

No incidents reported

Sunday 5th December 2021

No incidents reported

Saturday 4th December 2021

No incidents reported

Friday 3rd December 2021

No incidents reported

Thursday 2nd December 2021

No incidents reported

Wednesday 1st December 2021

No incidents reported

Tuesday 30th November 2021

No incidents reported

Monday 29th November 2021

No incidents reported

Sunday 28th November 2021

No incidents reported

Saturday 27th November 2021

No incidents reported

Friday 26th November 2021

No incidents reported

Thursday 25th November 2021

No incidents reported

Wednesday 24th November 2021

SysEleven Stack DNS API SysEleven Stack issues for DNS API

At the moment we are facing DNS resolution issues for SysEleven Stack.

We will keep you informed and apologize for any inconveniences.


We identified the issue and are working on a fix.


DNS resolution is working as expected again.


Reason for Outage (RfO) (de)

Cluster Nodes (SysEleven - CBK) Interruption in Metadata service in the CBK

Affected Systems: Cluster Nodes (SysEleven - CBK)

Incident Start: 11:22

Incident End: 12:17


Description:

There is currently a disruption in the Metadata service in the CBK region.


Customer Impact:

MetaKube Customers: Creating new machines is possible, but they don't join the Kubernetes cluster.

SysEleven Stack Customers: At the moment it is not possible to create new VMs.


Update: 12:00

We were able to identify the cause and are in the process of correcting the fault.


Update: 12:17

We have changed the configuration Item. All Services are up again.

Reason for Outages (RfO) (de)

Tuesday 23rd November 2021

SysEleven Stack maintenance announcement, scheduled 3 years ago

Between Nov 23rd, 2021 22:00 and 03:00 (CET) we will perform planned maintenance in Region CBK


During this period there will be:

  • Short interruptions of virtual machine network connectivity for up to 5 minutes
  • Increased storage Input/Output latencies

Should you have any trouble following the maintenance work please contact us.

Block Storage (Cinder Volumes / M1 Flavors) SysEleven Stack Block Storage issues in region DBL

At the moment we are facing issues with the distributed file system, a core component of the SysEleven Stack. At the moment it is possible that virtual machines can't be spawned or write access to their volumes is stuck or very slow.

We will keep you informed and apologise for any inconveniences.

UPDATE 11:50 CET: There are still ongoing block storage performance issues. The octavia API is impacted as well.

UPDATE 12:37 CET: We identified the issue and resolved it. The API and block storage performance is now back to normal. Please contact us at support@syseleven.de if you are still experiencing issues.


Reason for Outage (RfO) (de)

Monday 22nd November 2021

No incidents reported

Sunday 21st November 2021

No incidents reported

Saturday 20th November 2021

No incidents reported

Friday 19th November 2021

No incidents reported

Thursday 18th November 2021

No incidents reported

Wednesday 17th November 2021

SysEleven Stack Authoritative DNS SysEleven Stack Designate DNSaaS issues

At the moment we are facing an issue with DNSaaS.

We will keep you informed and apologise for any inconveniences.

UPDATE 23:35 CET: The service is now operational again.


Reason for Outage (RfO) (de)

Network SysEleven Stack network issues in region CBK

At the moment there might be network issues in region CBK.

We will keep you informed and apologise for any inconveniences.

EDIT 16:01: The issues are now resolved. Should you still experience any issues, please contact us at support@syseleven.de

Tuesday 16th November 2021

Hotline not available

Our hotline is currently not working. Please use our emergency number for urgent requests. Contact our support via email: support@syseleven.de

Update : 12:15 The hotline is reachable again.

Network SysEleven Stack issues in region CBK

At the moment we are facing connectivitiy issues in region CBK.

We will keep you informed and apologise for any inconveniences.

Update : 12:10 We identified the problem and rolled out a fix. Service is up again, we are still observing the situation.

Reason for Outages (RfO) (de)

Monday 15th November 2021

No incidents reported