SysEleven Status and Incidents https://syseleven-status.de Get all incidents by feed http://www.rssboard.org/rss-specification python-feedgen https://www.syseleven.de/wp-content/uploads/2020/10/SysEleven_XL_Logo_quer_RGB.png SysEleven Status and Incidents https://syseleven-status.de de Tue, 12 May 2026 11:11:49 +0000 INCIDENT: Performance Degradation of metakube platform on CBK region <p>Affected Components: <strong>Metakube Platform CBK region</strong></p> <p>Incident Start: 2025-10-24 07:00 UTC+01:00 (CET)</p> 584 Fri, 24 Oct 2025 09:37:02 +0000 Issues with PostgreSQL as a Service <p>Affected Components: DBaaS API and DBaaS Databases</p> <p>Incident Start: <strong>2025-11-26 14:00 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <p>We see issues that connecting to the PostgreSQL Databases may not work properly</p> <hr /> <p>Customer Impact:</p> <p>Issues with Database Access</p> 585 Wed, 26 Nov 2025 14:27:02 +0000 INCIDENT: SysEleven STACK DNSaaS API issues <p>Affected Components: <strong>SysEleven Stack DNSaaS API</strong></p> <p>Incident Start: <strong>2025-11-27 11:06 UTC+01:00 (CET)</strong></p> <p>Incident End: <strong>2025-11-27 12:07 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <ul> <li>SysEleven OpenStack Cloud DNSaaS (Designate) API issues</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Creating or updating DNS zones or records via the OpenStack DNSaaS (Designate) API fails</li> <li>No impact on existing records</li> </ul> <hr /> <p><strong>Update 12:07</strong></p> <p>DNSaaS API functionality is restored</p> 586 Thu, 27 Nov 2025 11:56:14 +0000 INCIDENT: helpdesk portal <p>Incident Start: <strong>2025-12-05 09:50 UTC+01:00 (CET)</strong></p> <p>Incident End: <strong>2025-12-05 10:45 UTC+01:00 (CET)</strong></p> <p>We are currently experiencing an outage of our customer portal. We are already in contact with the manufacturer and are working on restoring it. In urgent cases, please use our hotlines.</p> <p>UPDATE 10.27: While our helpdesk portal is already fully operational again we continue observing the situation. In case of difficulties reaching our portal please do not hesitate in calling us.</p> <p>Update 10:45: This incident has been resolved.</p> 587 Fri, 05 Dec 2025 10:03:29 +0000 INCIDENT: IAM / Dashboard <p>Affected Components: IAM / Dashboard</p> <p>Incident Start: <strong>2025-12-10 07:30 UTC+01:00 (CET)</strong></p> <p>Incident End: <strong>2025-12-10 09:30 UTC+01:00 (CET)</strong></p> <p>Update <strong>9:05</strong> Incident should be resolved, we are watching. Update <strong>9:30</strong> Incident is resolved.</p> 588 Wed, 10 Dec 2025 09:09:08 +0000 Incident: E-Mail <p>Affected Components: E-Mail</p> <p>Incident Start: <strong>2025-12-11 16:15 UTC+01:00 (CET)</strong> Incident Start: <strong>2025-12-11 17:20 UTC+01:00 (CET)</strong></p> <p>Update: Issue found. Broken config, got fixed.</p> 589 Thu, 11 Dec 2025 16:36:28 +0000 INCIDENT: network / DNS all regions <p>Affected Components: <strong>network in all regions</strong></p> <p>Incident Start: <strong>2026-01-08 16:50 UTC+01:00 (CET) </strong> Incident End: <strong>2026-01-08 17:50 UTC+01:00 (CET) </strong></p> <hr /> <p>Description:</p> <p>We are facing Network / DNS issues in CBK, DBL, FES, DUS and HAM. We are investigating.</p> <hr /> <p>Update: 17:26</p> <p>Services are running, network connectivity from outside have issues. Still investigating the network issues. Next update latest 18:00</p> <hr /> <p>Update: 17:48</p> <p>All systems are back to normal, still investigating</p> <hr /> <p>Update: 17:50</p> <p>incident is over</p> <hr /> <p><strong><a href="https://helpdesk.syseleven.de/hc/de/articles/32605555664274-2026-01-16-RFO-DNS-Outage-all-regions">Reason for Outage (RfO)</a> (de)</strong></p> 590 Thu, 08 Jan 2026 16:59:54 +0000 INCIDENT: SysEleven STACK issues in region DUS2 <p>Affected Components: <strong>SysEleven Stack, region DUS2</strong></p> <p>Incident Start: <strong>2026-01-19 16:48 UTC+01:00 (CET)</strong></p> <p>Incident Stop: <strong>2026-01-19 17:43 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Connectivity is restricted</li> </ul> 591 Mon, 19 Jan 2026 17:23:36 +0000 INCIDENT: SysEleven STACK issues in region DBL <p>Affected Components: <strong>SysEleven Stack, region DBL</strong></p> <p>Incident Start: <strong>2026-01-29 10:26 UTC+01:00 (CET)</strong> Incident End: <strong>2026-01-29 10:28 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> 592 Thu, 29 Jan 2026 10:51:33 +0000 INCIDENT: SysEleven STACK issues in region CBK <p>Affected Components: <strong>SysEleven Stack, region CBK</strong></p> <p>Incident Start: <strong>2026-01-29 10:26 UTC+01:00 (CET)</strong> Incident End: <strong>2026-01-29 10:28 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> 593 Thu, 29 Jan 2026 10:52:13 +0000 INCIDENT: SysEleven STACK issues in region FES <p>Affected Components: <strong>SysEleven Stack, region FES</strong></p> <p>Incident Start: <strong>2026-01-29 10:26 UTC+01:00 (CET)</strong> Incident End: <strong>2026-01-29 10:28 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Connectivity is restricted</li> </ul> 594 Thu, 29 Jan 2026 10:55:49 +0000 INCIDENT: Network <p>Affected Components: Uplink ISP mitigating DDOS, side effects for reachability of any resources hosted by SysEleven.</p> <p>Incident Start: **2025-02-04 21:15 UTC+01:00 (CET)</p> <p>22:30 ISP deployed countermeasures. Problems aren't noticeable anymore. Incident End: **2025-02-04 22:35 UTC+01:00 (CE</p> 595 Sat, 14 Feb 2026 22:12:49 +0000 INCIDENT: SysEleven STACK issues in region DUS2 <p>Affected Components: <strong>SysEleven Stack, region DUS2</strong></p> <p>Incident Start: <strong>2026-03-04 09:00 UTC+01:00 (CET)</strong> Incident End: <strong>2026-03-05 10:45 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Performance degradation of certain hardware nodes causing VMs to be slower</li> </ul> <hr /> <p>Update : <strong>2026-03-04 17:50 UTC+01:00 (CET)</strong></p> <p>Mitigation is taking effect, VMs are sequentially getting migrated towards new targets without the performance issues. This will take a bit longer to not make the current impact more severe</p> 596 Wed, 04 Mar 2026 15:39:53 +0000 INCIDENT: SysEleven STACK issues in region HAM1 <p>Affected Components: <strong>SysEleven Stack, region HAM1</strong></p> <p>Incident Start: <strong>2026-03-05 12:00 UTC+01:00 (CET)</strong> Incident End: <strong>2026-03-05 19:30 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Some compute nodes showed performance issues. They are being drained sequentially to avoid impact on workloads. Feel free to reach out to us in case of questions!</li> </ul> 597 Thu, 05 Mar 2026 12:12:23 +0000 INCIDENT: SysEleven STACK issues in region FES <p>Affected Components: <strong>SysEleven Stack, region FES</strong></p> <p>Incident Start: <strong>2026-03-26 15:14 UTC+01:00 (CET) </strong> Incident End: <strong>2026-03-26 15:16 UTC+01:00 (CET) </strong></p> <hr /> <p>Description:</p> <ul> <li>A change in configuration caused an unexpected network outage for the region for ~2 minutes</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Connectivity is restricted</li> </ul> 598 Thu, 26 Mar 2026 15:26:11 +0000 INCIDENT: SysEleven STACK API issues <p>Affected Components: <strong>SysEleven Stack API</strong></p> <p>Incident Start: <strong>2026-04-28 12:55</strong> Incident End: <strong>2026-04-28 13:15</strong></p> <hr /> <p>Description:</p> <ul> <li>Loss of control for DNSaaS API</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Creating new zones or adjusting DNS entries might currently not be possible</li> </ul> 599 Tue, 28 Apr 2026 13:00:20 +0000 INCIDENT: Partial outage of MetaKube Control Planes in DBL/CBK <p>Affected Components: <strong>MetaKube, region DBL/CBK</strong></p> <p>Incident Start: <strong>2026-05-05 15:00 UTC+02:00 (CEST)</strong></p> <hr /> <p>Description:</p> <p>Partial outage of etcd storage for a limited amount of clusters.</p> <hr /> <p>Customer Impact:</p> <ul> <li>Affected clusters do not have a working control plane</li> <li>Nodes and running workloads are not directly affected</li> </ul> <hr /> <p>Customer Actions:</p> <ul> <li>No customer actions needed</li> </ul> <hr /> 600 Tue, 05 May 2026 15:10:26 +0000 INCIDENT: SysEleven STACK network issues in region DBL <p>Affected Components: <strong>SysEleven Stack, region DBL</strong></p> <p>Incident Start: <strong>2026-05-05 15:00 UTC+02:00 (CEST)</strong></p> <p>Incident End: <strong>2026-05-05 16:05 UTC+02:00 (CEST)</strong></p> <hr /> <p>Description:</p> <ul> <li>Network connectivity issues in region DBL</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Connectivity is restricted</li> </ul> <hr /> <p>Update : <strong>2026-05-05 15:40 UTC+02:00 (CET)</strong></p> <p>Network connectivity is restored. We are monitoring the situation.</p> 601 Tue, 05 May 2026 15:19:32 +0000 INCIDENT: all platforms <p>Affected Components: <strong>SysEleven Stack, all regions</strong></p> <p>Incident Start: <strong>2026-05-05 21:39 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <p>We are currently experiencing partial issues with our DNS resolvers.</p> <p>The whole .de-zone is currently experiencing outages as reported by DENIC.</p> <p>We are investigating a workaround.</p> <hr /> <p>Customer Impact: This affects all services on our side and from customers.</p> <hr /> <p><strong>Update: 23:45 / 11:45 pm</strong></p> <p>We are monitoring the situation. We decided not to disable DNSSEC as a workaround to sustain integrity of DNS resolution, though this could ease the situation.</p> <hr /> <p><strong>Update: 00:31 / 12:31 am</strong></p> <p>DENIC is still working on the incident. The situation is improving. We are monitoring.</p> <hr /> <p><strong>Update: 07:31 / 07:31 am</strong></p> <p>DENIC status page says "All Systems Operational". Incident is resolved.</p> <hr /> 602 Tue, 05 May 2026 23:04:56 +0000 INCIDENT: SysEleven STACK API issues, Cloud Dashboard <p>Affected Components: <strong>SysEleven Stack API, all regions</strong></p> <p>Incident Start: <strong>2026-05-05 21:39 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <ul> <li>cloud.syseleven.de and dashboard.syseleven.de are not reachable</li> <li>Cloud APIs are working as expected</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Spawning new virtual machines (VMs) or changing existing resources is not possible.</li> </ul> <hr /> <p><strong>Update: 23:45 / 11:45 pm</strong></p> <p>This is related to the DNS outage reported by DENIC from the 05.05 at 21:39 / 9:39 pm.</p> <hr /> 603 Tue, 05 May 2026 23:31:40 +0000