All systems are operational

About This Site

Timezone: Europe/Berlin (CET - UTC +1/CEST - UTC +2)

We provide current information about infrastructure and service availability below. If you experience service impacts or performance issues please contact our helpdesk or our Service Desk.

Past Incidents

Friday 16th February 2024

No incidents reported

Thursday 15th February 2024

No incidents reported

Wednesday 14th February 2024

No incidents reported

Tuesday 13th February 2024

No incidents reported

Monday 12th February 2024

Object storage (S3 / SEOS) INCIDENT: SysEleven STACK Object Storage issues, region FES

Affected Components: SysEleven Stack Object Storage, region FES

Incident Start: 12th Feb. 2024/13:00 pm (CET - UTC +1)

Incident End: 12th Feb. 2024/14:15 pm (CET - UTC +1)


Description:

At the moment, we are facing issues with the Object Storage (Ceph) in Region FES.


Customer Impact:

  • API is not working correctly

Update 14:18 p.m.

  • Issue is fixed and service is operational

Sunday 11th February 2024

SysEleven Network MAINTENANCE: Dringende Wartungsarbeiten / Urgent maintenance work network component Fortigate

Affected Components: Network Component Fortigate

Scheduled Start: Feb. 11 2024 / 08:00 p.m.

Scheduled End: Feb. 11 2024 / 11: p.m.

State: Scheduled Maintainance


Description:

Urgent security update to Fortigate network component

(FortiOS - Out-of-bound Write in sslvpnd (RCE) (CVE-2024-21762))

Expected Impact:

About 1-2 short disconnections when the services are panning.


Customer Impact during the Maintenance:

  • Network Service might be interrupted when actual maintainance takes place

Update 4:30 p.m.

After further evaluating options to mitigate or patch the disclosed vulnerabilities, we learned that patches were anounced but not yet available for download. We continue to monitor the situation closely and will act as soon as patches and upgrades are available.


Update Sunday Feb. 11 7:00 p.m.

After evaluating and testing the provided FortiOS Updates in our datacenter Lankwitzer Straße (BLA) we will be rolling out the patches today between 8 and 11 p.m. in our datacenters BKI and BLU. We have contacted affected customers separately. As the maintenance work affects our VPN routers, short outages of VPN connections are to be expected during the maintenance window mentioned.


Update Sunday Feb. 11 11:00 p.m. All instances of FortiOS in our datacenters have been updated to versions secure against CVE-2024-21762.

Saturday 10th February 2024

Control Plane Services (SysEleven - CBK) INCIDENT: Partial outage of MetaKube apiserver reachability DBL/CBK

Affected Components: MetaKube apiserver external availability, some clusters, DBL/CBK

Incident Start: 10.Feb 2024/11:00 am Incident End: 10. Feb. 2024/12:35 am


Description:

An expired client certificate caused some load balancer configuration to go stale. Thus, a small number of clusters lost external availability of their apiservers.


Customer Impact:

  • External apiserver availability disrupted for at most 8 clusters at peak for a maximum of 95min

Customer Actions:

Examples:

  • No customer actions needed

Update 12:35 The certificate was renewed and a failover of the load balancer triggered. This restored availability of all clusters again.

Friday 9th February 2024

No incidents reported

Thursday 8th February 2024

No incidents reported

Wednesday 7th February 2024

MAINTENANCE: MetaKube announcement, scheduled 7 months ago

Affected Components: all regions

Scheduled Start: 07. Feb. 2024/06:00 pm Scheduled End: 07. Feb. 2024/09:00 pm

State: COMPLETED


Description:

Infrastructure security update in all regions.


Customer Impact during the Maintenance:

  • Short interruptions (<5min) of the control plane possible (e.g. controller-manager, scheduler), but not API
  • Running workloads will not be affected
  • Network availability will not be affected
  • Storage availability will not be affected

Customer Actions:

  • No customer actions needed
  • Please inform us if you notice any irregularities

Network INCIDENT: SysEleven STACK DNS issues

Affected Components: SysEleven Stack DNS Issues

Incident Start: 7 Feb 2024 12:15 Incident End: 7 Feb 2024 12:35


Description:

  • DNS servers overloaded
  • ns01.cloud.syseleven.net, ns02.cloud.syseleven.net, ns03.cloud.syseleven.net, ns04.cloud.syseleven.net

Customer Impact:

  • availability of dns servers is reduced

Update 12:35

Connectivity recovered

Tuesday 6th February 2024

Network INCIDENT: SysEleven STACK DNS issues

Affected Components: SysEleven Stack DNS Issues

Incident Start: 6 Feb 2024 17:44 Incident End: 6 Feb 2024 17:59


Description:

  • DNS servers overloaded
  • ns01.cloud.syseleven.net, ns02.cloud.syseleven.net, ns03.cloud.syseleven.net, ns04.cloud.syseleven.net

Customer Impact:

  • availability of dns servers is reduced

Update 18:00

Connectivity recovered

Monday 5th February 2024

MAINTENANCE: MetaKube Platform, scheduled 7 months ago

Affected Components: MetaKube Platform, all regions

Scheduled Start: 05. Feb. 2024 / 6:00 pm

Scheduled End: 05. Feb. 2024 / 7:00 pm

State: FINISHED


Description:

  • Scheduled maintenance of MetaKube platform infrastructure

Customer Impact during the maintenance:

  • Short interruptions (<1min) of the control plane possible (e.g. apiserver, controller-manager, scheduler), but not MetaKube API
  • Running workloads will not be affected
  • Network availability will not be affected
  • Storage availability will not be affected

Customer Actions:

  • No customer actions needed
  • Please inform us if you notice any irregularities

UPDATE 19:15

Maintenance finished succesfully.

Network INCIDENT: SysEleven STACK DNS issues

Affected Components: SysEleven Stack DNS Issues

Incident Start: 5 Feb 2024 03:03


Description:

  • DNS servers overloaded
  • ns01.cloud.syseleven.net, ns02.cloud.syseleven.net, ns03.cloud.syseleven.net, ns04.cloud.syseleven.net

Customer Impact:

  • availability of dns servers is reduced

Update 5 Feb 2024 03:22

Connectivity recovered

Sunday 4th February 2024

No incidents reported

Saturday 3rd February 2024

No incidents reported

Friday 2nd February 2024

No incidents reported

Thursday 1st February 2024

No incidents reported

Wednesday 31st January 2024

No incidents reported

Tuesday 30th January 2024

MAINTENANCE: SysEleven STACK announcement (CBK, DBL, FES), scheduled 7 months ago

Affected Components: SysEleven Stack API in all regions

Scheduled Start: 30 Jan 2024, 22:00 CET

Scheduled End: 31 Jan 2024, 02:00 CET

State: SCHEDULED


Description:

We will perform maintenance on the central authentication service of the SysEleven Stack.


Customer Impact during the Maintenance:

  • The SysEleven Stack APIs might be unavailable for a few seconds.
  • Running VMs are not affected.

Customer Actions:

  • no customer actions needed
  • please inform us if you notice any irregularities

Control Plane Services (SysEleven - All regions) INCIDENT: Partial outage of MetaKube API

Affected Components: MetaKube platform, region: FES

Incident Start: 30.01.2024 /11:00 am


Description:

Provisioning of new clusters does not succeed.


Customer Impact:

Currently, no new clusters can be provisioned in the region FES.


Customer Actions:

  • No customer actions needed

UPDATE 12:12 pm

A fix has been implemented and verified, waiting for automatic testing to complete.

UPDATE 12:50 pm

The fix has been rolled out and is being verified.

UPDATE 12:55 pm

The fix has been verified. Clusters created during the incident should automatically come up now. If you encounter any issues, please let us know.

Monday 29th January 2024

No incidents reported

Sunday 28th January 2024

INCIDENT: SysEleven STACK DNS issues

Affected Components: SysEleven Stack DNS Issues

Incident Start: 28th Jan. 2024/10:48 pm

Description:

  • DNS servers overloaded
  • ns01.cloud.syseleven.net, ns02.cloud.syseleven.net, ns03.cloud.syseleven.net, ns04.cloud.syseleven.net

Customer Impact:

  • availability of dns servers is reduced

Update 28 Jan 2024/11:07

Connectivity recovered

Saturday 27th January 2024

Network INCIDENT: SysEleven STACK network issues in region CBK

Affected Components: SysEleven Stack Networking, region CBK

Incident Start: 27th Jan. 2024/06:57 am


Description:

  • We currently experience issues with our SDN in cbk

Customer Impact:

  • Connectivity is restricted

Update 07:50 : The situation has been resolved.

INCIDENT: SysEleven STACK DNS issues

Affected Components: SysEleven Stack DNS Issues

Incident Start: 27th Jan. 2024/07:00 am


Description:

  • DNS servers overloaded
  • ns01.cloud.syseleven.net,ns02.cloud.syseleven.net,ns03.cloud.syseleven.net,ns04.cloud.syseleven.net

Customer Impact:

  • availability of dns servers is reduced

Update 27 Jan 2024/09:50

Connectivity recovered

Friday 26th January 2024

No incidents reported

Thursday 25th January 2024

No incidents reported

Wednesday 24th January 2024

MAINTENANCE: MetaKube Platform, scheduled 7 months ago

Affected Components: MetaKube Platform, all regions

Scheduled Start: 24. Jan. 2024 / 07:00 pm

Scheduled End: 24. Jan. 2024 / 10:00 pm

State: COMPLETED


Description:

  • Scheduled maintenance of the MetaKube platform in all regions
  • Small update to MetaKube infrastructure

Customer Impact during the maintenance:

  • Short interruptions (<5min) of the control plane possible (e.g. controller-manager, scheduler), but not API
  • Running workloads will not be affected
  • Network availability will not be affected
  • Storage availability will not be affected

Customer Actions:

  • No customer actions needed
  • Please inform us if you notice any irregularities

Tuesday 23rd January 2024

No incidents reported

Monday 22nd January 2024

No incidents reported

Sunday 21st January 2024

No incidents reported

Saturday 20th January 2024

No incidents reported

Friday 19th January 2024

No incidents reported

Thursday 18th January 2024

No incidents reported