All systems are operational

About This Site

Timezone: Europe/Berlin (CET - UTC +1/CEST - UTC +2)

We provide current information about infrastructure and service availability below. If you experience service impacts or performance issues please contact our helpdesk or our Service Desk.

Past Incidents

Tuesday 24th September 2024

Block Storage (Cinder Volumes / M1 Flavors) INCIDENT: SysEleven STACK Block Storage issues, region CBK

Affected Components: SysEleven Stack Block Storage, region CBK

Incident Start: 2024-09-24 10:55 UTC+02:00 (CEST)


Description:

At the moment we are facing issues with the Block Storage in Region CBK.


Customer Impact:

  • Volumes may be unavailable
  • Instances booted from a volume may be unavailable

Update: 2024-09-24 11:00 UTC+02:00 (CEST)

Issue identified and we are fixing the problem.


Update: 2024-09-24 12:15 UTC+02:00 (CEST)

Problem is fixed. Instances using volumes had to be restarted.

Monday 23rd September 2024

No incidents reported

Sunday 22nd September 2024

No incidents reported

Saturday 21st September 2024

No incidents reported

Friday 20th September 2024

No incidents reported

Thursday 19th September 2024

Block Storage (Cinder Volumes / M1 Flavors) INCIDENT: SysEleven STACK API issues, region DBL

Affected Components: SysEleven Stack API, region DBL

Incident Start: 2024-09-19 08:23 Incident End: 2024-09-19 08:35


Description:

  • Loss of control situation for volume and compute services

Customer Impact:

  • Creating new virtual machines (VMs) or changing existing resources is not possible.

Update: 2024-09-18 08:35

Some nova services also seem to be affected, we investigated the situation and bring the services back up

Wednesday 18th September 2024

No incidents reported

Tuesday 17th September 2024

SysEleven Stack DNS API INCIDENT: SysEleven STACK Designate API issues

Affected Components: SysEleven Stack Designate API

Incident Start: 2024-09-17 20:42 UTC+02:00 (CEST)

Incident End: 2024-09-17 22:15 UTC+02:00 (CEST)


Description:

  • Managing DNS zones and records through the Designate API may fail.

Customer Impact:

  • Loss of control for DNS resources.

Update: 2024-09-17 21:20 UTC+02:00 (CEST)

At 21:15, we reverted a configuration change that was rolled out earlier today, which fixed the issue. We are watching the situation.


Update: 2024-09-17 22:15 UTC+02:00 (CEST)

Underlying issues that were persistent, but not impacting the API, are now fully resolved. Incident is over.


Monday 16th September 2024

No incidents reported

Sunday 15th September 2024

No incidents reported

Saturday 14th September 2024

No incidents reported

Friday 13th September 2024

No incidents reported

Thursday 12th September 2024

No incidents reported

Wednesday 11th September 2024

No incidents reported

Tuesday 10th September 2024

No incidents reported

Monday 9th September 2024

No incidents reported

Sunday 8th September 2024

No incidents reported

Saturday 7th September 2024

No incidents reported

Friday 6th September 2024

No incidents reported

Thursday 5th September 2024

No incidents reported

Wednesday 4th September 2024

INCIDENT: SysEleven STACK API issues, region HAM1

Affected Components: SysEleven Stack API, region HAM1 and DUS2

Incident Start: 2024-09-04 15:20 UTC+02:00 (CEST)

Incident End: 2024-09-04 16:30 UTC+02:00 (CEST)


Description:

  • New VMs could not be created

Customer Impact:

  • Spawning new virtual machines (VMs)

Update: 16:20

We rolled out a configuration change and are observing.

Update: 16:30

We see that VMs can be created again in our automated tests. We are closing the incident

Tuesday 3rd September 2024

MAINTENANCE: MetaKube Platform, scheduled 2 months ago

Affected Components: MetaKube Platform, all regions

Scheduled Start: 2024-09-03 18:00 UTC+01:00 (CET)

Scheduled End: 2024-09-03 21:00 UTC+01:00 (CET)

State: SCHEDULED


Description:

  • Scheduled maintenance of the MetaKube platform in all regions
  • Small update to MetaKube infrastructure

Customer Impact during the maintenance:

  • Short interruptions (<5min) of the control plane possible (e.g. controller-manager, scheduler), but not API
  • Running workloads will not be affected
  • Network availability will not be affected
  • Storage availability will not be affected

Customer Actions:

  • No customer actions needed
  • Please inform us if you notice any irregularities

Monday 2nd September 2024

Cloudflare related network problems

Affected Components: Setups that route traffic through Cloudflare experience issues

Investigation Start: 2024-09-02 13:20 UTC+01:00 (CET)


Description:

We are investigating network problems reported by customers that route traffic through Cloudflare


Customer Impact:

Customers that are routing traffic to SysEleven networks through Cloudflare experience high retransmission rates and traffic loss.


Update: 2024-09-02 14:20 UTC+01:00 (CET)

We are still investigating.


Update: 2024-09-02 14:49 UTC+01:00 (CET)

We are observing significant improvements of the retransmit rates on some affected loadbalancers but will continue to observe and analyse the issues.


Update: 2024-09-02 16:25 UTC+01:00 (CET)

Since the Problem seems to be solved for most customers, we declare the incident as solved. Further investigations into the reasons for the problems will continue.

Sunday 1st September 2024

No incidents reported

Saturday 31st August 2024

No incidents reported

Friday 30th August 2024

No incidents reported

Thursday 29th August 2024

No incidents reported

Wednesday 28th August 2024

No incidents reported

Tuesday 27th August 2024

No incidents reported

Monday 26th August 2024

No incidents reported