Some systems are experiencing issues

About This Site

Timezone: Europe/Berlin (CET - UTC +1/CEST - UTC +2)

We provide current information about infrastructure and service availability below. If you experience service impacts or performance issues please contact our helpdesk or our Service Desk.

Scheduled Maintenance
MAINTENANCE: SysEleven STACK announcement (DBL)

Affected Components: SysEleven Stack Networking in region DBL

Scheduled Start: 19 Nov 2024, 22:00 CET

Scheduled End: 20 Nov 2024, 00:00 CET

State: SCHEDULED

Description:

We will perform maintenance on networking gateways.

Customer Impact during the Maintenance:

  • Interruption of connectivity of virtual machines of up to 2 minutes
  • VPN connections could be interrupted for up to 2 minutes

Customer Actions:

  • no customer actions needed
  • please inform us if you notice any irregularities

Past Incidents

Wednesday 17th August 2022

No incidents reported

Tuesday 16th August 2022

SysEleven Stack network maintenance announcement in region DBL, scheduled 2 years ago

Between August 16th, 2022 22:00 and 24:00 (UTC +2) we will perform planned maintenance in Region DBL

During this period there will be:

  • network availability of virtual machines will be interrupted for up to 5 min
  • access to the API/Dashboard will not be affected
  • SEOS (S3) will not be affected
  • storage availability will not be affected

Should you have any trouble following the maintenance work please contact us.

Monday 15th August 2022

No incidents reported

Sunday 14th August 2022

No incidents reported

Saturday 13th August 2022

No incidents reported

Friday 12th August 2022

No incidents reported

Thursday 11th August 2022

No incidents reported

Wednesday 10th August 2022

No incidents reported

Tuesday 9th August 2022

SysEleven Stack DNS API Fixed: DNS resolution appears to be impaired

Fixed: Our external DNS resolution has been restored

Previous Message: Our DNS resolution is currently experiencing a major outage. We are investigating the issue. Due to this circumstance, further service outages may occur. Should you experience any outages or are in need of further assistance, please open a ticket or contact us via our emergency hotline. Our gitlab instance code.syseleven.de is also affected by the outage.

Fixed: Issues with SysEleven Observer (aka Grafana)

Issues with Cortex which results in reduced or no metrics in customers Grafana Dashboards (SysEleven Observer).

We are investigating the issue.

Update: 09:32h

The issue has been fixed and metrics are being written and are viewable again. However, metrics between ~02:00AM and 09:30AM of today will not be available.

If you still experience issues, please contact us via e-mail or phone.

FIXED: Major Incident - Internal DNS Resolving

This issue is resolved. We experienced several major outages associated with internal DNS resolution issues. Should you have been affected, please open a ticket in our ticket system.

NFS FIXED: Managed Platform NFS partial outage in DC BLU1

This is fixed. We experienced a service failure in our central storage appliance in our datacenter Berlin . This may have lead to problems in accessing NFS shares.

Managed Platform NFS partial outage in DC BKI1

This is fixed. We experienced a service failure in our central storage appliance in our datacenter Berlin . This may have lead to problems in accessing NFS shares.

Monday 8th August 2022

No incidents reported

Sunday 7th August 2022

No incidents reported

Saturday 6th August 2022

No incidents reported

Friday 5th August 2022

Fixed: External DNS provider is currently impaired

This incident has been resolved.

Past message: Our DNS provider is currently experiencing a major outage. We are investigating the issue. Due to this circumstance, further service outages may occur. Should you experience any outages or are in need of further assistance, please open a ticket or contact us via our emergency hotline.

Fixed: SysEleven Stack Object Storage issues in all regions

Potentially due to the previous DNS resolving problems we are facing issues with the Object Storage in all regions. At the moment it is possible that objects can't be read or written. We will keep you informed and apologise for any inconveniences.

Thursday 4th August 2022

No incidents reported

Wednesday 3rd August 2022

Virtual Machines Managed Platform (PVC) failure of multiple hardware nodes

Our Monitoring identified multiple hardware node failures.

Affected customers were notified via E-Mail. If interaction is required customers will be notified individually.

Please contact us via e-mail or phone if you encountered problems but did not receive any mail.

Update 16:30: All hardware nodes are running again. We are currently working on resolving all remaining alerts. Update 16:50: All websites are reachable and working again. We are still working on some remaining minor problems and follow up tasks.

Tuesday 2nd August 2022

No incidents reported

Monday 1st August 2022

Control Plane Services (SysEleven - DBL) MetaKube: Control Plane for certain customer clusters degraded

For a number of customer clusters, the Kubernetes controller-manager and the scheduler were not functioning. Running workloads were not affected, but no scheduling decisions could be made.

Incident over: 22:30

Sunday 31st July 2022

No incidents reported

Saturday 30th July 2022

No incidents reported

Friday 29th July 2022

No incidents reported

Thursday 28th July 2022

Control Plane Services (SysEleven - FES) Metakube network issues in region FES

At the moment we are facing network issues in region FES that also affect our control plane services.

We will keep you informed and apologize for any inconveniences.

Update 12:20 : We have identified and resolved an issue and are now verifying the solution. Update 13:05 : After a short watch time we have verified that the solution did fix the issue.

Reason for Outage (RfO) (de)

Wednesday 27th July 2022

No incidents reported

Tuesday 26th July 2022

No incidents reported

Monday 25th July 2022

No incidents reported

Sunday 24th July 2022

No incidents reported

Saturday 23rd July 2022

No incidents reported

Friday 22nd July 2022

No incidents reported

Thursday 21st July 2022

No incidents reported

Wednesday 20th July 2022

Network SysEleven Stack network issues in region FES

At the moment we are facing connectivitiy issues in region FES.

We will keep you informed and apologise for any inconveniences.

update Since 15:20 the issues have been resolved

Tuesday 19th July 2022

SysEleven Login login.syseleven.de experiencing issues ("realm not found")

We are experiencing some issues on login via login.syseleven.de

Update: The issue was identified and fixed, login is working again