Affected Components: SysEleven Stack Object Storage, region FES, DUS2, HAM1
Incident Start: 2025-04-14 16:26 UTC+01:00 (CET)
Description:
At the moment we are facing issues with the Object Storage in Region FES, DUS2, HAM1.
Customer Impact:
- Some objects might be corrupted regarding latest known issues in Ceph. https://docs.ceph.com/en/latest/releases/squid/#v19-2-2-squid
- We are investigating the situation and working on updates to mitigate the issues
Update: 18:40
- The Ceph S3 endpoint of the DUS2 region is currently being updated, after successful testing we will proceed with the HAM1 and FES region
Update: 19:45
- The update for the DUS2 region went through without problems, we are proceeding to update HAM1 and the FES region
- Affected customers will be contacted individually once we got a complete overview of potential affected buckets and objects
Update: 21:10
- Update of HAM1 region also finished, FES region is in progress
Update: 22:52
- Update of FES region is still ongoing
Update: 15-04-2025 09:36
- Update of FES region is still ongoing and everything works as intended
Update: 16:40
- Update of FES region is expected to be finished to the end of the business day
Update: 16-04-2025 03:08
