Affected Components: MetaKube Control Plane Services, region FES
Incident Start: 2024-11-18 11:30 UTC+01:00 (CET)
Description:
Infrastructure hosting the MetaKube Control Plane Services has problems.
Customer Impact:
- MetaKube Control Plane might be slow or not answering
UPDATE 2024-11-18 12:30 UTC+01:00 (CET)
We have identified networking problems as the cause, currently working to resolve them.
UPDATE 2024-11-18 13:27 UTC+01:00 (CET)
We have increased conntrack table size on hardware nodes to avoid networking problems.
We continue to have issues with overloaded pods which we are working on.
UPDATE 2024-11-18 14:00 UTC+01:00 (CET)
We managed to get the overloaded pods running by isolating them on dedicated nodes and raising the resource limits. This stopped other issues as well.
We still need to investigate what caused the overloading of certain pods.
Incident is over.
