https://syseleven-status.de 2025-07-15T21:04:25.911987+00:00 SysEleven support@syseleven.de python-feedgen https://www.syseleven.de/wp-content/uploads/2020/10/SysEleven_XL_Logo_quer_RGB.png Get all incidents by feed 556 2025-07-15T21:04:26.073747+00:00 <p>Affected Components: <strong>SysEleven Stack, region FES</strong></p> <p>Incident Start: <strong>2025-03-08 13:00</strong></p> <p>Incident End: <strong>2025-03-08 15:30</strong></p> <hr /> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>IPv6 Connectivity is restricted</li> </ul> <hr /> <p><strong>Update: 16:25 </strong></p> <p>Flaky ipv6 connectivity was repaired around 15:30, a mismatch in configuration was identified</p> 2025-03-08T14:29:22+00:00 557 2025-07-15T21:04:26.069250+00:00 <p>Affected Components: <strong>SysEleven Stack API, region CBK</strong></p> <p>Incident Start: <strong>2025-03-13 14:30</strong> Incident End: <strong>2025-03-13 17:15</strong></p> <hr /> <p>Description:</p> <ul> <li>Accessibility of the SysEleven Stack API is not ensured.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Spawning new virtual machines (VMs) or changing existing resources is not possible.</li> </ul> 2025-03-13T14:43:01+00:00 558 2025-07-15T21:04:26.068597+00:00 <p>We're currently investigating an issue in our IAM systems that prevents creation and deletion of projects.</p> 2025-03-17T10:39:16+00:00 560 2025-07-15T21:04:26.066862+00:00 <p>Affected Components: <strong>All regions</strong></p> <p>Incident Start: <strong>2025-04-10 23:40 (CETS)</strong></p> <p>Incident End: <strong>2025-04-11 01:08 (CETS)</strong></p> <hr /> <p>Description:</p> <p>We have major outage in all regions</p> <hr /> <p><strong>Update: 01:01 am</strong></p> <p>Issue identified and we are working on it.</p> <hr /> <p><strong>Update: 01:08 am</strong></p> <p>Issue is resolved and we are whatching it.</p> 2025-04-11T00:22:15+00:00 561 2025-07-15T21:04:26.062731+00:00 <p>Affected Components: <strong>SysEleven Stack Object Storage, region FES, DUS2, HAM1</strong></p> <p>Incident Start: <strong>2025-04-14 16:26 UTC+01:00 (CET)</strong></p> <hr /> <p>Description:</p> <p>At the moment we are facing issues with the Object Storage in Region FES, DUS2, HAM1.</p> <hr /> <p>Customer Impact:</p> <ul> <li>Some objects might be corrupted regarding latest known issues in Ceph. https://docs.ceph.com/en/latest/releases/squid/#v19-2-2-squid</li> <li>We are investigating the situation and working on updates to mitigate the issues</li> </ul> <hr /> <p><strong>Update: 18:40</strong></p> <ul> <li>The Ceph S3 endpoint of the DUS2 region is currently being updated, after successful testing we will proceed with the HAM1 and FES region</li> </ul> <p><strong>Update: 19:45</strong></p> <ul> <li>The update for the DUS2 region went through without problems, we are proceeding to update HAM1 and the FES region</li> <li>Affected customers will be contacted individually once we got a complete overview of potential affected buckets and objects</li> </ul> <p><strong>Update: 21:10</strong></p> <ul> <li>Update of HAM1 region also finished, FES region is in progress</li> </ul> <p><strong>Update: 22:52</strong></p> <ul> <li>Update of FES region is still ongoing</li> </ul> <p><strong>Update: 15-04-2025 09:36</strong></p> <ul> <li>Update of FES region is still ongoing and everything works as intended</li> </ul> <p><strong>Update: 16:40</strong></p> <ul> <li>Update of FES region is expected to be finished to the end of the business day</li> </ul> <p><strong>Update: 16-04-2025 03:08</strong></p> <ul> <li>Update of FES is over</li> </ul> 2025-04-14T16:51:56+00:00 562 2025-07-15T21:04:26.060382+00:00 <p>Affected Components: <strong>SysEleven Stack, region DBL</strong></p> <p>Incident Start: <strong>2025-04-17 10:00</strong></p> <p>Incident End: <strong>2025-04-17 10:45</strong></p> <hr /> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> <p>Update [2025-04-17 10:28 am]: Alloy (GUI) &amp; API Authentication HAM &amp; DUS, Metakube API in DBL &amp; CBK are also affected</p> <p>Update [2025-04-17 10:40 am]: DBL Cloud Gateways have been identified as possible source of malfunction, we are bringing them back to function, first services are back online, next Update at 11:00 am</p> <p>Update [2025-04-17 11:00 am]: All services are up and running, we are still watching all environments, incident is declared as ended</p> 2025-04-17T10:11:22+00:00 563 2025-07-15T21:04:26.058994+00:00 <p>Affected Components: <strong>SysEleven Stack, region CBK</strong></p> <h2>Incident Start: 2025-04-25 15:55:00 UTC+01:00 (CET)</h2> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>We are expirencing a partial outage of the Network where some connection might get dropped.</li> </ul> 2025-04-25T17:18:12+00:00 564 2025-07-15T21:04:26.058040+00:00 <p>Affected Components: SysEleven Managed Hosting Network partial outage, region BLU1</p> <p>Incident Start: 2025-04-25 15:55:00 UTC+01:00 (CET)</p> <p>Description: - Occurring errors are currently being investigated.</p> <p>Customer Impact: - We are expirencing a partial outage of the Network where some connection might get dropped.</p> 2025-04-25T17:46:54+00:00 565 2025-07-15T21:04:26.055936+00:00 <p>Affected Components: <strong>SysEleven Stack Network, region CBK</strong></p> <p>Incident Start: <strong>2025-04-29 17:30 UTC+02:00 (CEST)</strong> Incident End: <strong>2025-04-29 17:50 UTC+02:00 (CEST)</strong></p> <hr /> <p>Description:</p> <ul> <li>Network connectivity to cloud region CBK was impaired</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Connectivity is restricted</li> </ul> <hr /> <p><strong>Update: 17:59</strong></p> <p>Network connectivity issues were resolved around 17:50</p> 2025-04-29T18:01:27+00:00 566 2025-07-15T21:04:26.053459+00:00 <p>Affected Components: <strong>MetaKube Control Plane, region DUS</strong></p> <p>Incident Start: <strong>2025-04-30 16:00 UTC+02:00</strong></p> <hr /> <p>Description:</p> <p>The loadbalancer for the kubernetes apiserver does not properly work for some clusters.</p> <hr /> <p>Customer Impact: - Potential connection loss to the apiserver</p> <hr /> <p>Update: <strong>2025-04-30 16:30 UTC+02:00</strong></p> <p>We have recovered the affected clusters but haven't found the root cause. Right now we can't see any other affected clusters but will continue to monitor the situation.</p> <p>Update: <strong>2025-04-30 16:52 UTC+02:00</strong> We have identified and fixed the root-cause of the incident. All clusters are working properly again.</p> 2025-04-30T16:10:27+00:00 567 2025-07-15T21:04:26.051674+00:00 <p>Affected Components: <strong>MetaKube Control Plane, region HAM</strong></p> <p>Incident Start: <strong>2025-05-08 14:43 UTC+02:00</strong></p> <hr /> <p>Description:</p> <p>The loadbalancer for the kubernetes apiserver does not properly work for some clusters.</p> <hr /> <p>Customer Impact: - Potential connection loss to the apiserver</p> <hr /> <p>Update: <strong>2025-05-08 15:02 UTC+02:00</strong> We have identified and fixed the root-cause of the incident. All clusters are fully operational again.</p> 2025-05-08T15:20:50+00:00 568 2025-07-15T21:04:26.049908+00:00 <p>Affected Components: <strong>SysEleven Stack Network, region CBK</strong></p> <p>Incident Start: <strong>2025-05-08 16:45 UTC+02:00 (CEST)</strong></p> <p>Incident End: <strong>2025-05-08 17:00 UTC+02:00 (CEST)</strong></p> <hr /> <p>Description:</p> <ul> <li>Network connectivity to cloud region CBK was impaired</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Connectivity is restricted</li> </ul> <hr /> <p>Network connectivity issues were resolved around 17:00</p> 2025-05-08T17:09:28+00:00 569 2025-07-15T21:04:26.047735+00:00 <p>Affected Components: <strong>SysEleven MetaKube, region HAM and DUS</strong></p> <p>Incident Start: <strong>2025-05-09 09:30 UTC+02:00 (CEST)</strong></p> <hr /> <p>Description: - The cluster creation in DUS and HAM is currently not working. We are currently investigating.</p> <hr /> <p>Customer Impact: - No new clusters can be created</p> <hr /> <p>Update: <strong>2025-05-09 11:23 UTC+02:00 (CEST)</strong> - We are continuing to work on rectifying the situation</p> <hr /> <p>Update: <strong>2025-05-09 11:43 UTC+02:00 (CEST)</strong> - Currently testing a possibly fix.</p> <hr /> <p>Update: <strong>2025-05-09 12:53 UTC+02:00 (CEST)</strong> - We are currently in the process of applying the proposed fix.</p> 2025-05-09T10:36:19+00:00 570 2025-07-15T21:04:26.045841+00:00 <p>Affected Components: <strong>DBaaS in dus2</strong></p> <p>Incident Start: <strong>2025-05-09 12:00 UTC+02:00 (CEST)</strong><br /> Incident End: <strong>2025-05-09 12:10 UTC+02:00 (CEST)</strong></p> <hr /> <p>Description:</p> <ul> <li>We are currently seeing connectivity issues with DBaaS in our dus2 region</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Databases in dus2 are partly unreachable</li> <li>New Databases in dus2 can not be created</li> </ul> <hr /> <p>Update: <strong>2025-05-09 12:10 UTC+02:00 (CEST)</strong> - The issue has been identified and resolved</p> 2025-05-09T12:12:14+00:00 571 2025-07-15T21:04:26.044138+00:00 <p>Affected Components: <strong>SysEleven Stack, Block Storage, region cbk</strong></p> <p>Incident Start: <strong>2025-05-20 12:46 UTC+02:00 (CEST)</strong></p> <p>Incident End: <strong>2025-05-20 13:30 UTC+02:00 (CEST)</strong></p> <hr /> <p>Description:</p> <p>We are facing issues with the distributed block storage system in region cbk.</p> <hr /> <p>Customer Impact:</p> <ul> <li>Reduced performance of block storage</li> </ul> 2025-05-20T13:17:37+00:00 572 2025-07-15T21:04:26.042225+00:00 <p>Affected Components: <strong>Hardware node failure, region bki</strong></p> <p>Incident Start: **2024-05-28 17:15 UTC+01:00 (CET)</p> <hr /> <p>Description:</p> <ul> <li>Malfunction of a hardware node</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>During the period of restart, there will be a short interruption in the availability of the systems.</li> <li>Affected customers were notified via E-Mail.</li> <li>Please check the affected systems for their full functionality.</li> </ul> 2025-05-28T18:13:11+00:00 573 2025-07-15T21:04:26.040579+00:00 <p>Affected Components: <strong>SysEleven Stack API, region FES</strong></p> <p>Incident Start: <strong>2025-05-31 16:00</strong> Incident End: <strong>2025-05-31 16:30</strong></p> <hr /> <p>Description:</p> <ul> <li>Accessibility of the SysEleven Stack API is not ensured.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Spawning new virtual machines (VMs) or changing existing resources is not possible.</li> </ul> 2025-05-31T16:27:46+00:00 574 2025-07-15T21:04:26.038871+00:00 <p>Affected Components: <strong>SysEleven Stack API, region HAM1</strong></p> <p>Incident Start: <strong>2025-05-31 16:00</strong> Incident End: <strong>2025-05-31 16:30</strong></p> <hr /> <p>Description:</p> <ul> <li>Accessibility of the SysEleven Stack API is not ensured.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Spawning new virtual machines (VMs) or changing existing resources is not possible.</li> </ul> 2025-05-31T16:28:18+00:00 575 2025-07-15T21:04:26.037139+00:00 <p>Affected Components: <strong>SysEleven Stack API, region DUS2</strong></p> <p>Incident Start: <strong>2025-05-31 16:00</strong> Incident End: <strong>2025-05-31 16:30</strong></p> <hr /> <p>Description:</p> <ul> <li>Accessibility of the SysEleven Stack API is not ensured.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Spawning new virtual machines (VMs) or changing existing resources is not possible.</li> </ul> 2025-05-31T16:29:09+00:00 576 2025-07-15T21:04:26.034896+00:00 <p>Affected Components: <strong>SysEleven Stack, region XXX</strong></p> <p>Incident Start: <strong>2025-07-04 04:18</strong> Incident End: <strong>2025-07-04 04:45</strong></p> <p>Second Incident Start: <strong>2025-07-04 05:15</strong> Second Incident End: <strong>2025-07-04 05:42</strong></p> <hr /> <p>Description:</p> <ul> <li>Occurring errors are currently being investigated.</li> </ul> <hr /> <p>Customer Impact:</p> <ul> <li>Connectivity is restricted'</li> </ul> 2025-07-04T04:44:35+00:00