Past Incidents

Affected Components: Metakube API, region CBK and FES

Incident Start: 24th Nov. 2022/12:15 pm

Description:

We are currently experiencing a partial outage with control plane services in our DBL region. This may result in issues when accessing the Metakube API in the Regions CBK and FES as well. We will update you shortly.

Update 01:10 pm:

Metakube API timeouts have been resolved.

Update 01:35 pm:

All arisen issues have been mitigated and we continue to monitor the issue at hand. Should you experience any limitations or reduction / disruption please contact us.

Update 04:09 pm: We were able to fix the underlying issue with an update. We also haven't seen any issues with any control planes.

Affected Components: SysEleven Network, SysEleven API, all regions potentially affected**

Incident Start: 19th Nov. 2022/07:35 pm

Description:

We are currently experiencing issues within our network. Should you be affected, please contact us via our emergency hotline. Opening a ticket prior to calling us, helps us to evaluate, rectify and resolve issues that may persist with your setup.

Update 09:10 pm: All issues have been resolved. As far as we know at the moment, one of the couters at our Berlin Luetzowstraße datacenter was unfunctional and needed to be resetted. These routers are redundant, however, there was impact seen shortly after the failure of the router until all connections successfully moved to the redundant system.

Affected Components: SysEleven Network Stack, region DBL

Incident Start: 17th Nov. 2022/02:45 am Incident End: 17th Sept. 2022/04:30 am

Description:

• We experience network outage on our gateways which blocks all network traffic for DBL

Customer Impact:

• Connectivity is lost

Update: 04:20 am

The problem has been identified and network is back recovered.

Affected Components: Monitoring Icinga2 problems, region BKI/BLU

Incident Start: 14th Sept. 2022/03:30 am

Incident End: 14th Sept. 2022/5:45 am

Description:

• Problem with icinga monitoring system
• expected false positive alarms

Customer Impact:

• false positive alarms
• potential missed alarms

Affected Components: Hardware node failure, region BLU1

Incident Start: 10th Nov. 2022/10:17 am Incident End: 10th Nov. 2022/10:35 am

Description:

• Malfunction of a hardware node
• Restart of the hardware node was necessary

Customer Impact:

• During the period of restart, there was a short interruption in the availability of the systems.
• Affected customers were notified via E-Mail.
• Please check the affected systems for their full functionality.

Affected Components: SysEleven CloudObserver

Scheduled Start: 08. Nov. 2022/01:45 pm Scheduled End: 11. Nov. 2022/06:00 pm

State: SCHEDULED

Description:

• We are migrating various components of the SysEleven CloudObserver from Cortex to Mimir.

Customer Impact during the Maintenance:

• No customer impact expected

Customer Actions:

• No customer actions needed
• Please inform us if you notice any irregularities

Affected Components: MetaKube Core

Scheduled Start: 07. Nov. 2022/07:00 pm Scheduled End: 07. Nov. 2022/10:00 pm

State: Completed

Description:

We're updating the management systems of MetaKube Core to keep compatibility with future Kubernetes versions.

Customer Impact during the Maintenance:

• MetaKube API will be unavailable
• MetaKube Dashboard will be unavailable

The control planes or nodes of existing clusters will not be affected.

Update:

10:00 PM: Everything is going according to plan, except it's taking a little bit longer than anticipated. The expected end time is now 08. Nov 2022 12:00 AM.

11:45 PM: We completed the maintenance. All MetaKube Core functions are available again.