Monday 25th March 2024

Block Storage (Cinder Volumes / M1 Flavors) INCIDENT: SysEleven STACK issues in region DBL

Affected Components: SysEleven Stack, MetaKube clusters, MetaKube API, region DBL

Incident Start: 2024-03-25 10:18 UTC+01:00 (CET)

Incident End: 2024-03-25 13:00 UTC+01:00 (CET)


  • Major Outage of multiple Hardware Nodes
  • Occurring errors are currently being investigated.

Customer Impact:

  • Storage and Compute services in the DBL region are disrupted

Update : 10:39

  • One Rack in the DBL Datacenter seems to be down
  • Rack is getting investigated by remote hands

Update : 10:50

  • Datacenter operator confirmed power outage
  • Hardware nodes are coming back online
  • Follow up problems are being tackled

Update : 11:03

  • We are still stabilizing the affected systems
  • We see some recoveries coming in already

Update : 11:17

  • We still see issues on software loadbalancers (Octavia) which we are being investigated at the minute
  • We see a few CRC problems and will contact the affected customers individually after doing a deep scrub on the storage

Update : 11:40

  • Octavia loadbalancer recoveries and storage scrub tasks are ongoing