Tuesday 19th April 2022

Partial Outage at DC Berlin Lützowstraße

We are facing problems at our datacenter location Berlin Lützowstraße. Some systems are currently unreachable, including the Openstack API. We are investigating the problem.

Update 2022-04-19 15:45: Our datacenter provider reported an external power outage at our datacenter Berlin Lützowstraße. While most systems were not affected, some have been shut down unexpectedly. Power supply has been recovered and systems are being ramped up again.

Update 2022-04-19 15:53: Several systems are recovering. We are still working on the recovery of all affected systems.

Update 2022-04-19 16:32: Part of the Openstack nodes are still not operational. This affects the availability of the cluster's network storage. We are still working on the problem with highest priority.

Update 2022-04-19 17:57: Recovery of underlying services such as Quobyte (SDS) and Midonet (SDN) to full functionality is still ongoing. All involved teams are working on the problems with high pressure and we will keep you updated with the progress.

Update 2022-04-19 19:45: All services have been recovered to work, which includes Network, Storage, API, DNS, and Loadbalancers. The consistency checks of existing data are still running.