Past Incidents

At 15:11 CEST we experienced an unexpected restart of services that are part of our software defined networking (SDN).

This led to 30 second downtime for some VMs and VPN services in DBL.

Update 16:00: We have found the root cause and will make sure that this will not happen again.

We are currently facing a failure in outgoing DNS resolving. We are in contact with the supplier and are working on a solution with high pressure.

Update: The issue has been resolved.

We received reports for an intermittent partial outage in networking and storage in DBL. Total duration was around minute.

The issues are resolved already. We are looking into what caused these issues.

Update 10:36: We identified an issue in our switching fabric. The issues were caused by the short time the network took to reconverge.

Update 10:42: We observed slow IO in our distributed storage layer. We are investigating.

Update 11:11: All systems are operational again. We are watching closely and continue investigating.

Update 11:28: We observed slow IO in our distributed storage layer. We are investigating.

Update 11:38: We identified the root cause in our storage layer. All issues are now resolved. We will conduct a post-mortem to make sure this will not happen again.