Past Incidents

At the moment we are facing timeouts in the Network (Neutron) API in region CBK. Managing network resources may not work at this time.

We will keep you informed and apologise for any inconveniences.

Update 2019-03-06 17:12: The issue has been resolved. We applied a bug fix and now the Neutron Network API in region CBK is fully available again.

Currently we are facing issues with the SysEleven Stack API.

We have noticed, that when creating a new Cinder Volume, it will be in "error" state immediately.

Cinder volumes that have been created before already are not affected.

We are working on a solution and keep you informed.

We apologise for any inconveniences.

UPDATE 16:56: The issues have been resolved. We are now investigating the root cause.

We observed an elevated rate of errors, creating local storage virtual machines (L1.* flavor).

Engineers are now looking into the problem and we will keep you informed here.

UPDATE 15:45: Creating L1.* virtual machines currently requires unusual amounts of disk space on the hypervisors. The elevated error rate was because not enough space was left, for short amounts of time. Our engineers have resolved this issue for now, and are still working on a permanent solution.

Please contact us at cloudsupport@syseleven.de if you are still encountering any issues.

At the moment we are facing storage issues in region DBL. Some IO requests may take longer than usual.

We will keep you informed and apologise for any inconveniences.

Currently we are facing issues with the SysEleven Stack API in the DBL Region. We are working on a solution and keep you informed.

The outage only affects spawning new virtual machines or changing existing resources.

We apologise for any inconveniences.

UPDATE 00:30:

We have identified the cause of the API issues in DBL and resolved the issues for most API operations.

Due to a bug in OpenStack API components, the DBL API services used CBK identity URLs for validating tokens, which did not work due to the planned maintenance.

We changed all internal identity URLs in the endpoint registry to region-agnostic URLs.

We are now working on fully resolving this issue for all API operations in DBL.

UPDATE 00:50: All issues are resolved.

Currently it will take a long time to create servers with the Local SSD Storage (L1.*) flavors. The wait time increases linearly to Local SSD Storage capacity of the chosen flavor. Eventually, creating the server will succeed.

This is due to a mitigation we put in place that solves sporadic performance problems, when a virtual machine has been created or deleted. We are right now working on a better mitigation, that will not affect creating servers.

The issue only affects spawning new virtual machines with the L1.* flavor (Local SSD Storage).

We apologise for any inconveniences.

UPDATE 2019-02-13: The issues are fully resolved now. Creating and deleting Local SSD Storage instances will only take a few seconds now, and the time for creating and deleting VMs is constant, no matter what disk capacity.