Resolved. Maintenance on US Node-002 is complete. Servers on that node are back online with additional memory available. No server files or world data were changed.

We will carry out brief planned maintenance on US Node-002 on Wednesday 16 September at 5:00 AM Central time to increase the memory available to servers on that node. Servers on US Node-002 will be offline for a few minutes and will start again automatically once the work is complete. No other nodes are affected, the game panel and billing portal stay available throughout, and no server files or world data are changed.

This incident is now resolved. Connectivity to our European region has been fully restored and servers in that region are reachable and running normally again. Game servers were not lost or restarted during the disruption, so worlds and files are intact. The interruption was a network connectivity fault affecting reachability of the region rather than a fault with the servers themselves. We are continuing to review the cause with our upstream provider to reduce the chance of a repeat, and we apologise for the disruption.

We are aware that game servers hosted in our European region are currently unreachable, and that the control panel is unable to reach them. Customers with servers in this region will not be able to connect to or manage their servers during this time. Servers in our other regions are unaffected and running normally. We are actively investigating and will post a further update as soon as we have more information.

Resolved: EU Node-001 is back online and all servers on it are running normally again.

We have verified the node is healthy, game traffic is flowing, and customer servers have returned. No customer data was lost. Server files, worlds and backups on the node are intact exactly as they were before the outage.

The cause was an administrative account issue on our side with one of our upstream infrastructure providers. It was not a hardware fault and not a security incident. It was avoidable, and it was our mistake.

We are putting checks in place so this particular failure cannot repeat.

If your server does not start, or anything looks wrong with your files, please open a ticket and we will sort it out for you directly.

Thank you for your patience through a long and frustrating outage. We are genuinely sorry.

Identified: we have determined the cause of the EU Node-001 outage. It is an administrative account issue with one of our upstream infrastructure providers, not a fault with the hardware, the node, or any customer data. This was our responsibility, not a failure on the provider's side.

We are actively working to resolve it now. Once the account matter is settled we expect the node and all servers on it to come back up without further intervention.

Customer data is unaffected. Server files, worlds and backups on EU Node-001 are intact and have not been touched by this at any point.

US Node-001, US Node-002, the game panel, the client API and billing all remain fully operational.

We are sorry. This outage was avoidable and it is on us. We will post another update as soon as the node is back or if anything changes.

Update: EU Node-001 remains offline and unreachable. Our investigation is ongoing and we do not yet have a confirmed cause, so we are not going to give a restoration estimate we cannot stand behind. We would rather tell you what we actually know.

What we know: connectivity to the node was lost at approximately 10:30 UTC and has not returned. The node has not been reachable by any of our access paths since then.

Customer data is not affected. This is a connectivity and availability issue. Server files, worlds and backups on the affected node are intact, and nothing has been lost.

Unaffected and running normally: US Node-001, US Node-002, the game panel, the client API and the billing portal.

We are sorry for the disruption and for the length of this outage. We will post the next update as soon as we have either a confirmed cause or a restoration time.

Servers hosted on EU Node-001 are currently offline and cannot be reached. We identified the loss of connectivity as beginning at approximately 10:30 UTC today and we are actively investigating.

Services on US Node-001 and US Node-002, the game panel, the client API and billing are all unaffected and operating normally.

Customer data on the affected node is not impacted by this. We will post a further update as soon as we have more information.

The cause was resource exhaustion on one of our front-end systems. Capacity has been increased, and control panel, billing and webmail access are restored and verified