[RESOLVED] Scorpion node emergency maintenance

News / Updates

Today is not our day 🙁

Our monitoring system has identified a critical hardware issue on Scorpion node indicating a possible impending disk failure.

For this reason, we have immediately turned off provisioning on this node and are moving all servers away from it so we can have the disk replaced and the node setup again, this node is expected to be unavailable for around to 24 hours.

We will update this when more information is available

Please keep in mind that our global server capacity is currently reduced below 50%, as two of our largest nodes are facing issues, this one and Neptune, so you might experience start queues longer than usual.

Update 1 @ 2022-11-06 0118Z: All servers have been safely moved away from the Scorpion node, the node has been shutdown and is waiting for hardware replacement by our provider, when the node is back a full OS install will be performed and some tests to confirm that the issue is fixed, once we can confirm the issue is no longer present it will be turned on again for usage.

Update 2 @ 2022-11-06 0228Z: Scorpion node SSD has been replaced by the provider, after reinstalling the node OS and performing some tests we can confirm the issue is fixed and the node is back online and ready for use 🙂