Last night, we began seeing abusive traffic, via both apache and mail, to themis. This combination, plus the overhead that is common on cpanel servers, resulted in high loads, individual service failures, and sluggish response times. After blocking IPs and restarting,all seemed fine on themis, and services recovered normally.
In the wee hours of the morning, we found that once again, abusive traffic was heading toward the server. At the same time, multiple hardware notices made their way out of the server to our monitoring address, indicating issues with a few components. These issues cannot be resolved while the server is in production.
Given the hardware notices, and the abusive traffic (which is not directed at any one site, but to the server directly and specifically), we are opting to make emergency migrations away from themis to other servers in the network. Sites without mailman mailing lists and sites that were not put on themis for other cpanel reasons will be relocated to DirectAdmin servers. Sites with mailman mailing lists and sites that were placed on themis for cpanel reasons will be relocated to other cpanel servers.
We will be sending notices; however, due to the severity of this issue, those notices will be sent fairly soon before a site move is being performed. Our apologies for this short notice timeframe, but the situation is severe enough to warrant immediate action.
If you have any questions related to this emergency migration, please contact us via the usual means. IMPORTANT: please contact us from an email address NOT at your domain, if that domain is on themis, given the issues there.
Wednesday, August 26, 2020