Downtime

Europe Node 4 SMTP, Europe Node 4 IMAP, Europe Node 4 POP3, Europe Node 4 Task Scheduler, Europe Node 4 CP, and 3 other services are down

Europe Node 4 SMTP and Europe Node 4 POP3 went down.

Downtime

Europe Node 4 SMTP and Europe Node 4 CP are down

Europe Node 4 CP went down.

Downtime

Europe Node 1 POP3 and Europe Node 1 CP are down

Europe Node 1 CP went down.

Downtime

Europe Node 5 Web, Europe Node 1 SMTP, Europe Node 2 SMTP, Europe Node 4 SMTP, Europe Node 5 CP, and 3 other services are down

Europe Node 5 Web went down.

Downtime

Europe Node 1 Backup is down

Europe Node 1 Backup went down.

Downtime

Europe Node 4 SMTP, Europe Node 1 Backup, Europe Node 2 Backup, Europe Node 5 CP, and 2 other services are down

Europe Node 2 Backup went down.

Resolved

Unavailable websites after monthly check

A service called cgclassify errors out after monthly check, leading to PHP-FPM units to fail to start. Websites appear to be unavailable as requests to be processed by PHP can't be executed.

While we're investigating on a subset of affected nodes, we went ahead and performed reboots on client production nodes.