Here you will find real-time information about the state of our services.
A production database exceeded internal memory limits and internally corrupted several indexes. This fault did not cause an immediate outage on the database, but rather short periods of inaccessibility that gradually increased in duration. The index corruption issue was persisted through backups, complicating the restore process. Eventually, an uncontaminated backup was from approximately 24 hours prior to the beginning of instability was found. Once this had been restored to provide stability; we were able to begin diagnosing the issue on the corrupted production database. It was possible to recover the contents of the production database; and these have since been merged. A small set of data, primarily custom images, could not be restored past the backup window, and may be reverted.
Jun 10, 06:41 - Jun 10, 11:32
Get an email notification whenever Wetu creates or updates an incident.