Non-responsive web app
Incident Report for Clayton
Postmortem

At 7.47 GMT we started investigating a high error rate in our application, that was seemingly causing a very slow response when browsing our web application, as well as a high number of API timeouts and significantly slower-then-normal code review performances.

As per our incident management process, we have started reviewing all application and database logs looking for a possible cause of such performance degradation.

We have later identified that some of our database indexes weren't updated correctly since our latest platform upgrade. We have then restored all database indexes manually.

As part of our post-mortem assessment, we will be updating our release management process to ensure database indexes are checked, to prevent similar problems from happen again in the future.

Posted Jan 11, 2019 - 16:47 GMT

Resolved
This incident has been resolved.
Posted Jan 11, 2019 - 16:46 GMT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Jan 11, 2019 - 16:00 GMT
Investigating
We are currently investigating this issue.
Posted Jan 11, 2019 - 07:47 GMT
This incident affected: Web.