On June 16th at 7:30 am Pacific time, we experienced a massive increase in the number of events that trigger webhook callbacks within our platform. This caused a delay in the processing of webhook callbacks and for these to become queued up.
Through our alerting system, we were notified of the event and moved to increase capacity in the platform. By 1:00 pm PT the queued webhook callbacks had been processed and the platform callbacks were back to normal levels.
We are still investigating what underlying customer behaviour drove this significant change in the number of webhook callbacks being raised. Currently, we believe this to have been a one-off customer event.
We have left the platform with increased capacity while we work on a long-term solution. Please refer to release notes to see how any further changes we make in this area will have an impact on your current configuration.
Posted Jun 17, 2020 - 18:45 PDT
This incident affected: Integrations, Developer API, and Rest API & Web Application.