API Service Disruption
Incident Report for Copper
Postmortem

At 11:32 AM PST / 2:32 PM EST our monitoring systems alerted us to increased error rates. We lost the ability to access Google’s GMail API from within Copper. The API is a core part of our service. We contacted Google and were advised that a fix was being implemented.

At 3:30 PM PST / 6:30 PM EST Google informed us that they had completed the fix and Copper was able to connect to the GMail API. Service returned to normal.

We will continue to work with Google to understand how this happened and what steps will be taken to mitigate future incidents.

Posted Jul 06, 2020 - 16:47 PDT

Resolved
The incident has been resolved. We’re sorry for this inconvenience. We will provide more information on the root cause of this incident as soon as possible.
Posted Jul 06, 2020 - 15:32 PDT
Identified
We’ve learned that one of our partners rolled out code changes that impacted the Gmail API. This also impacted Copper’s API. We have confirmed this will be fixed within the next several hours.

The next update will be provided once service is restored. Thank you for your continued patience.
Posted Jul 06, 2020 - 13:52 PDT
Investigating
We’re currently experiencing a service disruption with our API. Our team is working to identify the root cause and implement a solution. All users may be affected.

The next update will be provided within 60 minutes. Thank you for your patience as we work to fix this issue as quickly as possible.
Posted Jul 06, 2020 - 12:44 PDT
This incident affected: Google Sync, CRM for Gmail Chrome Extension, Reporting, Custom Reports Builder, Developer API, and Rest API & Web Application.