Resolved -
Build start times for all executor types have returned to normal. We are continuing to monitor the situation for reoccurrence.
Mar 27, 21:48 UTC
Monitoring -
We are continuing to monitor the queue for macOS builds.
Mar 27, 21:43 UTC
Update -
The Incident has been resolved for the Docker and Machine executors. We are monitoring the queue for macOS builds.
Mar 27, 21:38 UTC
Update -
macOS builds have begun running however we are seeing degraded performance as the queue is processing.
We are seeing degraded performance with Google Cloud Platform Google Compute Engine.
Mar 27, 21:02 UTC
Update -
We are continuing to work on a fix for this issue.
Mar 27, 20:48 UTC
Update -
We are continuing to work on a fix for this issue.
Mar 27, 20:03 UTC
Update -
We are continuing to work towards a fix for the inability to start macOS builds.
Mar 27, 19:34 UTC
Update -
We are continuing to work on a fix for this issue.
Mar 27, 19:07 UTC
Update -
We are continuing to work on a fix for this issue.
Mar 27, 18:38 UTC
Update -
We are continuing to work on a fix for this issue.
Mar 27, 18:12 UTC
Update -
We are continuing to work on a fix for this issue.
Mar 27, 17:40 UTC
Update -
We have disabled a replica sync process that appeared to be placing extra load on Mongo.
Mar 27, 17:13 UTC
Update -
macOS builds are running however builds may queue over the next few hours as we process existing builds.
Mar 27, 16:52 UTC
Update -
We are continuing to work on a fix for this issue.
Mar 27, 16:47 UTC
Update -
We are continuing to work on a fix for this issue.
Mar 27, 16:40 UTC
Identified -
The issue has been identified and a fix is being implemented.
Mar 27, 16:24 UTC
Investigating -
We are investigating the issue, the impact is that some jobs are being impacted across all build types.
Mar 27, 16:21 UTC