Long provisioning times for VM and remote docker jobs
Incident Report for CircleCI
Resolved
We are marking the incident as resolved but realize that macOS builds will take a few hours to process the current backlog, as such that component will remain as degraded.
Posted Sep 17, 2018 - 22:23 UTC
Update
We are continuing to monitor for any further issues.
Posted Sep 17, 2018 - 22:08 UTC
Monitoring
We are seeing active provisioning of new VMs and the backlog is draining. We will continue to monitor the situation and update again in 30 minutes.
Posted Sep 17, 2018 - 21:55 UTC
Identified
We are scaling capacity to meet current recovery demands, some queueing is anticipated. We will update in 30 minutes.
Posted Sep 17, 2018 - 21:44 UTC
Monitoring
All jobs are executing normally. We will continue to monitor the stability and performance of the platform, and provide an update within 40 minutes.
Posted Sep 17, 2018 - 21:09 UTC
Update
VM, macOS 2.0 and remote docker jobs are not executing. Our engineers are working to mitigate the issue. We will provide an update within 20 minutes.
Posted Sep 17, 2018 - 20:53 UTC
Update
VM and remote docker jobs are not executing. Our engineers are working to mitigate the issue. We will provide an update within 20 minutes.
Posted Sep 17, 2018 - 20:33 UTC
Identified
A regression has been identified and we are rolling back to a known good version. We will provide an update within 20 minutes.
Posted Sep 17, 2018 - 20:11 UTC
Investigating
Some VM and remote docker jobs are not being executed in a timely manner. We are investigating.
Posted Sep 17, 2018 - 20:09 UTC
This incident affected: Docker Jobs and macOS Jobs.