502 errors in UI - Builds Queueing
Incident Report for CircleCI
Resolved
Our job queue has returned to normal levels.
Posted Oct 28, 2019 - 23:33 UTC
Monitoring
Our backlog is now at normal levels. Due to limited capacity, our macOS jobs will experience delays due to backlogged jobs.
Posted Oct 28, 2019 - 23:09 UTC
Update
Our engineers have almost cleared our backlog, but due to limited capacity, our macOS jobs will still experience issues due to backlogged jobs.
Posted Oct 28, 2019 - 23:05 UTC
Update
Our engineers are currently working through the backlog and are actively working on improvements in builds queueing.
Posted Oct 28, 2019 - 22:17 UTC
Update
Our engineers have improved the throughput of our systems and are now working through the backlog more effectively.
Posted Oct 28, 2019 - 21:54 UTC
Update
Our engineers are still working to resolve this incident. Our systems are working through the backlog of queued builds.
Posted Oct 28, 2019 - 21:16 UTC
Update
Our engineers are still working to resolve this incident. Our systems are working through the backlog of queued builds.
Posted Oct 28, 2019 - 20:40 UTC
Identified
We have identified the issue and our engineers are currently deploying a fix. The backlog of builds queued is beginning to clear.
Posted Oct 28, 2019 - 20:18 UTC
Update
We are continuing to investigate. We have marked all potentially impacted components accordingly on our status page.
Posted Oct 28, 2019 - 19:33 UTC
Update
We are still investigating the cause of build queuing. Our engineers have stabilized our systems and some customers are able to see their builds running now. We will continue to update our status page.
Posted Oct 28, 2019 - 19:05 UTC
Update
Our engineers are continuing to investigate why builds are queueing.
Posted Oct 28, 2019 - 18:14 UTC
Investigating
Some users are unable to access our UI. We are currently investigating.
Posted Oct 28, 2019 - 17:50 UTC
This incident affected: Docker Jobs, Machine Jobs, macOS Jobs, Workflows, and CircleCI UI.