We've been wanting to answer "which CI/CD platform is actually fastest?" with real numbers rather than marketing claims, so we ran a benchmark on Semaphore, GitHub Actions, GitLab CI, Buildkite, and CircleCI.
Workload: Redmine (open source Ruby on Rails app) — dependency install + full test suite. Same workload, same OS, same Ruby version, same DB backend across all platforms.
Hardware: 2 vCPU machines on all providers (matched as closely as possible).
Method: 10 consecutive runs after cache warmup. No outliers removed.
Results:
| Platform | Avg Build Time | Cost/Run |
|---|---|---|
| Semaphore | 5:01 | $0.04 |
| Buildkite | 7:15 | $0.09 |
| Github Actions | 9:44 | $0.06 |
| Gitlab CI | 11:15 | $0.11 |
| Circle CI | 13:18 | $0.08 |
CircleCI's variance was notable — runs ranged from 9:25 to 17:08, which makes it hard to plan deploys around.
For a team running 100 builds/day, the difference between Semaphore and GitHub Actions is roughly 6.5 engineer hours saved daily.
Full methodology, raw run data, and cost breakdown here: https://semaphore.io/semaphore-ci-cd-benchmark-performance-and-cost-analysis
Disclosure: I work at Semaphore. I'm sharing this because the methodology is open and reproducible — the workload is public (Redmine on GitHub), so you can run it yourself. Happy to answer questions about how we ran this.
What CI/CD are you running and what build times are you seeing on similar workloads?