One of the most important DevOps metrics at companies of any size is the feature lead time - that is, the amount of time it takes for a feature to go from PR to production.
Lead times provide key insights into how a company's deployment process is structured, and how efficiently it can iterate on its product to meet consumer needs. Though lead times generally increase with the size of the feature being deployed, longer average lead times usually indicate that the development process is inefficient. There are likely bottlenecks in code review or testing, or the organization's structure itself may not be conducive to easily signing off on features.
It's hard to contextualize your own lead times, though, without knowing what goes into those of peer engineering teams. We recently conducted a survey with over 100 startups (from 2 to 200 engineers) to better understand what their lead times are like, and what makes up those lead times.
What is the typical amount of time from PR to production at your company?
The first thing to note is the distribution of lead times - note that ~75% of surveyed startups have lead times of more than a day, and almost half of them take more than 3 days to deliver a feature once it's been coded up.
What percentage of your lead time is inactive (e.g. a feature sitting in a queue)?
A great way to tell whether a lead time is indictive of inefficiency is to see how much of that lead time is spent idling. Note that, at over 80% of surveyed startups, some amount of lead time is spent idle - and at half of them, over a quarter of lead time is spent idling. If your deployment process involves idling, those queues will only get longer as your engineering team and feature volume grow, unless you restructure.
If you have inactive lead time, which of the following create the biggest queues (e.g. >1 day) for your features?
Over 70% of respondents have significant queues for manual QA. Furthermore, combining different kinds of automated testing queues, ~30-40% of surveyed companies see queues for non-manual testing processes as major contributors to long lead times. This isn't surprising; if an E2E test takes an hour to run, and your company has grown to 50 engineers actively submitting PR's, there's likely a wait time of at least 2 days just to get an environment to run tests on.
Curious about how you can reduce idling, eliminate queues, and expedite the PR-to-prod process? With Gallery, you can bring the superpowered CI processes of companies like Facebook and Lyft straight to your engineering team; Gallery reduces lead times dramatically by spinning up ephemeral environments for every PR and every test, so you can parallelize your manual QA and CI builds and avoid bottlenecking on a fixed set of cloud environments. It takes just 30 minutes to get started - schedule a call to learn more, or check out our docs and demo first!
- Gallery Team