With scrum, metrics can be powerful tools for planning, inspecting, adapting, and understanding progress over time. Rates of success or failure can let a scrum team know whether it needs to make positive changes or keep up its good work. Time and cost numbers can highlight the benefits of agile projects and provide support for an organization’s financial activities.
Metrics that quantify people’s satisfaction can help a scrum team identify areas for improvement with customers and with the team itself. Following are ten key metrics to help guide scrum project teams.
Sprint goal success rates
One way to measure scrum project performance is with the rate of sprint success. The sprint may not need all the requirements and tasks in the sprint backlog to be complete to minimally realize the sprint goal. However, a successful sprint should have a working product increment that fulfills the sprint goal and meets the scrum team’s definition of done: developed, tested, integrated, and documented.
Throughout the project, the scrum team can track how frequently it succeeds in reaching the sprint goal and use success rates to see whether the team is maturing or needs to correct its course. Teams should always be stretching themselves, so 100 percent of the sprint backlog is not necessarily the goal. Sprint success rates are a useful launching point for inspection and adaptation.
Velocity is not a goal; it is a postsprint fact. An increasing velocity and an increasing sprint goal completion rate are both key indicators that a scrum team is continually improving efficiency.
Defects
To be truly agile, scrum teams need to implement agile practices like test-driven development (TDD) and continuous integration (CI). Without quality practices like these, scrum teams will be ineffective at delivering quality as fast as the market demands because of the overhead of manual testing before each release and the amount of defects introduced that automation could easily catch.
It’s unlikely that any scrum team will be able to accomplish perfection in these areas, so any project is likely to have some defects. Agile techniques combined with the scrum framework help development teams proactively minimize defects.
Tracking defect metrics can let the development team know how well it’s preventing issues and when to refine its processes.
The number of defects and whether defects are increasing, decreasing, or staying the same are good metrics to spark discussions on project processes and development techniques at sprint retrospectives.
Time to market
Time to market is the amount of time that a project takes to provide value by releasing working products and features to users. Organizations may perceive value in a couple of ways:
When a product directly generates income, its value is the money it can make.
When a product is for an organization’s internal use, its value will be the employees’ ability to use the product and will contain subjective factors based on what the product can do.
When measuring time to market, consider the following values:
Measure the time from the project start until you first show value to the customer.
Some scrum teams deploy new product features for use at the end of each sprint. For scrum teams with a release with every sprint, the time to market is simply the sprint length, measured in days.
Other scrum teams plan releases after multiple sprints and deploy product features in groups. For scrum teams that use longer release times, the time to market is the number of days between the start of development of a feature and the next release.
Time to market helps organizations recognize and quantify the ongoing value of scrum projects. Time to market is especially important for companies with revenue-generating products, because it aids in budgeting throughout the year.
Return on investment
Return on investment (ROI) is income generated by the product, less project costs: money in versus money out. On scrum projects, ROI is fundamentally different from ROI on traditional projects. Scrum projects have the potential to generate income with the very first release (which can potentially be as soon as the end of the first sprint) and can increase revenue with each new release.
Like time to market, ROI metrics are a great way for an organization to appreciate the ongoing value of a scrum project. ROI metrics help justify projects from the start because companies may fund projects based on ROI potential. Organizations may track ROI for individual projects as well as for the organization as a whole.
Capital redeployment
When the cost of future development is higher than the value of that future development, it’s time for the project to end.
The product owner prioritizes requirements, in part, by their ability to generate revenue. If only low-revenue requirements remain in the backlog, a project may need to end before the scrum team has used its entire budget. The organization may then use the remaining budget from the old project to start a new, more valuable project. The practice of moving a budget from one project to another is called capital redeployment.
To determine a project’s end, you need the following metrics:
The value (V) of the remaining requirements in the product backlog
The actual cost (AC) for the work to complete the requirements in the product backlog
The opportunity cost (OC), or the value of having the scrum team work on a new project
When V < AC + OC, the project can stop. The cost you will sink into the project will be more than the value you will receive from either continuing the project or starting the next project.
Capital redeployment allows an organization to spend efficiently on valuable product development and maximize the organization’s overall ROI.
Satisfaction surveys
A scrum team’s highest priority is to satisfy the customer, both early and often, by delivering value. At the same time, the scrum team strives to motivate individual team members and promote sustainable development practices.
A scrum team can benefit from digging deeper into customer and team member experiences. One way to get measurable information about how well a scrum team is fulfilling its purpose is through satisfaction surveys:
Customer satisfaction surveys measure the customer’s experience with the project, the process, and the scrum team.
Team satisfaction surveys measure the scrum team members’ experience with the organization, the work environment, processes, other project team members, and their work. Everyone on the scrum team can take team surveys.
You can put together informal paper surveys or use one of the many online survey tools. Some companies even have survey software available through their human resources department.
Team member turnover
Scrum projects tend to have higher team member morale. One way of quantifying morale is by measuring turnover. You can look at the following metrics:
Scrum team turnover: Low scrum team turnover can be one sign of a healthy team environment. High scrum team turnover (resulting from things like burnout, ineffective product owners who force development team commitments, personality incompatibilities, or a scrum master who fails to remove impediments, making the development team look bad in sprint reviews) can indicate problems with the project, the organization, the work, individual scrum team members, or overall team dynamics.
Company turnover: High company turnover, even if it doesn’t include the scrum team, can impact morale and effectiveness. High company turnover can be a sign of problems within the organization. As a company adopts scrum, it should see turnover decrease.
When the scrum team knows turnover metrics and understands the reasons behind those metrics, it may be able to take actions to maintain morale and improve the work environment.
Project attrition
Organizations with any size project portfolios should look at the rate of projects being cut short. Capital redeployment should not be confused with thrashing teams between projects at the whim of senior managers. Tracking project durations against capital redeployment analyses may expose trends of either ending projects prematurely or letting them run longer than needed, but most likely the former.
From these trends, portfolio managers can begin to look into reasons why they are getting cut short. High attrition may indicate such issues as thrashing, planning, prioritization, impediments, or cross-functionality.
Skill versatility
Strong scrum teams are typically more cross-functional than weaker scrum teams. By eliminating single points of failure in a scrum team, you increase its ability to move faster and produce higher quality. Tracking skill versatility allows scrum teams and functional managers to gauge growth of cross-functionality.
When starting out, capture the existing skills and levels contained at each of the following organizational structures:
Per-person skills and levels
Per-team skills and levels
Per-organization skills and levels
Over time, as each person increases the quantity and level of skills, each team and the organization will increase. It’s not about how many managers or directors you have by title in the organization that will deliver quality products and services to your customers. It’s about having team members who can all contribute to the sprint goal each day without the risk of single points of failure.
Manager:creator ratio
Typically, the larger the organization, the more likely a heavy middle layer of managers exists. Many organizations haven’t figured out how to function well without managers to handle personnel and training and development issues. However, you need to strike the right balance of managers and individuals who produce product.
Track your manager:creator ratio to help you identify bloat and ways to minimize the investment that you’re making in people who don’t create product.