Every software methodology tries to incorporate estimates/actuals and variance reports. By the time you get to SCRUM or very agile methodologies, you should be measuring your team’s velocity. You might be looking at the backlog count & estimates, the tasks per sprint, time for a sprint and the number of sprints … but you are measuring.

In my career, one of the biggest lessons is measure everything. If you don’t measure your project (a) you can only guess at completion dates; (b) you will not improve; (c) you will have issues with management and/or customers. I’m not talking about measuring the number of lines of code or anything that crude – I’m talking about measuring what’s important to your organization.

I have heard it argued that the difference between a software development team and a software engineering team is that engineers track and measure their results scientifically. You might argue that the creative side of software design is more an art than a science but software development can be measured and you (your team) will be better off for doing that measurement, and reviewing that data to improve the next time through a software cycle. Measurement is good. Even Art is comparatively measured, so be creative and chose the metrics that fit your team, but take the time to measure.

For improving development of requirements, I suggest you measure the number of tasks per requirement (as defined by an Epic or Story). During your sprint/release retrospective meeting review the metrics and decide if requirements are clear enough or need to be broken into separate pieces. This will improve the product owner’s ability to define the requirements and will clarify the outstanding backlog items related to this requirement. You can also review the requirements outstanding during the sprint retrospective or at the end of the sprint demo meeting to determine if they are still needed and relative order of importance. Reducing requirements and measuring that over the course of the sprints to release gives you a good measure of the focus and respective understanding of your audience.

If you start with 20 requirements for a product and only deliver on 10, or more typically, start with 10 and go to 20, perhaps your method for collecting those requirements should change. One way to do that is with better Voice of the Customer requirements gathering or appointing several of the technical sales team to represent your primary audience in coordination with your product owner, so you avoid feature creep. Beware however, that during development insights can be gleaned from showing a sprint demo’s results that lead to innovative ideas – you don’t want your process to stymie innovation – revising a requirement during iteration is one of the best characteristics of Scrum!

I like to measure the backlog items outstanding vs sprint items accomplished. Scrum is all about velocity. Completing more tasks of the same difficulty as compared to earlier sprints, is a great measure of your velocity. Of course, this can have a subjective component unless time estimates vs actuals on tasks are tracked and measured.

For improving sprint planning, your retrospectives should always review the task estimates vs actuals. As you are planning a sprint, you should add in-context estimates (estimates based on what has been done before, and what dependencies exist for this task in this sprint). Another related metric is the delta between initial estimate in the backlog item and the in-context estimate during the sprint planning. The accuracy of those estimates are essential to improving the sprint plans and managing scrum team members’ ability to estimate improves their skills as software engineers.

There is also a whole category of testing metrics that are necessary – test failures, escape rates, blocking issues, static analysis results, code reviews, code to test failures (for weak areas in the code), repeat failures (the number of develop/test cycles for a task), build failures, etc. All these are essential to good quality code and can be an objective way to look at effectiveness of your team.

Warning: metrics should always be used to improve your team and product releases, not as a weapon for performance discussions. If you start evaluating your team’s performance, salary or bonus on the process metrics you will almost always see your team “play” the metrics which will have a terrible effect on your team and product release. Rather, use the metrics above to establish a path for each member’s success and remember that your primary goal should be to produce high quality innovative product that thrills customers. If you know your team well enough you will assign the right backlog items to the right people and over time raise the performance of every member.