Measuring the effectiveness of agile processes

What problems arise in agile teams and how to measure them?

Swapna M
Product Coalition

--

Product Agility

As our product scales, team size grows and we adopt more processes and infrastructures, the complexities to which they adhere to become immense. Sometimes processes malfunction, team is devoid of motivation or lacks the necessary impetus to deliver the best version of the product.

This can be due to a variety of factors — lack of vision from the management team, mismanagement of resources, lack of budget and resources, dearth of leadership and many more.

In this article however, we look at some of the obstacles that arise in agile delivery teams, how to measure if they’ve improved and a detailed analysis of one of the solutions we implemented within our team.

Problems to solve and processes to enhance

Some of the problems we wanted to resolve and processes we wanted to enhance were —

  • Standardize how we work across teams — align how we work as a holistic team
  • Clarify Roles & Responsibilities — between team members to avoid duplication of work and redundancy
  • Obtain input earlier in the story creation — to avoid blockages/issues with stories close to the sprint planning or within the sprint
  • Increase Team Engagement and ownership — to make the team super motivated to take on their tasks and feel a part of the larger goal
  • Avoid team overcommitting — improve estimation of stories and tasks
  • Standardize Reporting — to avoid different reporting styles across multiple teams
  • Improve or relook the Ratio of Dev/QE — to avoid having bottleneck of stories in the QE lane
  • Improve production support — to provide product owners look at more strategic tasks
  • Detailed requirements — requirements are to be changes at a minimum and need to be fleshed out cunningly well.
  • Empowerment and Decision making rights — More of a broader discussion with the management team, but empowering the solution team to raise questions, concerns and feedback and empowering them to make collaborative decisions is key to have a shared product responsibility.

Establish a baseline and measure improvements

After understanding the issues to solve and the job to be done within our solution/delivery team, how did we quantify the issues today and measure them against any improvements that we proposed?

Some of the success criteria of improvements were —

  1. Quality — Number of bugs arising out of stories delivered in that sprint
  2. Cycle time — average time from when the ticket starts to when the ticket is done
  3. ‘Planned’ to ‘Done’ ratio — Number of stories planned that are completed divided by the number of stories that were committed to
  4. Number of User Stories groomed prior to Sprint Planning

The last metric we found particularly interesting — Number of user stories groomed prior to sprint planning. We had a very low ratio of the number of fully fleshed-out or groomed user stories prior to the start of the next sprint.

A user story is said to be fully groomed (even if business requirements, design/content, data mapping, etc. were completed) is when it is presented and reviewed with the solution team (engineers & testers) to get their buy-in and to identify any gaps in the story.

If the team thinks that the user story has sufficient detail, only then they should estimate it; otherwise, further grooming is required.

Traditionally, this grooming task is done during Sprint Planning. The issue with this approach is that there is no time to close any grooming gaps prior to scoping the story into the sprint. Hence closing the gaps during sprint execution becomes a disruptive and inefficient approach as the solution team is put into a pressured and stressful position to complete the user story. This, we hypothesized, is the root-cause that impacts the other stats/metrics.

The Agile best practice is to have the userstories completely groomed (that includes reviewing it with solution team) at-least 2 sprints in advance. The suggestion we proposed was to have a separate ceremony with the solution team (engineers and testers) to present them the groomed user stories — therefore, having this done earlier and outside Sprint Planning. Although it would be a new ceremony, it did not require extra time commitment since this was the same time spent during sprint planning, just a few days or a week in advance. It would result in an efficient sprint planning ceremony since (hopefully) all the gaps identified in the “Pre-planning” session were closed by this time.

The result was that the number of defects raised within a sprint and planned-to-done ratio metrics dramatically changed. The “Happiness” stat from our last Agile Health Assessment was 97% .

More on this story later….Stay tuned!

--

--