Discussing Teamwork and Measures on Agile for Humans

Ryan Ripley interviewed me on his podcast, Agile for Humans 83 about Create Your Successful Agile Project. We had a blast.

I didn't stint on my opinions or on my experience with agile teams. One of those opinions was about teambuilding, which I wrote about in Creating an Environment of Teamwork.

The other opinion (based on my experience) was that of using ROI to predict which product, project, or feature a team should do first. I also bashed earned value.

First, here are circumstances in which I've seen people successfully use ROI and earned value.

• You don't need to change the requirements or the roadmap. You can set the roadmap and the requirements at the beginning and not change the work.
• You know your set of customers, and they don't change.
• You don't need to collaborate with your customers about what they receive and when.
• You have a customer who has committed to buying this product from you. That's not going to change.

In these circumstances, you have a good chance of being correct with an ROI prediction for the features, project, or product. (Remember that ROI is the (Gain-Cost)/(Cost). If you have a feature that costs $1000, and you gain $5000, the ROI is ($5000-$1000)/$5000 = 4/5 = .8, or 80%.)

When things don't change much, I have seen people use a predicted ROI to decide which features to do first. I've measured those predictions, and when I worked with those people, they guessed right about half the time. Maybe half was good enough. (The rest of the time they were so far under their predictions, they should have put the money in a bank account at 1% interest.)

I keep working with organizations who cannot predict and “pin down” the roadmaps or the features in advance. They need to run experiments to see what features the customers want. Sometimes, that's because their customers evolve. Sometimes, especially with IT groups, they need to collaborate with their customers.

My IT clients have a huge problem: they have no guarantee their customers will use the product. My engineering clients can't predict sales because they can't predict the market.

You might not be like my clients. However, if you can't predict sales or predict adoption, ROI is not such a good measure of what to do first. Cost of Delay is a better predictor, especially for features.

The faster you either release an MVE or an MVP, the more likely you are to have a return on your investment. If you select a feature based on what's most valuable to you now (Cost of Delay), you might have a better selection process.

I'm not guaranteeing CoD is best. For my clients, it appears to be better than ROI.

Now, let me take on earned value. If your requirements don't change,  it's possible to calculate earned value. I don't find it useful in an agile project unless you use incremental funding. That's me.

Even on those projects and programs, you might want agile approaches to see the progress of the product towards done. However, those projects don't need the change that agile projects provide you.

Now, what happens if you work in a rapidly changing environment? If you are like most of my clients, your market changes at least once a month. Some other company releases some kind of feature or capability and all of a sudden, the Big Thing you were working on has no more value. You need to change, to work on something else. Your ROI has fallen through the floor. So has your earned value.

When you work in a rapidly changing environment and you need a new direction for your product, previous work might have no value. That means earned value doesn't work and neither does ROI.

What if your set of customers changes? You cannot be assured of who will buy what, which changes your ROI calculation. It might change your earned value because you have to ask, “value to whom?”

What if you need to change the roadmap and requirements? Again, the ROI prediction doesn't work. Earned value only tells you what you spent and what you finished so far. Well, so does run rate and cumulative flow.

The more change you have in your market and your potential customers, the less ROI and earned value mean to your project. Instead, the measures that make sense are the time to releasable product (this is why I like one-day or smaller stories), and the customer adoption of that product.

In Manage Your Project Portfolio, I said that ranking with ROI was a trap. It's a trap because it doesn't account for Cost of Delay and when this project still has value (or has no more value to the organization).

In Create Your Successful Agile Project, I recommend ranking your project work by using: shortest work first (assuming it is short as in one day or less), Cost of Delay, or learning. I have found that helps everyone think of MVEs and MVPs.

In Agile and Lean Program Management, I cautioned against using a predictive ROI to decide what to do. I recommended the product backlog burnup chart instead of an Earned Value calculation.

Can you still measure these calculated measures? Of course. How much value do they provide a project or a program in flux? I don't see much.

Remember, everyone's context is different. I do know that calculating ROI at the beginning of a project and expecting it to be correct at the end has not worked for my clients or for me because of the changes we cannot anticipate. Maybe it can work for you.