The crime of estimating value
My very own Ivan Drago
As a Product Owner, I have consistently encountered one single difficulty to which I still haven’t found a perfect solution. That difficulty is to measure ROI when you don’t have the resources to determine revenue and/or cost of delay.
You are a Product Owner, and during years of studying, you learn that you must order your backlog by whatever delivers most value. You learn:
- Value is preferably measured in $$$, but otherwise is something very subjective.
- Subjectiveness + stakeholders = disaster
- Knowing ROI for each feature you deliver is essential part of your work.
So your challenge is to find a way to measure value in a non-subjective way, preferably in $$$. I say preferably, because I found subjectivity to be a far more dangerous enemy than measuring value in something that is not $$$.
Life’s not fair
Teams frequently estimate work in points (and POs use this as estimation of cost), and to have an estimated ROI you must determine the expected value a certain feature will aggregate. Right? That’s how a lot of agile teams work. However, when I told agile coaches I wanted to estimate value in points, they went berserk, and said I was introducing a dysfunction in the framework. From that, I derived the following conclusion:
Estimating work in points = good
Estimating value in points = bad
So I went to check Scrum.org’s material on ordering backlogs and I found this article, marking the difference between prioritizing and ordering. How awesome and refreshing to read a PO’s supposed to make decisions and that sometimes the order of a backlog is not a direct product of the equation return ($$$) / investment ($$$).
That being said, if a team can estimate in points (and, later on, we can verify the real cost of each sprint/PBI), as a PO, I am also entitled to estimate in points, establish indicators to check if my estimate was achieved and use points as a single unit to guide my ordering of a backlog.
But you are cheating!
I heard the fact that I didn’t use revenue or cost of delay throughout my backlog meant I was cheating. I didn’t know how much the company stood to gain with a certain feature, and therefore I was guessing. Well, isn’t a hypothesis exactly that? An educated guess? If I guess the cost of delay of said feature, is that any different? I would LOVE to be able to always estimate value in $$$ without guessing, but if you are to guess, doing so in currency is just as bad as guessing in hours (which is what the slowest scrum teams do, according to Ken Schwaber and Jeff Sutherland). They mention having gathered the numbers of 70k scrum teams. I wonder if there is any similar study with POs, in which people compare POs that measure value in $$$, in points/gummy bears/beans/etc or don’t measure at all.
A pet hypothesis
To further base my assumption, I would like you to imagine we have a startup for cosmetics. We have a merely informative website and we sell in physical retail. In my backlog, I list commercial and non-commercial needs with some to no direct correlation to revenue. I order the backlog thinking on how important a certain thing is, but there is a lot I don’t know in terms of financial return, and that should be fine, if I am testing hypothesis. What I can’t have is the key result I am considering to validate that hypothesis to be vacant.
Below you can check, in blue, the items I believe I wouldn’t be able to directly correlate with revenue. In yellow, an item I believe I would be able to correlate with revenue after it has been implemented, but not before. And in green, items I can infer an estimate of revenue based on a previous hypothesis validation.
To normalize this backlog and measure value, not only in $$$ but also intangible value, I need to apply one single unit throughout the backlog (some options here). But if the aggregated value is not tangible, $$$ is a faulty way to measure.
A millennial’s view
The idea of intangible value is very millennial-friendly. We search for autonomy, mastery and purpose and reach the best results not motivated by unthinkable amounts of money, but when we earn enough so that the issue of money is off the table and are driven by belief. We’d rather live a productive, happy, healthy and sustainable life than being filthy rich and miserable.
I believe that view should be reflected in the way we think product value as well, otherwise we risk looking at immediate profit instead of long-run sustainability. Not that profit is a bad thing. But a profit-only view would discard the items in blue in my ficcionary startup, and risk an unforeseen failure, due to disregarding actions that are not directly profitable, but will come back to bite you in the butt later on.
A profit-only driven product management is something I can do. But it’s not the best I can do. There is much more value in addition to profit, and it would be a waste not to pursue it.
If you like this post, don’t forget to recommend and share it. Check out more great articles at Code Like A Girl.