Data Products: Tell Me How And What
In the first part of this series I covered the why behind data products. This article covers how we make them and what they are.
In the previous article I gave examples of some of the data products you interact with everyday. To recap super quickly on those, here they are:
- When Netflix gives you a recommendation of which shows to watch next 🍿
- Spotify’s daily mix recommending what you might enjoy listening to next 🎵
- The price you see in the Uber app is different to the price someone standing next to you, going to the same place 🚕
- The planning feature to set time to arrive by and therefore the time you’re told to leave by in Google Maps 🗺
The key thing to remember is that a data product is
“…facilitating an end goal using data” (DJ Patil).
‘Data Products’ come in all shapes and sizes, from dashboards to APIs.
For readers that don’t know what an API is, it’s an Application Programming Interface. See this as the waitress/er coming out the kitchen with your favourite dish, and the chef in the kitchen as the place where all the data is sitting
From an ‘inside my company what could these look like’, here’s some examples that’ll resonate beyond big tech:
- A personalised price for each customer achieved by a machine learning algorithm that’s appearing on the website, agent desktop or banner via an API
- A short-term sales forecast that’s presented in a dashboard, but before that it could be in Excel to validate it with a functional owner
- A plan that’s arrived at by using lots of different data to optimise the outcome for whatever metric you’re shooting for
- A recommender that suggests a product for each customer that they’re most likely to upsell to
- A prompt that’s delivered to an agent to respond to a customers message
Serving many
Data products in the form of let’s say an API, are channel agnostic. This means that we make them available for any channel / application around the company to use. For example:
- Digital product teams building customer journeys for the website could integrate our recommender to show logged in customers the best product to sell
- A mobile app used by your frontline teams could be surfacing the next thing to do to improve a customers’ service
Data products are reusable and can be adapted for new use cases, so we can create 2x or more of the value through expanding into new places.
Digital Product vs Data Product
I spent the first part of my career predominantly in digital product, which meant that I was working with cross-functional product teams that looked a little different to the teams I work in today.
We integrated with APIs to power various elements and features within the things we were building such as a website, mobile app, back office system etc. Those APIs were doing the job of passing basic data from one system to another e.g., customer name, bill date, and sometimes an output which had been derived from basic rules applied to the data, with limited focus on the data itself.
There wasn’t any focus on creating value from the data, it was more about getting data points from a to b.
Today, the team I’m part of focuses on creating value from data which then fuels the experiences our colleagues and customers. We apply product management to data.
In digital product teams teams, data people often sit in those teams, they’re typically analysts that are focusing on where could the product team go next: is how the product performing, what are the customer behaviours we’re seeing.
In data product, as a product manager the job hasn’t changed, but the people me and my team work with, and the things we build has.
In digital product you see teams using the likes of Sketch, Figma, Zeplin, usertesting.com, where as in data product you’ll see teams using the likes of Notebooks, dbt, BigQuery.
But in both teams, you’ll see Confluence, Jira, Slack, Lucid (or similar like Miro) and so on. I mention Jira, and I hear some of you – this is used by the teams however they wish. In software / platform based products great, but for data products it is used in a much lighter fashion, sometimes Trello is chosen, it’s up to the team.
When it comes to technologies, you’ll hear the engineering teams building the customer journeys talking about the likes of JavaScript, Angular and on data product side, you’ll hear Python, SQL etc.
When it comes to machine learning based data products, you’ll hear teams talking about the most impactful features. I like to compare this to Moneyball and Brentford FC.
Teams will apply SHAP, a methodology to see which combination of features are best performing on the pitch. So you’ll see graphs like this, at first it’s like what the, and you become a geek for graph summaries, titles and axis labels 🤓
Measure the impact, value realisation
In the digital world, you’ll hear things like CRO (conversation rate optimisation), maybe someone is testing the layout, steps in the journey or hey, even the colour of a call to action button.
Over here, we might be testing one churn model over another to see which one is the most accurate, or what is the optimum price for a product for a specific type of customer. On both sides experiments are ran.
Between data product teams, and digital product teams, experiments are ran together too. For everything we build, we need a partner in crime to take our thing to the wild. Imagine the call centre, we want to show agents a relevant set of recommendations for the customer on the phone. We need to get our pipe (API) into their platform, and the same with the website, the app.
These are the teams that help us realise the value we’ve created, or not created, and learn about it. A great example is a product recommender.
So for every data product, we need partners that make the bet with us, and we need customers to be part of that bet!
Building a Data Factory 🏭
In many companies, data teams are everywhere, and the level of maturity differs. If you compare a bunch of companies and look at the infrastructure, the platform they’ve got up and running to ‘do data’ you’ll be surprised what it takes. Platform teams are underrated (an applause for them please 👏). Not only do you need to invest in the platform and tooling, you need time and people. People first, always.
I didn’t coin ‘data factory’, it’s a concept I learnt from our Data Leader, Alberto, and a great one too.
This is how I think of the data factory:
I’ve said it many times, but people first, unleashed with processes that help not hinder, and a solid platform, able to serve millions of customers every second, of every day.
They’re the overarching component parts, and as part of the journey to building your data factory, you need to pick a vendor for the platform part, it could be Google Cloud, Microsoft Azure, Snowflake amongst others. Then there’s a whole load of other tools to make work easier (and better), like dbt, Slack, Lucid, Atlassian for documentation etc.
Data Factory in comparison to a Car Factory
The factory is the platform, in my world today it is Google Cloud, but that could be any of the ones mentioned above.
The machinery is the suite of applications and tooling setup and supported by our platform team, e.g. BigQuery, Airflow, dbt.
The lorry carrying the raw materials to make the car is the data ingestion process, which Data Engineering handle. Moving raw materials (the data) from one place to another e.g., a source system or legacy data warehouse to a new one.
The organising of the raw materials happens in the Parts department, this is the modelling team organised the data and making it make sense with meta-data etc, as well as analytics engineers that know the domain inside out. Analysts are asked questions all the time, and many times, the answers they find are needed again and again, so we ensure our analysts are up-skilled in making those things available for everyone after, by evolving a model.
The getting materials from Parts into a place they can be worked on is Data Engineering again.
The teams analysing the parts and figuring out what we could make is analysts and data scientists, as well as domain experts, driven forward by data product managers through workshops and collaboration sessions.
The people building out the chassis which the engine goes into, they’re the software engineers. They’re making sure the infrastructure around the engine is rock solid, that the needs of many teams are catered for, and it’s scalable.
The people going out to customers and suppliers figuring out which problems exist today that we could solve and what should we build are the data product managers, not only to they focus on creating value for the customer, but they bring the teams together. The PM’s will do whatever it takes, could be social officers, parents, whatever is needed for the product and team to succeed and be great. And our privacy and security friends, maybe they’re the health and safety officer, ensuring risks are foreseen and handled accordingly.
Of course with every factory, there’s someone overseeing it all. That’s the data leader; chief data officer, whatever they’re called, they’re the ones driving the overarching vision and strategy for the companies data strategy.
My journey creating value from data
It’s been 2.5 years, a team that didn’t exist, no people, no platform, no process, no products, and now we’ve got millions of people experiencing our data products every day. A 24/7, 365 day operation, running vital experiences that drive revenue growth, cost efficiencies and customer experience.
Everyday, more data is made available, usable, and enriched, meaning that our IKEA for data gets better and better all the time.
Data product teams are as fascinating as the data companies hold. By far the most diverse minds I’ve come across, which is what makes it much more exciting to see a problem / opportunity realising itself into a data product. When pairing up with domain experts, these teams can achieve anything, and it’s pretty special from what I’ve seen so far. Give the people the right tools, the space and the opportunity, and they’ll fly. Let them be a product team, not a delivery team or a feature team.
And now with GenAI on the lips of every CEO, unlike Candyman the movie, let’s not keep saying “AI, AI, AI” in the mirror, let’s experiment and build some prototypes, testing with real people to find further value from this technology – whether it’s answering queries, classifying data or summarising videos or emails!
In this series, I covered the why, shared some of the what and how. If you enjoyed it, keep following for more or check out my previous articles. Thanks for reading!
