Agile in Data Science
Agile in Data Science: How To Split Scope?
Learn what are spikes and user stories in Agile Data Science teams and how to use them to become a predictable team
Agile in Data Science it’s like no other. Data Science Teams are very different than the other Scrum teams because:
- the result is hard to describe as you never know what’s technologically possible
- the work is always innovative. You haven’t done it before
- the timelines are unpredictable as it’s hard to know how long and how much data you’ll need to reach stakeholders’ expectations.
But also, the Data Science teams are expected to be Agile.
How do we do Agile in Data Science?
Decomposing the scope into research and operational is key in your success of the Agile in Data Science teams.
The data science workflow is a mix between waterfall and iterative:
- Data exploration
- Algorithm research
- more data exploration
- more algorithm trials
- model improvements
- data exploration
- model optimizations
- show something working
And most of the teams are dealing with multiple algorithms or parameters at once.
To successfully apply Agile Methodology in Data Science, we’ll want to categorize the Data Scientist's work into 2.
- Research.
This category falls all the exploration work that we can’t commit for a clear deliverable—for example, data exploration and searching for the best algorithms.
The research work will be called spikes: time-boxed user…