Agile in Data Science

Agile in Data Science: How To Split Scope?

Learn what are spikes and user stories in Agile Data Science teams and how to use them to become a predictable team

Anca Onuta
Published in
3 min readJul 12, 2021

--

Agile in Data Science it’s like no other. Data Science Teams are very different than the other Scrum teams because:

  • the result is hard to describe as you never know what’s technologically possible
  • the work is always innovative. You haven’t done it before
  • the timelines are unpredictable as it’s hard to know how long and how much data you’ll need to reach stakeholders’ expectations.

But also, the Data Science teams are expected to be Agile.

How do we do Agile in Data Science?

Decomposing the scope into research and operational is key in your success of the Agile in Data Science teams.

Agile Methodology in Data Science — how to decompose Machine Learning User Stories
Agile Methodology in Data Science — how to decompose Machine Learning User Stories

The data science workflow is a mix between waterfall and iterative:

  • Data exploration
  • Algorithm research
  • more data exploration
  • more algorithm trials
  • model improvements
  • data exploration
  • model optimizations
  • show something working

And most of the teams are dealing with multiple algorithms or parameters at once.

To successfully apply Agile Methodology in Data Science, we’ll want to categorize the Data Scientist's work into 2.

  1. Research.

This category falls all the exploration work that we can’t commit for a clear deliverable—for example, data exploration and searching for the best algorithms.
The research work will be called spikes: time-boxed user…

--

--