Good metrics to measure user retention in content apps
According to the latest report by Comscore, audiences around the world are spending more time online. Mobile minutes as a percentage of total online minutes has also been rising. India and APAC countries have shown significant growth in mobile news consumption. The share of mobile time spent on News is reaching historic levels in U.S. markets too. However, with the growing number of apps everyday and falling user retention rates, many content based apps face the challenge of rightly engaging the user and retaining them for a longer duration.
Personalization of your content/feed is a highly effective way of driving high user retention. It improves user experience and increases user engagement. What helps here is using the right metrics to measure user retention and making use of these metrics to personalize your content.
What are the good metrics to measure user retention in content/news apps?
Mobile news apps are built for daily consumption/usage. Users are expected to return to the app every day and engage with the app for some time. Hence, Daily Retention is an important metric to keep track, measure and analyse.
However, Daily Retention is just a lagging indicator of how your app is performing. To really break down the task of improving your user retention and get to the leading metrics, you should first understand whether you want to tackle New User Retention or Mature User Retention.
What is an important metric for understanding new user retention?
If you want to tackle New User Retention, then the most important metric you should be working on is your ‘Feed Rank’. Feed Ranking is the process of organizing all your stories so that the most relevant content is seen on top. Ranking your feed helps you understand the order in which your posts/feed should appear on your user’s screen. By improving your feed ranking and making it optimal for your new user, you can nudge your users towards early activation, thereby ensuring Day 1 retention and Week 1 retention.
The question here is how do you rank your feed for new users of whom you have no understanding of their likes and preferences. Check out this detailed blog on increasing user activation by optimizing home feed here.
What is an important metric for understanding mature user retention?
If you want to tackle Mature User Retention, then the most important metric for you is ‘Content Immersiveness’. Let’s get a clearer picture of what could ‘Content Immersiveness’ be. Any user who has tapped on a news article is showing interest in that article. However, to understand how immersive or interesting your content is, you need to find out the time spent by the user on your post. However, it is not enough if you just measure the time spent. What you need to be actually measuring is the Effective time spent by the user on your content by subtracting the content loading time from the time spent. This metric would still not indicate the exact Content immersiveness. You also need to find the Scroll Depth, which indicates the percentage or length of the content the user has read through.
The Effective Time Spent on your content and the Scroll Depth together would indicate the Content Immersiveness. This could be one way of measuring Content Immersiveness.
How do we arrive at a metric which indicates the engagement levels of your content?
Total Time Spent on a screen/content can be defined as the time spent between two navigations. It is the time difference between the launch of the first screen and the launch of the next screen. It can also be defined as an aggregate of Content Loading Time and Effective Time Spent. Hence,
Effective Time Spent = Total Time Spent — Content Loading Time
What is Content Loading Time?
Content Loading Time theoretically has to be measured as the time taken between screen (activity / viewcontroller) start and time it takes for UI to get stable so that it is ready for consumption — Content Ready State. This might involve fetching data (from the local cache, remote servers) and a Computational overhead if any.
This can be understood with the simple illustration of the UI State Graph:
However, it is very difficult to measure the Intermittent Period from screen launch to content ready state. There are a few complicated methods to measure this which would require a separate post to explain. For now, we can focus on an easily measurable proxy metric — Network Time.
Content Loading Time is a combination of Network I/O Time and CPU Time. Usually, in content apps the CPU Time is fixed and often very negligible. We can safely say that the Network Time, that is the time taken for a network call to fetch data is a pseudo indicator of Content Loading Time. To be more precise,
Network Time = Timestamp at network call success — Timestamp at screen start.
Hence, Effective Time Spent = Total Time Spent — Network time.
What is Scroll Depth?
Scroll Depth is another good indicator of a user's interest in the content. A well engaged user scrolls through the complete content, reads it end to end and spends a significant amount of time on it.
For a feed with multiple cards and infinite scrolls like Facebook feed, or LinkedIn feed etc; three useful metrics would be:
Number of impressions
Number of times scroll bounds have been reached
Number of scrolls
For detailed content like story content, a better indicator of Scroll Depth is the % (or fraction) of the total content viewed, that is, the content which has been covered in the viewport. The formula for this is
(Max Offset + Viewport Size) / Content Length
Check the illustration indicating the above.
To arrive at a good metric for the Content Immersiveness you should be checking the Effective Time Spent, % of content consumed, along with Interactions patterns. With these, you get a very good understanding of how engaging your content is.
If you take Content Immersiveness and Feed Ranking into consideration for your content app, instead of just clicks, and build personalization algorithms, you can give your users highly personalized content and engage them much better and longer.
