Understanding Content through the Lens of Your Customer, Attention Index

Content is the core of a video-based business. Smart marketing and a sleek UI may attract users to your service, but they won’t stay unless you deliver on your promise to entertain them. So, how do you measure which content captures their attention and increases their subscription renewals? Answer: the Attention Index.

At Wicket Labs, we think the continued subscription of your customers and evidence of their enjoyment of content best reflects the value they place on your service. The challenge only lies in correlating the observed viewing behaviors and customer subscription length. The Wicket Scorecard is built by connecting these disparate data sets. We are excited to demonstrate how we help you see your content in this way.

Attention Index Provides Immediate Insights

Our LTV calculation is a powerful indicator of content value and allows you to understand the impact on your audience. But, with its time requirements, it is clearly limited to evaluating your content over the long term.

attention index iconYour most recent content also needs to be promoted correctly as soon as it enters availability. So, in conjunction with our LTV updates, we have just added an Attention Index column to our “Content Explorer” (Fig. 2) and “Top Titles” Wickets to allow for immediate feedback on the value your content provides to subscribers. This is a powerful metric that ignores the Audience Size and the fuzzy information from Average Completion to help you immediately know whether your content’s audience recognizes and enjoys the video.

The Attention Index measures the effectiveness of the promise a title and thumbnail makes – “You’re going to enjoy this!” Your subscriber provides their attention in hopes of being entertained and it’s the content’s job to deliver on that promise. This index is on a scale of -100 to 100 and compares the number of dissatisfied viewers, who watch at least 1 minute of the video but do not get to at least 25%, with the number who watch to at least 75% completion. The calculation subtracts the percentage of dissatisfied viewers from the percentage of satisfied viewers, ignoring the ‘meh in the middle’ folks.

Content Explorer

Fig. 2 – Content Explorer showing Attention Index for titles

For example, the fictional video “Dinosaurs Attack!” that, despite its exciting title, is a dry exploration of dinosaur hunting behaviors. Let’s say 40% of the people who start watching this hour-long title quit between 1 and 10 minutes. 10% of the viewers really like dinosaurs and watch all the way to the credits, which start about 95% of the way through. 10 minus 40 gives this content a -30 Attention Index score. It reflects the mismatch between the promise of the metadata and the reality of the video. Popular, well-watched content tends to have scores greater than 50, while scores of 30 or lower indicate that your editors might consider updating the metadata to better match the video to the audience who views it.

This Attention Index gives your editors immediate insight into content that is well targeted and really resonates with your audience. This can help to uncover hidden gems in your library, which are worthy of additional promotion. Or, it can help you identify content to de-emphasize.

Using Lifetime Value (LTV) to Evaluate Video Value

So, how do we correctly evaluate a specific video’s effect on the continued subscription of a customer? We start by leveraging the lifetime value (LTV) approach from my previous blog post. That method looks at a group of users and uses their subscription tenures to calculate a survival curve for that group. This curve lets us understand how many subscribers remain for any subscription tenure, like 6 months or 1000 days. The key is that we apply a survival curve to any group of users and compare that to another user segment. Statistically, significant differences mean that there is a correlation that is worth considering.

With a recent update, we’ve brought our improved LTV calculation into the Content Explorer. Using Content Explorer, you can compare the LTV of groups of users based on the content they consume with just a few mouse clicks. (Fig. 1)

Content Explorer LTV

Fig. 1 – Compare lifetime value of users watching the same content with Content Explorer

Our analyses have shown that there are some pitfalls when groups of subscribers are naively compared to each other. We’ve taken steps to ensure the comparisons you make with Content Explorer actually reflect content-driven group differences, rather than statistical anomalies or content-agnostic trends.

For example, consider users who watched a recently released movie. If you compare them to another popular movie available in a service, you nearly always find that the recently released movie has an audience with an LTV 5-10 times more than the other movie!

Acting on this difference would be a huge mistake. There are two factors likely driving a much more robust survival for viewers of the more recent movie:

  1. The recent release of the movie means that the population who used your service when this was available are your most frequent users. These users have lower churn for reasons that are not related to the content grouping we are applying.
  2. In a contractual business where renewals happen on a monthly basis, content for which all the views occurred in the last 2 weeks only had half of the audience (at most!) who had a renewal event occur. In subscription businesses, LTV is only a useful tool for comparing groups when enough time has passed to include involuntary churn and post-billing event voluntary churn.

The Wicket Scorecard suppresses LTV calculations when content has been available for less than 1 month. Additionally, since small sample sizes often don’t provide statistically significant results, we suppress the LTV when the audience size is less than 250 customers.

Taken together, the Attention Index and LTV values for titles, movies, and series help you manage and value your content library throughout its lifecycle.

 

Tags:
Share This