To Improve the Whole, First Isolate the Parts: Finding the Culprit 📊🕵️♂️
Let's also understand ‘Planning Fallacy’ and Percentage Points (pp)
Hello, data-driven minds, welcome to the 11th edition of DataPulse Weekly!
We’ve discovered that most of our readers are analysts or data scientists. While our newsletter benefits anyone seeking to become more data-driven, you will see more customization for data professionals. This shift allows us to provide deeper, more relevant content tailored to data-related topics.
Rest assured, we will continue to simplify complex data concepts. If you're not a data analyst or scientist, read a few more editions before you decide that you don’t want to see us in your inbox in the future. The choice is always yours!
Now, let’s dive straight into today’s Data Menu!
Today’s Data Menu
Data Case Study: Category Impact on Overall Metric
Metric: CSAT
Human Bias Focus: Planning Fallacy
Data Nugget: Percentage Points (pp)
Data Case Study: Category Impact on Overall Metric
As a data analyst, you will face challenges when trying to determine how a particular category impacts your overall metric. For instance, consider a retailer like Amazon, which sells multiple categories and wants to understand how these categories are impacting overall customer loyalty.
Customer loyalty is often tracked using a key metric known as Net Promoter Score (NPS). We have explained NPS in great detail in our previous newsletter here.
The NPS journey starts with a simple question:
On a scale of 10, how likely are you to recommend our product/service to a friend or colleague?
This is how users are segmented into three buckets based on ratings:
NPS is calculated as (# Promoters - # Detractors) / # Total Responses * 100.
The retailer runs a user survey by category and receives responses as in the table below:
They calculate category-level NPS along with overall NPS. Here’s what it will look like:
Assuming the responses are proportional to the total orders in each category, which category do you think needs the most attention?
One might say Furniture, as it has the lowest NPS of 10! While the category has the lowest NPS and definitely needs a deep dive, we might be overlooking one key aspect - how big the category is!
To account for the size of each category, we need to isolate the impact of each category. To do this, we calculate the NPS as if the specific category doesn’t exist. By removing a certain category, our data will look like this:
We recalculate the Revised Overall NPS with one category exclusion. Here’s what it will look like:
This table shows how our overall NPS would be if the category did not exist!
The final step is to subtract the Revised Overall Category NPS from the Original Overall NPS (36) to get the impact of that category on the Overall NPS:
And, here we go! The table above illustrates the impact of each category on the overall NPS. Notably, the Appliances category has the highest negative effect, reducing the NPS by 4 points.
This framework accounts for the size of each category by excluding it from the NPS calculation. Our analysis revealed that even though Furniture has the lowest NPS, the Appliances category has the highest negative impact on the overall NPS due to the high number of responses.
Conclusion
Understanding which segment has the highest positive or negative impact on your overall metric is crucial. Isolating the impact of individual categories provides a better framework for prioritizing resource allocation to achieve the highest ROI. In our example, prioritizing improvements in the Appliances category would yield the most substantial benefits for overall customer loyalty.
This approach is not only applicable to NPS but also to other metrics like Conversion Rate, ARPU, and Fill Rate.
In the next section, we will explore CSAT, another technique to understand customer experience.
Metric of the Week: CSAT
While NPS provides a broad understanding of customer loyalty, focusing on the Customer Satisfaction Score (CSAT) can help the company address specific issues.
CSAT measures customer satisfaction by asking, "How satisfied are you with our service, product, or experience?"
Customers rate their satisfaction on a scale from 1 to 5, and the average score is known as CSAT. This metric offers immediate, actionable feedback on specific touchpoints.
Companies usually run CSAT surveys on multiple touchpoints—such as product quality, delivery experience, and customer service. By analyzing these CSAT scores for each interaction, the company can identify common pain points. For instance, if CSAT is low for product quality, the category owner can conduct a detailed product quality analysis to identify specific issues.
In conclusion, while NPS can highlight the need for improvement in a specific category for customer loyalty, leveraging CSAT allows the company to pinpoint specific issues and interactions that need fixing.
💡 Remember that building a data mindset is effective only when we focus on solving data-related problems. The below question is designed for exactly this kind of practice. We will address this in the last section of this newsletter.
Food for thought:
Your company’s product market share has increased from 10% to 20% year-on-year (YoY). Can you say the market share has increased by 10%?
Human Bias Focus: Planning Fallacy
Did you know there are more than 180 ways your brain can trick you? These tricks, called cognitive biases, can negatively impact the way humans process information, think critically and perceive reality. They can even change how we see the world. In this section, we'll talk about one of these biases and show you how it pops up in everyday life.
Imagine a growth analyst named Venky, tasked with a new data project to develop user intent scoring. Eager to deliver valuable insights, Venky estimates the project will take two weeks and convey the same to their line manager.
However, as the project progresses, Venky faces challenges with the size of the data, bad data quality, and multiple model iterations. On top of that, there are Ad-hoc data requests and fixing old reports due to automation errors. Each of these hurdles adds time and complexity, pushing the project well beyond the initial two-week estimate.
Venky ended up delivering the project three weeks late. That doesn’t get Venky any accolades but a 30-minute meeting with their skip-level manager for sure.
You must be feeling bad for Venky but if you are a working professional, you must have seen similar situations in the past.
This scenario exemplifies the planning fallacy, a cognitive bias where we underestimate how long projects will take while overestimating how quickly we can get things done.
To avoid planning fallacy, do this:
Base your estimates on past experiences
Add buffer time to account for potential delays
Consult experienced peers to gain realistic insights and feedback
Break projects into smaller tasks with incremental milestones
Understanding and addressing the planning fallacy is crucial for any data professional aiming to enhance efficiency and reliability in project delivery.
This brings us to our final section where we address the previous question on market share change.
Data Nugget: Percentage Points
Your company’s product market share has increased from 10% to 20% year-on-year (YoY). Can you say the market share has increased by 10% YoY?
You shouldn’t, and here’s why -
If you have $100 and it increases by 10%, it becomes $110. Similarly, if a 10% market share increases by 10%, it becomes 11%, not 20%.
This is where percentage points come into play. Percentage points represent the absolute difference between two percentages. For instance, moving from 10% to 20% is a 10 percentage point increase, not a 10% increase.
Percentage points (often denoted as 'pp') are particularly useful for comparing percentage changes over time or between groups, providing a clear, direct measure of change, free from the relative nature of percentages.
That wraps up our 11th edition! We've broken down complex data concepts and will continue to do so in future editions. If you found this helpful, please subscribe and share it with others who might benefit. Your support inspires us to create even more valuable content for you.
Intresting read 👍🏻