DATA4500 Marketing and Social Media Analytics

Week 11
Forecast, predict, keep

Predictive modelling for a coffee shop that wants to keep its customers

Kaplan Business School · Lesson 11 of 12

Where we are

Last week we measured customer value and churn. This week we predict them, then act.

WeekTopic
9Strategy optimisation: SEO, A/B tests, crisis handling
10Customer lifetime value and churn: what they are, how to measure them
11Predictive modelling: forecast sales, predict who leaves, decide what to do
12Assessment
Three things you will be able to do by the end
  1. Use a sales forecast model and explain it in plain words
  2. Use a churn model to say how likely each customer is to leave
  3. Turn that prediction into a decision about keeping customers
The one idea for today

A prediction model does the same three jobs as your weather app

24°

Job 1: forecast a number

Tomorrow will be 24 degrees.

BrewLab: how much will we sell next week?

60%

Job 2: give a chance

There is a 60% chance of rain.

BrewLab: how likely is this member to leave?

Job 3: make a decision

Take an umbrella, or don't.

BrewLab: what do we do about it?

A model is only useful when it changes what somebody does. Keep asking: what decision does this number help Dana make?
Our case

Meet Dana, who owns BrewLab and has three questions

BrewLab is a specialty coffee shop in Melbourne. Dana runs a loyalty app: members earn a free drink after every ten purchases. Four thousand people have joined.

Dana is not a data person. She wants answers she can act on, in words she understands.

Dana's questions this week
  1. How much will we sell next week?
  2. Which members are about to stop coming?
  3. What should we do to keep them?
Sales history 2 years, weekly Loyalty app data 4,000 members Models Decisions Dana can make
Part 1 of 3

Forecasting a number

How much will BrewLab sell next week? Regression is a recipe that turns known facts into a sales estimate.

1.1

A regression model is a recipe: a few ingredients, each with a fixed amount

Look at two years of BrewLab weeks. Weeks with more marketing spend sold more. The dots do not sit on a perfect line, but a line through the middle captures the pattern.

That line is the model. Give it a spend, it gives back a sales estimate.

In words

Start with base sales of about $1,800. Every extra $100 of marketing adds about $12.

Sales = 1,800 + 12 × Spend
Marketing spend that week ($) Weekly sales ($) 3002,000 1,8002,600 The model (the line)
1.2

Real recipes have more than one ingredient. Each number tells Dana one thing.

Sales = 1,800 + 12×Spend + 650×Promo − 90×Rain + 400×Dec
NumberWhat it means for Dana
1,800Base sales when nothing special happens: no marketing, no promo, no rain, not December
+12 per $100 spendEach extra $100 of marketing adds about $12 in sales
+650 if promoA promotion week sells about $650 more than one without
−90 per rainy dayEach rainy day costs about $90
+400 if DecemberA December week sells about $400 more
How much each ingredient moves weekly sales Promo week+650 December+400 $100 more spend+12 One rainy day−90 Five rainy days−450
The hidden phrase in every row: "if everything else stays the same". Each number describes one change on its own.
1.3

Using the recipe: forecasting one week, step by step

Next week

Marketing spend $1,200 · promotion on · 2 rainy days forecast · not December

  1. Write spend in hundreds: $1,200 becomes 12
  2. Promo is on, so Promo = 1. Not December, so Dec = 0
  3. Fill in the recipe and add it up
1,800 + 12×12 + 650×1 − 90×2 + 400×0
= 1,800 + 144 + 650 − 180 + 0
= $2,414

What Dana hears

"Expect about $2,400 next week. The promotion is worth roughly $650 of that. If the rain clears, add another $180."

Common slip

Typing 1,200 instead of 12 for spend. The forecast jumps to $16,000 for a coffee shop. If a number feels silly, it usually is.

1.4

The recipe only works inside the range it was built from

Dana asks: "If I spend $9,000 in one week, the recipe says $3,530. Let's do it."

Her maths is right. Her trust is wrong. The model only ever saw weeks with spend between $300 and $2,000. It has no idea what happens at $9,000.

A coffee shop fills up. Extra advertising past a point brings nobody new through the door. The straight line keeps climbing anyway, because a straight line does not know about full shops.

Rule

Trust a forecast inside the data range. Outside it, the model is guessing, and so are you.

Data the model saw No data here Marketing spend ($) 3002,0009,000 What the line says What probably happens
1.5

There are many kinds of regression. You need two of them.

Linear regression: forecast a number

Answers "how much?" Sales, price, visits, spend.

Output: any number, like $2,414.

Part 1

Logistic regression: predict a chance

Answers "will it happen?" Will they buy, will they leave, will they click.

Output: a chance between 0 and 1, like 0.82.

Part 2

The others you may hear about

Ridge, Lasso and ElasticNet are linear regression with a brake that stops the model over-reacting to noisy data. Polynomial regression lets the line bend. Quantile regression forecasts a "worst case" or "best case" instead of the middle. All are variations on the same recipe idea. You will meet them properly in DATA4400.

Check

Knowledge check: forecasting a number

In the BrewLab recipe, the number next to Rain is −90. What does it mean?
A coefficient is the change in sales for one unit of that ingredient, holding the rest still. One rainy day, about $90 less.
Dana wants a forecast for a week with $6,000 of marketing. The model was built on spend between $300 and $2,000. What is the right advice?
A model is reliable inside the range of the data it learned from. Far outside, it extrapolates a straight line the real world may not follow.
Part 2 of 3

Predicting a chance

Which members are about to leave? Logistic regression gives every customer a number between 0 and 1, like a chance of rain.

2.1

"Will this member leave?" is a yes/no question, and a straight line answers it badly

Try the Part 1 recipe on a yes/no question. Leaving = 1, staying = 0. The straight line quickly predicts values like 1.4 or −0.3. There is no such thing as "140% leaving".

We want a curve that starts near 0, ends near 1, and never goes outside. That curve is logistic regression.

Weather app version

The app never says "140% chance of rain". It squeezes everything it knows into a number between 0% and 100%.

Months since last visit 01 Left? Straight line: goes past 1 S-curve: stays between 0 and 1
2.2

Logistic regression in two steps: add up risk points, then read the chance off a curve

Step 1: the risk checklist gives a score

Score = −1.5 + 0.8×Months away − 0.4×App orders + 1.0×Complaint
  • Every month away adds 0.8 points
  • Every app order last month takes off 0.4
  • A complaint adds a full point
  • Start at −1.5, so a fresh, happy member sits low

Step 2: the S-curve turns the score into a chance

High score, high chance. Score 0 lands exactly on 0.50. The curve never leaves 0 to 1.

Score Chance of leaving −30300.51 Score 1.5 = 0.82 Score −1.5 = 0.18
Score−3−2−10123
Chance0.050.120.270.500.730.880.95
2.3 · Try it

Score a BrewLab member yourself

Try these

Mia: 1 month, 2 orders, no complaint.
Tom: 3 months, 1 order, complaint.
Priya: same as Tom but no complaint. One complaint moves Priya from 0.62 to Tom's 0.82.

Score = −1.5 + 0.8×3 − 0.4×1 + 0 = 0.5
0.62
chance of leaving in the next 3 months
0.50 cut-off
Flagged: at risk
2.4

The 0.50 cut-off is a choice, not a fact. Use the chance, not just the label.

The model labels anyone above 0.50 "at risk". That is a shortcut. Priya at 0.62 and Tom at 0.82 both get the same label, but Tom is the more urgent call.

A chance of 0.62 means: out of 100 members like Priya, about 62 would leave and 38 would stay. Priya is not certain to leave.

Dana can move the line

Set it at 0.70 if she can only afford to contact the most likely leavers.
Set it at 0.30 if she wants to catch people early, before they drift.

00.501 Default cut-off Mia0.18 Priya0.62 Tom0.82 Cut-off at 0.70: only Tom Cut-off at 0.30: Priya and Tom
2.5

A decision tree asks the same questions, but as a flowchart a manager can follow

Instead of adding points, a decision tree asks one question at a time and splits the customers. It ends in groups, each with its own leaving rate.

Trees are easy to read out loud. Logistic regression gives a smoother, more precise chance. In the lab today you will run both on the same data and compare.

Which should Dana use?

Use the tree to explain the story to staff. Use logistic regression to rank who to contact first.

Months away > 2? no yes App orders > 2? Complained? yesnonoyes Safe8% leave Watch30% leave Watch55% leave At risk85% leave
2.6 · Lab

Lab: run the churn models on real telco data in Colab

Data: WA_Fn-UseC_-Telco-Customer-Churn.csv (7,000 phone customers, with a column saying who left).

  1. Open Google Colab and upload the notebook from MyKBS
  2. Click the folder icon on the left, then the upload icon, and add the CSV
  3. Run one cell at a time. Read the output before you run the next one
  4. Your facilitator will help you read the results

Notebook 1: clustering

DATA4500-cluster-analysis-on-customer-churn.ipynb

Groups similar customers together so you can see which groups leave most. No target needed; the computer finds the groups.

Notebook 2: logistic regression and decision tree

DATA4500_Telco_Churn_prediction_and_explanation.ipynb

Builds both Part 2 models on the same data and shows which columns matter most.

While it runs, ask: which three columns would you expect to push the chance of leaving up? Check whether the model agrees.

Check

Knowledge check: predicting a chance

A member's chance of leaving comes out at 0.62. Which statement is right?
A chance is a rate across similar people, not a verdict on one person. 0.62 is above the cut-off but far from certain.
Dana can only phone 50 members this month. How should she use the model output?
The label throws information away. The chance lets Dana rank customers and spend limited effort where it matters most.
Part 3 of 3

Deciding what to do

The umbrella moment. A list of at-risk members is worth nothing until Dana does something with it.

3.1

People leave for reasons. Find the reason and you can often fix it.

Bad experience with the product

The app checkout freezes. The coffee is inconsistent. The queue is too long at 8am.

Fixable

Bad customer service

A complaint went unanswered. Staff were rude on a bad day. A refund took three weeks.

Fixable

Bad fit

They moved suburbs. They stopped drinking coffee. The product never suited them.

Let them go gracefully

Why this matters for the model

The churn model tells Dana who. Only the reason tells her what to do. A voucher does not fix a frozen checkout.

3.2

Customers do not give many second chances

A PwC survey asked people when they would stop dealing with a brand they love.

  • About 1 in 3 walk away after a single bad experience
  • Almost half walk away after a few bad experiences
  • In Australia, 74% say customer experience decides which option they buy

Every frozen checkout at BrewLab is not one annoyed customer. It is a coin flip on whether they ever come back.

32%After one badexperience 47%After several badexperiences Share of customers who stop interacting with a brand they love (all countries)
Source: PwC Future of Customer Experience Survey 2017/18
3.3

Six moves that keep customers

1. Ask for feedback at the right moments

Just after the first purchase, and when visits start to drop. Listen before they decide.

2. Reach out before they complain

Spot the problem first. A short "sorry, here is a free drink" beats a long apology later.

3. Watch the Net Promoter Score

"How likely are you to recommend us?" Promoters (9 to 10) minus detractors (0 to 6).

4. Give members a path

Onboarding for new members. Extra recognition for the loyal ones.

5. Show the value, not the features

"Free coffee every fortnight", not "10-stamp digital card".

6. Set expectations and meet them

Promise what you can deliver, then deliver it. Over-promising creates leavers.

3.4 · Activity

Activity: what is Starbucks Rewards really collecting?

In pairs. Watch the short clip My Starbucks Rewards: Now on Android and iOS, then discuss and be ready to share.

  1. What are the key features of the program?
  2. What customer data does Starbucks collect through it?
  3. Which of that data would go straight into a churn model like BrewLab's?

Question 3 is the one that matters. Every "feature" in a loyalty app is also a data column.

A loyalty app seen as a data pipe Order ahead Pay in app Collect stars Personal offers Time, store, drink Spend per visit Visit frequency Offer response
3.5

A CRM system is the shared notebook where predictions and actions meet

Imagine every staff member keeping notes on customers in their own head. Nothing is shared, nothing is followed up. A CRM system is one shared notebook for the whole business.

A good CRM helps Dana:

  • Predict who is at risk (the Part 2 model runs inside it)
  • Act with the right offer, at the right time, through the right channel
  • Report so she can show the results to her business partner
  • Share so the barista and the marketing person see the same customer

Common tools: Salesforce, Microsoft Dynamics, Zoho, SAS Customer Intelligence.

Shared customer view Predict Promote Serve Report Order Loyalty
3.6

CRM can be a piece of software or a way of running the business. Aim for the second.

Narrow"We bought Salesforce" Middle"Our tools talk to each other" Broad"Every decision starts with the customer"
Workforce benefits
  • Data shared across the team, fewer silos
  • Clear ownership: who is doing what for which customer
  • Routine tasks automated, so staff spend time on people
For BrewLab

The churn model, the voucher, the follow-up call and the result all live in one place. Next month, Dana can see whether the umbrella worked.

Source: Payne and Frow (2005), A Strategic Framework for Customer Relationship Management, Journal of Marketing 69(4).

3.7 · Case

Case: Philips turned a product company into a customer-relationship company

Watch Salesforce: Philips is a Trailblazer, a short video about how the Dutch health-technology company uses CRM.

Discuss
  1. What stands out about Philips' approach to its customers?
  2. Where on the narrow-to-broad line from the last slide does Philips sit?
  3. Which of the three model jobs (forecast, chance, decision) can you spot in the video?

Why a health company in a coffee lesson?

Because the pattern is the same at any size. Philips connects devices, patients and hospitals in one customer view. BrewLab connects an app, a till and a barista. Both are trying to notice a problem before the customer walks away.

Check

Knowledge check: deciding what to do

The churn model flags 400 members. 60% of their complaints are about the app checkout freezing. Dana has $2,000. Which move fits the evidence best?
The model says who; the complaints say why. Fixing the cause helps every future customer too. A voucher treats the symptom.
Three months later Dana asks whether the money was wasted. What is the fairest test?
Total sales move for many reasons. A comparison group removes the effect of things that changed for everyone, so any difference is down to the action.
Summary

Three jobs, one test: did the model change what Dana did?

Forecast a number

Linear regression is a recipe. Each number is a plain sentence. Trust it inside the data range only.

Predict a chance

Logistic regression adds up risk points and reads a chance off an S-curve. Use the chance to rank, not just the label to sort.

Make a decision

Find the reason people leave. Choose the action that fixes it. Measure against a comparison group.

Before next week

Complete the three pen-and-paper exercises on MyKBS (Dana's sales recipe, the risk checklist, the umbrella decision). Week 12 is the final assessment.

← → or click to move · T for contents
KAPLAN BUSINESS SCHOOL

Contents

Press T or Esc to close