Teaching computers to read the mood of customer messages β and why marketers care. We'll use a real dataset of 100 BrewLab customer messages throughout.
Section 1
Understanding Sentiment Analysis
What it is, why it matters for marketing, and the categories we sort messages into.
1.1
1.1 What is sentiment analysis?
Definition
Sentiment analysis is the automatic process of reading a piece of text and deciding whether the feeling behind it is Positive, Neutral, or Negative.
A human can read one review and tell you the customer is happy. The problem is scale: a brand may receive thousands of reviews, tweets, and messages every day. No team can read them all.
The core idea
Sentiment analysis lets us measure emotion at scale β turning a mountain of free-text into a simple, countable signal we can chart and act on.
1.2
1.2 Why marketers care
Every message a customer writes is a tiny piece of feedback. Read together, they answer questions marketers ask constantly:
Question
How sentiment helps
How do people feel about our brand right now?
Track the share of positive vs negative mentions over time
Did our new campaign land well?
Compare sentiment before and after launch
What are people complaining about?
Zoom into the negative messages to find recurring issues
Is a problem going viral?
Spot a sudden spike in negative sentiment early
In this course
We'll treat our 100 messages as feedback for BrewLab, our running coffee retailer. The same techniques apply to reviews, survey comments, and social media posts.
1.3
1.3 The three sentiment classes
In this dataset, every message is labelled as one of three classes. Here is a real example of each:
Positive "Excellent the product! A joy to interact with. π"
Neutral "Average experience. Not bad but not great. π"
Negative "Frustrated with service. Wait times were unbearable. π "
Notice
Neutral is not "no opinion." It usually means mixed or lukewarm β the customer is neither delighted nor upset. This class is the hardest to get right.
1.Q
Knowledge Check β Section 1
Q1. What is the main reason marketers use sentiment analysis instead of reading messages by hand?
The value is scale. Humans are still better at nuance, but they cannot read everything β sentiment analysis turns huge volumes of text into a countable signal.
Q2. A message says: "It's fine. It does the job, nothing more." Which class fits best?
"Fineβ¦ nothing more" is lukewarm β neither happy nor upset. That mixed, in-between tone is exactly what Neutral captures.
Section 2
Exploring Our Dataset
Before analysing anything, always look at your data: its shape, its balance, and how messy it is.
2.1
2.1 The dataset structure
Our file has just three columns and 100 rows. Simple, but enough to learn the whole workflow.
ID
Text
Sentiment
1
Excellent the product! A joy to interact with. π
Positive
2
Worst service. Everything broke immediately. π
Negative
3
So-so experience. Did the job with no frills. π
Neutral
β¦
β¦
β¦
Text
The raw customer message. This is the input β what we analyse.
Sentiment
The correct answer, added by a human. This is the label we learn from and check against.
2.2
2.2 How the classes are balanced
Counting the labels reveals the messages are not split evenly across the three classes:
Positive messages outnumber Negative ones nearly 4 to 1. Keep this in mind β it will come back to bite us when we measure accuracy in Section 5.
2.3
2.3 Real text is messy
Real customer text is never tidy. Look closely at these actual rows and spot the problems:
"Excellent the product!excellentchoice A joy to interact with. π"
"It's fine experience. Managed my tasks well. π adequate"
Broken grammar β "Excellent the product!" is not a normal sentence.
Stuck-together words β "excellentchoice" should be two words.
Stray words, mixed CAPS, punctuation, and emojis all mixed in.
Why this matters
A computer treats "excellentchoice" and "excellent choice" as completely different things. Before analysis, we must clean the text β that's Section 4.
2.Q
Knowledge Check β Section 2
Q1. Our dataset has 60 Positive, 24 Neutral, and 16 Negative messages. What is this situation called?
When one class (here, Positive) appears far more often than the others, the classes are imbalanced. This affects how we should measure success.
Q2. Why is "excellentchoice" a problem for a computer?
A computer matches exact words. "excellentchoice" won't match the positive word "excellent" unless we split it during cleaning.
Section 3
How It Works
The overall pipeline, and the two main approaches to deciding sentiment.
3.1
3.1 The sentiment analysis pipeline
Whatever method we use, the message travels through the same four steps:
Figure 3.1 β The four-step sentiment pipeline. Everything else in this lecture fits into one of these boxes.
3.2
3.2 Approach A β the dictionary method
Lexicon (dictionary) approach
We keep a list of words with known feelings β positive words score +1, negative words score β1. We add up the scores in a message and read off the total.
Example on a real message β "Excellent product, but slow support and poor packaging."
Total score = +1 β1 β1 = β1 β below zero β Negative
Strength & weakness
Simple and needs no training data β but it can't understand context, sarcasm, or new slang, and someone must build the word list.
3.3
3.3 Approach B β the learning method
Machine learning approach
Instead of writing rules by hand, we show the computer many labelled examples (like our 100 messages). It learns the patterns that separate Positive from Negative on its own.
Figure 3.2 β Learn patterns from labelled data, then apply them to new, unseen messages.
Strength & weakness
Adapts to your own data and often more accurate β but it needs many labelled examples to learn from.
3.4
3.4 Comparing the two approaches
Dictionary method
Learning method
How it decides
Adds up word scores from a fixed list
Learns patterns from labelled examples
Needs training data?
No
Yes β the more the better
Set-up effort
Build/choose the word list
Collect and label examples
Handles your slang & context
Poorly
Better
Good first choice whenβ¦
You have no labels yet
You have plenty of labelled data
Practical tip
Teams often start with the dictionary method to get going quickly, then switch to the learning method once they have collected enough labelled feedback.
3.Q
Knowledge Check β Section 3
Q1. Using a positive/negative word list that we add up is which approach?
Scoring words from a fixed list and summing them is the dictionary/lexicon method. No learning from examples is involved.
Q2. What does the learning method most need that the dictionary method does not?
Machine learning discovers patterns from labelled data, so it depends on having enough examples with correct sentiment labels.
Section 4
Preparing the Text
Cleaning messy customer text so the computer can read the real signal β including the emojis.
4.1
4.1 Why we clean the text first
Computers match words exactly. To them, all of these look like different words:
These should be the same wordβ¦
β¦but the computer sees
Excellent, excellent, EXCELLENT, excellent!
4 different words
excellentchoice
1 unknown word
The goal of cleaning
Make every version of a word look identical, so real signals like "excellent" get counted every time they appear.
4.2
4.2 The cleaning steps
We tidy the text one step at a time. Watch a real message get cleaned:
startExcellent the product! excellentchoice A joy. π
lower-caseexcellent the product! excellentchoice a joy. π
remove punctuationexcellent the product excellentchoice a joy π
fix stuck wordsexcellent the product excellent choice a joy π
split into words (tokenise)[ excellent Β· the Β· product Β· excellent Β· choice Β· a Β· joy Β· π ]
Tokenising
The last step splits the message into a list of individual words (called tokens). Now each word can be looked up and counted.
4.3
4.3 Don't throw away the emojis
It is tempting to delete emojis as "junk." In this dataset that would be a mistake β the emoji is a perfect clue to sentiment:
π
βPositiveappears in all 60 positive messages
π
βNeutralappears in all 24 neutral messages
π
βNegativeappears in all 16 negative messages
Lesson
A "clue" (or feature) is anything that helps predict the answer. Here, keywords and emojis are both strong features. Know your data before deleting anything.
4.Q
Knowledge Check β Section 4
Q1. Why do we convert all text to lower-case during cleaning?
Case is not meaningful for sentiment, so we standardise it β otherwise every capitalisation would be treated as a separate word.
Q2. In our dataset, should we delete the emojis before analysis?
π/π/π line up exactly with Positive/Neutral/Negative. Deleting them would throw away the single most reliable feature in this data.
Section 5
Measuring Performance
How do we know if our sentiment analysis is any good? And why the obvious answer can fool us.
5.1
5.1 Accuracy β the simplest score
To check a method, we compare its guesses against the human labels and count how many it got right.
$$\text{Accuracy} = \frac{\text{Number of correct predictions}}{\text{Total number of messages}}$$
Example
If a method labels 82 of our 100 messages correctly, its accuracy is 82 Γ· 100 = 82%.
Accuracy is easy to explain to a client β but on its own it can be misleading. Here's why.
5.2
5.2 The imbalance trap
Remember our data is 60% Positive. Imagine a lazy method that ignores the text and just guesses Positiveevery single time:
The trap
A useless method scores 60% just by exploiting the imbalance. So 60% is the bar to beat, not a good result. With imbalanced data, always ask: "better than just guessing the biggest class?"
This is why analysts also check each class separately β especially the rare but important Negative messages, which a lazy method would miss entirely.
5.Q
Knowledge Check β Section 5
Q1. A method gets 45 out of 60 messages correct. What is its accuracy?