DATA4500 · Week 3 · Hands-on Lab

Sentiment Analysis in Orange Data Mining

A no-code, drag-and-drop walkthrough using 100 BrewLab customer messages. No programming required.

In the lecture we saw two ways to work out the sentiment of a message: the dictionary method (add up positive and negative words) and the learning method (learn from labelled examples). Orange lets us build both by connecting boxes on a canvas — no code at all.

What you'll do

  1. Set up Orange and the Text add-on
  2. Load the BrewLab messages (Corpus)
  3. Read the messages (Corpus Viewer)
  4. See the class balance (Distributions)
  5. Score sentiment automatically with the dictionary method (Sentiment Analysis)
  6. Inspect the scores (Data Table)
  7. Compare the computer's scores against the human labels (Box Plot)
  8. Optional: word cloud, and the machine-learning approach

The finished workflow

Here is the canvas you'll build. Each box is a "widget," and each line passes data from one widget to the next. Build it left to right.

Corpusload the data Corpus Viewerread the messages Distributionsclass balance Sentiment Analysisthe dictionary method (VADER) Data Tablesee the scores Box Plotscores vs labels Text Mining widget Core Orange widget
Figure 1 — The main workflow. The dictionary-method path runs Corpus → Sentiment Analysis → Box Plot.

Getting ready

0
Install Orange and the Text add-on
  1. Download and install Orange (free) from orangedatamining.com if you haven't already.
  2. Open Orange. Go to the menu: Options → Add-ons…
  3. Tick the box next to Text (this is the text-mining pack that adds the Corpus and Sentiment Analysis widgets).
  4. Click OK / Install, then restart Orange when prompted.
You'll know it worked when…a new Text Mining group appears in the widget list on the left, containing Corpus, Preprocess Text, Sentiment Analysis, and more.

Save the file brewlab_sentiment_w3.csv somewhere easy to find, like your Desktop.

How to build any workflow in OrangeDouble-click a widget in the left-hand list (or drag it) to place it on the canvas. To connect two widgets, click and hold the dashed arc on the right edge of the first widget and drag the line onto the second. Double-click any widget to open its settings.

Part A — Load and explore the data

1
Load the messages with Corpus

The Corpus widget is how Orange loads text. Place it on the canvas and open it.

Settings:
  • Click Browse and choose brewlab_sentiment_w3.csv.
  • In Used text features, make sure Text is on the right-hand (selected) side. This tells Orange the message column is the text to analyse.
  • Set Title variable to ID (so each message is labelled by its number).
You should seenear the bottom: Corpus of 100 documents, with Sentiment recognised as a category (Positive / Neutral / Negative).
2
Read the messages with Corpus Viewer

Connect Corpus → Corpus Viewer and open it. This simply lets you scroll through the raw messages.

Look for the messNotice the quirks we discussed in the lecture: the repeated "Excellent the product!", the stuck-together word "excellentchoice", and the emojis 🙂 😐 😠 at the end of each message. Real customer text is never tidy.
3
See the class balance with Distributions

Connect Corpus → Distributions. In its settings, set Variable to Sentiment.

You should seea bar chart with three bars: Positive 60 Neutral 24 Negative 16. This is the same class imbalance from the slides — Positive dominates.
Why this mattersBecause 60% of messages are Positive, any method must beat 60% accuracy to be useful. Keep this number in your head.

Part B — The dictionary method (automatic scoring)

Now the main event: let Orange read every message and score its sentiment automatically, using a built-in word dictionary called VADER.

4
Score sentiment with Sentiment Analysis

Connect Corpus → Sentiment Analysis (connect it to the original Corpus, not to a cleaned version — see the note below). Open it and choose the method:

Setting:   Method → Vader

VADER adds four new number columns to every message:

ColumnMeaning
poshow strongly positive (0 to 1)
neghow strongly negative (0 to 1)
neuhow strongly neutral (0 to 1)
compoundthe overall score, from −1 (very negative) to +1 (very positive). Around 0 means neutral.
Why feed it the raw, un-cleaned text?Unlike the learning method, the dictionary method uses capital letters, exclamation marks, and emojis as clues — exactly the "signals" we said not to throw away in the lecture. So we deliberately skip cleaning here and let VADER read the message as the customer wrote it.
5
Inspect the scores with Data Table

Connect Sentiment Analysis → Data Table and open it. You'll see every message alongside its new compound score and the human Sentiment label.

Click the compound column header to sort by it.

You should seethe most positive messages (highest compound, near +1) are the Positive ones; the lowest (near −1) are the Negative ones. The dictionary method is working.
6
Compare computer vs human with Box Plot

This is the key comparison. Connect Sentiment Analysis → Box Plot and set:

Settings:
  • Variable: compound  (the computer's score)
  • Subgroups: Sentiment  (the human label)

This draws one box per human class, showing the range of VADER scores inside each.

What good agreement looks likeThe Positive box sits high (positive compound), the Negative box sits low, and Neutral sits in the middle. When the boxes barely overlap, the computer and the humans mostly agree.
DiscussWhich class overlaps most with the others? Almost always it's Neutral — just as the lecture predicted, the lukewarm middle is the hardest for any method to pin down. VADER is a general-purpose dictionary, not tuned to BrewLab, so some disagreement is expected and worth discussing.
OPTIONAL

Extra 1 — A word cloud of the messages

For a quick visual of the most common words: place a Preprocess Text widget and connect Corpus → Preprocess Text → Word Cloud.

Preprocess Text settings (tick these): Transformation → Lowercase; Tokenization → Regexp; Filtering → Stopwords (English).

The Word Cloud will show "excellent", "service", "experience" and friends sized by how often they appear. This is the cleaning step from the lecture, done with one widget.

OPTIONAL · ADVANCED

Extra 2 — The learning method (machine learning)

Instead of a fixed dictionary, we can let Orange learn sentiment from the labelled examples. This path does want cleaned text, so it starts with Preprocess Text.

Corpusraw data PreprocessText Bag ofWords SelectColumns Test andScore ConfusionMatrix LogisticRegression
Figure 2 — The learning-method path. The model (Logistic Regression) feeds into Test and Score, which is checked with a Confusion Matrix.
  1. Preprocess Text — same settings as the word cloud above (lowercase, tokenize, English stopwords).
  2. Bag of Words — connect Preprocess Text → Bag of Words. Defaults are fine; this turns each message into word counts the model can use.
  3. Select Columns — connect Bag of Words → Select Columns. Put the word-count columns in Features and put Sentiment in Target.
  4. Test and Score — connect Select Columns → Test and Score. Also add a Logistic Regression widget and connect it in as the learner. Use Cross validation (5 folds).
  5. Confusion Matrix — connect Test and Score → Confusion Matrix to see exactly which classes get confused.
Read it honestlyLook at the CA (Classification Accuracy) in Test and Score. Remember the 60% baseline — always-guess-Positive already scores that. A good model must clearly beat 60%. In the Confusion Matrix, check the Negative row: with only 16 examples, the rare class is the one models most often miss.

What you built

Discussion questions
  1. For which sentiment class did the computer disagree most with the humans, and why might that be?
  2. Why did we feed raw text to the dictionary method but cleaned text to the learning method?
  3. If BrewLab used its own slang (e.g. "this brew is sick" as praise), which method would adapt better, and why?

Troubleshooting

ProblemFix
No "Text Mining" widgets appearThe add-on didn't install. Redo Step 0 and make sure you restarted Orange.
Corpus says "0 documents" or the text column is missingOpen Corpus and move Text into Used text features.
Sentiment Analysis has no effect / no new columnsMake sure a line actually connects Corpus into it, and that Method is set to Vader.
Box Plot won't let you pick compoundYou connected it to the plain Corpus. Connect it to Sentiment Analysis instead, which is where the scores are created.
Emojis show as boxesHarmless — it's just a font display issue. VADER still reads them correctly.

DATA4500 — Marketing & Social Media Analytics · Week 3 Lab · Dataset: brewlab_sentiment_w3.csv (100 messages)