Topic 4: Clustering

Introduction to Business Analytics — HKUST

Prof. Xuhu Wan

Topic 4: Finding Hidden Groups — Clustering Methods

ISOM 2600 — Business Statistics with Python

Professor Xuhu Wan · HKUST Business School

A conceptual shift from Topics 1–3: we are moving from supervised to unsupervised learning. No more \(Y\) variable — we discover structure in data without labels.

Outline

0. Why Clustering?
1. How Do We Measure Similarity?
2. K-Means Clustering
3. Hierarchical Clustering
4. Case Study: Forbes Financial Data
5. Case Study: Customer Segmentation
6. Summary

Why Clustering?

Learning Goals — Section 0

After this section you will be able to:

1. Explain the difference between supervised and unsupervised learning
2. Give three real business examples of clustering
3. Outline the five-step clustering workflow

Why Clustering? — Business Motivation

Clustering is everywhere in business:

• 💻 Netflix segments viewers into taste groups to recommend movies
• 💻 Spotify creates playlist groups from listening behaviour
• 💻 Banks segment credit-risk customers for pricing and marketing
• 💻 Retailers group products by purchasing patterns

Imagine a scatter plot with three clearly separated point clouds — Group A, Group B, Group C — sitting in different regions of (Feature 1, Feature 2) space.

Why it matters

In supervised learning (regression, Topics 1–3) we had labels \(Y\). In clustering we discover natural groups with no labels — this is unsupervised learning!

The Clustering Workflow

The five-step pipeline:

1. Collect Data → 2. Standardize Features → 3. Choose \(K\) → 4. Run Clustering → 5. Interpret Clusters

• Step 2 is critical — variables on different scales distort distances
• Step 3 uses the Elbow Plot (K-Means) or Dendrogram (Hierarchical)
• Step 5 is where business value lives — naming and acting on segments

Key takeaway

Clustering = finding natural groups in data without a target variable. Today we learn two methods: K-Means and Hierarchical Clustering.

Clustering in Everyday Life

You already cluster things every day:

• 🎵 Music playlist: You sort songs into “Workout,” “Chill,” “Study” — songs in the same playlist sound similar to each other
• 👕 Sorting laundry: Whites, darks, colours — you group by a feature (colour) without anyone telling you the groups
• 🍽️ Grocery store aisles: Dairy, produce, snacks — items that “belong together” are placed together

In each case, you look at features (tempo, colour, food type) and group items that are similar.

Three laundry baskets — Whites, Darks, Colours — illustrate that no one labels the items; you discover the groups yourself.

Key takeaway

Clustering is the same idea applied to data: the computer looks at numbers (features) and groups data points that are close to each other. The key question is: how do we measure “closeness”?

How Do We Measure Similarity?

Learning Goals — Section 1

After this section you will be able to:

1. Compute Euclidean distance between two data points by hand
2. Compare Euclidean, Manhattan, and correlation-based distance
3. Explain why standardization is essential before clustering
4. Apply StandardScaler in Python

Euclidean Distance

For two points \(A=(a_1,\dots,a_p)\) and \(B=(b_1,\dots,b_p)\):

\[ d(A,B) = \sqrt{(a_1-b_1)^2 + (a_2-b_2)^2 + \cdots + (a_p-b_p)^2} \]

Example (2 features):

• Customer \(A\): Age = 35, Income = 60k
• Customer \(B\): Age = 28, Income = 80k

\[ d = \sqrt{(35-28)^2 + (60-80)^2} = \sqrt{49+400} = \sqrt{449} \approx 21.2 \]

Euclidean Distance: Step-by-Step Hand Calculation

Let us slow down and compute the distance between two customers, one step at a time.

	Age	Income ($k)	Spending Score
Customer A	25	40	70
Customer B	30	55	50

Step 1: Subtract each feature of \(B\) from \(A\): \[(25-30),\quad (40-55),\quad (70-50) = -5,\quad -15,\quad 20\]

Step 2: Square each difference: \[(-5)^2,\quad (-15)^2,\quad (20)^2 = 25,\quad 225,\quad 400\]

Step 3: Add the squared differences: \(25 + 225 + 400 = 650\)

Step 4: Square root: \(d(A,B) = \sqrt{650} \approx 25.5\)

Important

Notice that Spending Score (difference of 20) and Income (difference of 15) contribute much more than Age (difference of 5). Features with larger numbers dominate the distance — this is why we need standardization!

Other Distance Measures

Distance	Formula	When to use
Euclidean	\(\sqrt{\sum (a_i - b_i)^2}\)	Default; continuous features
Manhattan	\(\sum \lvert a_i - b_i \rvert\)	Robust to outliers; grid-like data
Correlation	\(1 - \text{corr}(A,B)\)	When shape matters, not magnitude

Important

Different distance metrics can produce very different clusters! Always think about what “similar” means for your business problem.

Why Standardize?

Problem: Without standardizing, the axis with larger numbers dominates K-Means.

Solution: \(z_i = \dfrac{x_i - \bar{x}}{s_x}\) (mean \(=0\), std \(=1\))

Key takeaway

Always standardize before clustering so every feature gets an equal vote.

K-Means Clustering

Learning Goals — Section 2

After this section you will be able to:

1. Describe the five steps of the K-Means algorithm
2. Use sklearn.cluster.KMeans to cluster data
3. Construct and interpret an Elbow Plot to choose \(K\)
4. Explain what WCSS (inertia) measures

K-Means: The Algorithm

1. Choose \(K\) initial centroids (randomly)
2. Assign each point to the nearest centroid
3. Recompute centroids as cluster means
4. Reassign points to new nearest centroids
5. Repeat steps 2–4 until no points change cluster

The algorithm alternates between assignment and update. It always converges, though possibly to a local optimum.

What Is a Centroid?

A centroid is simply the average position of all points in a cluster — the “center of gravity.”

Analogy: Imagine balancing a cardboard cutout on your fingertip. The balance point is the centroid.

Numerical example:

	Age	Income
Customer 1	20	30
Customer 2	30	50
Customer 3	40	40
Centroid	30	40

\[\text{Centroid} = \left(\frac{20+30+40}{3},\; \frac{30+50+40}{3}\right) = (30, 40)\]

Key takeaway

The centroid is the mean of all points in the cluster. K-Means works by repeatedly moving centroids to better represent their members.

K-Means Step 1: Initialize Random Centroids

How it starts:

1. We choose \(K\) (e.g., \(K=3\))
2. The algorithm picks 3 random points as initial “centroids”
3. All data points are currently unassigned

Important

Different random starting positions can lead to different final clusters! That is why n_init=10 runs the algorithm 10 times and keeps the best result.

K-Means Step 2: Assign Each Point to Nearest Centroid

Assignment rule: For each data point, compute the Euclidean distance to every centroid. Assign the point to the closest one.

Example: Point \(P=(1.0, 4.0)\)

• \(d(P, \mu_1) = \sqrt{0 + 1} = 1.0\)
• \(d(P, \mu_2) = \sqrt{4 + 4} = 2.8\)
• \(d(P, \mu_3) = \sqrt{2.25 + 0.25} = 1.6\)

\(\Rightarrow\) Closest is \(\mu_1\), so \(P\) joins the red cluster.

After assigning all points, each one is coloured by its cluster.

K-Means Step 3: Recompute Centroids (Cluster Averages)

Update rule: compute the mean of all member points \(\rightarrow\) new centroid.

Red cluster: \((0.5,1.5),\;(1.0,2.0),\;(0.8,1.2),\;(1.5,1.8),\;(1.2,1.0)\)

\[\mu_1 = \left(\frac{0.5+1.0+0.8+1.5+1.2}{5},\;\frac{1.5+2.0+1.2+1.8+1.0}{5}\right) = (1.0,\,1.5)\]

Centroid moved from \((1.0, 3.0)\) to \((1.0, 1.5)\)!

⭐ Key: This is why it is called K-Means — centroids are the mean of their members.

K-Means Step 4: Repeat Until Convergence

Now repeat Steps 2–3:

1. Re-assign each point to the nearest new centroid
2. Recompute centroids again
3. Repeat until no points change cluster

Typically converges in 10–20 iterations.

⭐ The full K-Means loop:

Assign \(\rightarrow\) Update \(\rightarrow\) Assign \(\rightarrow\) Update \(\rightarrow\) … until nothing changes.

K-Means in Python

• n_clusters: number of clusters \(K\) (you choose)
• random_state: reproducibility seed
• n_init=10: run algorithm 10 times, keep best

Choosing \(K\): The Elbow Plot

WCSS (Within-Cluster Sum of Squares): \[\text{WCSS} = \sum_{k=1}^{K} \sum_{x_i \in C_k} \| x_i - \mu_k \|^2\]

• WCSS always decreases as \(K\) increases
• Look for the “elbow” — the point where adding more clusters gives diminishing returns

WCSS: What Does It Actually Measure?

WCSS stands for Within-Cluster Sum of Squares. In plain language:

1. For each cluster, measure how far each point is from the cluster’s centroid
2. Square those distances (so bigger gaps count more)
3. Add them all up across every cluster

Lower WCSS = tighter clusters = better fit.

Think of it like this: WCSS measures how “spread out” the points are within their assigned groups. If every point sat exactly on its centroid, WCSS would be zero (perfect, but unrealistic).

Imagine a centroid star with dashed lines connecting it to each member point. WCSS is the sum of the squared lengths of all those dashed lines.

Why it matters

The Elbow Plot shows WCSS for different values of \(K\). As you add more clusters, WCSS always goes down (more centroids = shorter distances). The “elbow” is where the improvement slows dramatically — adding more clusters is not worth the extra complexity.

Try It! — K-Means on Customer Data

Try it!

Challenge: You have a dataset with 200 mall customers (Age, Annual Income, Spending Score).

1. Standardize the features using StandardScaler
2. Create an Elbow Plot for \(K = 1, 2, \ldots, 10\)
3. Apply K-Means with your chosen \(K\)
4. Add the cluster labels to the DataFrame and create a scatter plot of Income vs. Spending Score, coloured by cluster

Hint: Look at the notebook Section 5 for the full walkthrough!

K-Means: Pitfalls and Limitations

1. Sensitive to initialization Different random starts → different clusters. Fix: n_init=10 (run 10 times, keep best).

2. Sensitive to outliers Extreme points pull centroids away from true centers.

3. Assumes spherical clusters Fails on elongated or crescent-shaped groups.

4. Must choose \(K\) in advance Elbow plot helps, but it is a judgment call.

K-Means struggles with: (i) two interlocking crescent-shaped clusters, (ii) an extreme outlier that drags the centroid toward itself.

Key takeaway

⚠️ Always visualise your clusters and check if they make business sense!

Hierarchical Clustering

Learning Goals — Section 3

After this section you will be able to:

1. Explain agglomerative (bottom-up) hierarchical clustering
2. Compare four linkage methods: Single, Complete, Average, Ward
3. Read and interpret a dendrogram
4. Cut a dendrogram to obtain \(K\) clusters
5. Implement hierarchical clustering in Python with scipy and sklearn

Hierarchical Clustering: Bottom Up

Picture three stages:

• Start: \(N\) clusters — each point A, B, C, D is its own cluster
• Merge closest — A and B join into one cluster
• Merge again — {A, B} joins with C into {A, B, C}; D remains separate

Agglomerative algorithm:

1. Start with each data point as its own cluster (\(N\) clusters)
2. Compute distance between all pairs of clusters
3. Merge the two closest clusters
4. Repeat steps 2–3 until only 1 cluster remains

Why it matters

Unlike K-Means, hierarchical clustering does not require you to pre-specify \(K\). You choose \(K\) after by cutting the dendrogram.

How to Measure Cluster Distance? — Linkage Methods

Linkage	Distance between clusters	Characteristics
Single	Min distance between any two points	Can create long “chaining” clusters
Complete	Max distance between any two points	Tends to produce compact clusters
Average	Mean distance between all point pairs	Compromise between single & complete
Ward	Increase in total within-cluster variance	Most balanced, compact clusters

Important

Ward linkage tends to give the most balanced, well-separated clusters. It is the recommended default for most business applications.

Computing Average Linkage: A Worked Example

Suppose we want to merge cluster \(\{A, B\}\) with cluster \(\{C, D\}\) using Average Linkage.

Average linkage = mean of all pairwise distances between the two clusters.

Pair	Distance
\(d(A, C)\)	5
\(d(A, D)\)	9
\(d(B, C)\)	4
\(d(B, D)\)	7
Average	\(\frac{5+9+4+7}{4} = \mathbf{6.25}\)

There are \(2 \times 2 = 4\) pairs between the two clusters, so we average all 4 distances.

Compare with other linkages:

• Single = \(\min(5,9,4,7) = 4\)
• Complete = \(\max(5,9,4,7) = 9\)
• Average = \(6.25\) (in between!)

Hierarchical Clustering: Merge Example with 5 Points

Distance matrix (Euclidean):

	A	B	C	D	E
A	0	2	6	10	9
B	2	0	5	9	8
C	6	5	0	4	5
D	10	9	4	0	3
E	9	8	5	3	0

Step-by-step merges (single linkage — use minimum distance):

1. Merge A & B (distance = 2, the smallest)
2. Merge D & E (distance = 3, next smallest)
3. Merge C with {D,E} (distance = min(5,4,5) = 4)
4. Merge {A,B} with {C,D,E} (distance = min(6,5,…) = 5)

The resulting mini-dendrogram has merges at heights 2, 3, 4, and 5.

The Dendrogram: Reading the Tree

A dendrogram tree:

• A and B merge at height 1
• D and E merge at height 1.5
• {A,B} joins C at height 3
• {A,B,C} joins {D,E} at height 5.5
• A red dashed cut line at height 2.5 gives K = 3

How to read:

• Leaves (bottom) = individual data points
• Height of a merge = distance at which two clusters joined
• Cut the tree horizontally to get \(K\) clusters
• Cutting at height 2.5 gives 3 clusters: {A,B}, {C}, {D,E}

Key takeaway

Tall vertical lines before a merge mean those clusters are well separated. Short lines mean the groups are similar.

Dendrogram in Python

scipy gives you the dendrogram visualization; sklearn gives you the cluster labels directly.

Try It! — Hierarchical vs. K-Means

Try it!

Challenge: Using the same customer dataset:

1. Build a dendrogram using Ward linkage
2. Cut it to get 5 clusters
3. Compare the cluster assignments to those from K-Means (\(K=5\))
4. Do the two methods agree? On how many customers do they disagree?

Hint: Use scipy.cluster.hierarchy.fcluster(Z, t=5, criterion='maxclust')

K-Means vs. Hierarchical: When to Use Which?

Decision flowchart:

• Know \(K\)?
  • No → Hierarchical (explore tree)
  • Yes → next question
• Large data (\(n > 1000\))?
  • Yes → K-Means (faster)
  • No → Try both!

Practical decision rules:

• K-Means: fast, simple, works well with round, evenly sized clusters
• Hierarchical: see the full merging structure; decide \(K\) after
• When in doubt: run both and compare — if they agree, more confidence in your clusters

⭐ Bottom line: No single “correct” method. Try both, compare, and let the data guide you.

Case Study: Forbes Financial Data

Forbes Data: Overview

Dataset: 25 Forbes-listed companies

Variables (7 financial metrics):

• PE ratio, 5-year return on revenue
• Debt-to-equity ratio
• 5-year sales growth, 5-year EPS growth
• Net profit margin, payout ratio

Known industries: Chemical (14), Healthcare (5), Grocery (6)

The question

Can the algorithm discover industry groups without knowing the labels?

# Static example — uses a Google Drive CSV not reachable from Pyodide
Forbes = pd.read_csv(
  "https://drive.google.com/uc?id=1atoN7jinnluf9U3vhTJ7b4j3xhCJEuVP")
X = Forbes.iloc[:, 3:]        # numeric columns only
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

Forbes: Building the Dendrogram

• We use Ward linkage (minimises within-cluster variance) — the recommended default
• The red dashed line shows where we cut to get \(K=3\) clusters
• We then compare the algorithm’s clusters to the known industry labels

Forbes: The Dendrogram

Reading the Forbes dendrogram (from the previous slide’s plot):

• Companies that merge at low height are very similar (e.g., chemical firms with comparable PE and payout ratios)
• The red dashed line cuts the tree into 3 clusters
• Read bottom to top: which companies group together?

Trace specific company codes (C1, C2, …, H1, H2, …, G1, G2, …) and note that the healthcare firms (H*) tend to form their own branch high up — they look very different from the chemicals and groceries on financial metrics alone.

Forbes: Interpretation and Cross-Tab

Cluster vs. Industry cross-tabulation:

Industry	C1	C2	C3
Chemical (14)	0	11	3
Grocery (6)	0	4	2
Healthcare (5)	4	0	1

Reading the table:

• C1 = almost pure Healthcare (4 of 5)
• C2 = mostly Chemical + Grocery
• C3 = mixed (3 Chem + 2 Groc + 1 Heal)

Key insights:

• Healthcare is the most distinct — high PE, high growth, high margins
• Chemical and Grocery overlap — similar margins and payout ratios
• “Misclassified” companies have financial profiles resembling another industry

⭐ Takeaway: Clustering recovered meaningful industry groups from financial data alone — companies in the same industry share “financial DNA.”

Case Study: Customer Segmentation

Mall Customer Segmentation: Business Context

Scenario: A shopping mall wants to understand its customers to target marketing.

Variables:

• 💻 Age (18–70)
• 💻 Annual Income ($15k–$140k)
• 💻 Spending Score (1–100)

Goal: Segment 200 customers into meaningful groups for targeted marketing campaigns.

A 2D map of (Income, Spending) showing five blobs in different corners — high-income/high-spending VIPs, high-income/low-spending careful spenders, low-income/high-spending impulsive buyers, low-income/low-spending budget shoppers, and middle-of-the-pack average customers.

Mall Customers: Elbow Plot

The elbow at \(K=5\) suggests five natural customer segments.

Mall Customers: K-Means with K=5

The plot shows five clearly separated groups in (Income, Spending) space — one in each “corner” plus one in the middle.

Mall Customers: Naming the Segments

Cluster	Name	Income	Score	Marketing Strategy
1	VIP Clients	High	High	Loyalty programs, exclusive events
3	Impulsive Buyers	Low	High	Affordable premium, BNPL
0	Average Customers	Mid	Mid	Newsletters, general promos
2	Careful Spenders	High	Low	Personalized outreach, rewards
4	Budget Shoppers	Low	Low	Discount bundles, value packs

Key takeaway

Clustering transforms raw data into actionable business segments. Each cluster gets a tailored strategy — that is the real value of unsupervised learning.

Profiling Clusters with groupby

How do we know what each cluster “looks like”? Use groupby to compute the mean of each feature per cluster — the same pandas skill from Topic 2!

Example output (numbers vary slightly by seed):

Cluster	Avg Age	Avg Income ($k)	Avg Spending Score
0	42.7	55.3	49.5
1	32.7	86.5	82.1
2	41.1	88.2	17.1
3	25.3	25.7	79.4
4	45.2	26.3	20.9

Why it matters

The cluster profile table is how you name your segments. Cluster 1 has high income and high spending \(\Rightarrow\) “VIP.” Cluster 4 has low income and low spending \(\Rightarrow\) “Budget.” The numbers tell the story!

Try it!

Extension: Try mall.groupby('Cluster').describe() to see not just the mean, but also min, max, and standard deviation for each cluster. Which cluster has the most variation?

Summary

Supervised vs. Unsupervised Learning

	Supervised	Unsupervised
	(Topics 1–3)	(Topic 4)
Goal	Predict \(Y\)	Find groups
Labels?	Yes (need \(Y\))	No (no target)
Methods	Regression, classification	Clustering, PCA
Evaluation	\(R^2\), MSE, accuracy	Inertia, silhouette, domain expertise
Example	Predict house price	Segment customers

K-Means vs. Hierarchical Clustering

	K-Means	Hierarchical
Choose \(K\)?	Before running	After (cut dendrogram)
Speed	Fast (large datasets)	Slower (\(O(n^2)\) memory)
Visualization	Scatter plot	Dendrogram
Shape	Spherical clusters	Any shape
Reproducibility	Depends on init	Deterministic
Best for	Large \(n\), clear groups	Small \(n\), exploring structure

Key takeaway

Use K-Means for speed and simplicity. Use Hierarchical for richer exploration and when you are unsure about \(K\). In practice, try both and compare!

End of Topic 4

← Back to home