Workshop 9 - Data clustering

• Activity 9.1: Data clustering methods

In this activity, we introduce methods to evaluate the quality of clustering, including the silhouette and elbow methods. We also explore K-means and hierarchical clustering models.

• Activity 9.2: Validation and evaluation

In this activity, we explore how to measure and address model uncertainty. We introduce cross-validation and bootstrapping methods.

Unsupervised learning

• Not all data can be driven by theory
• We still need work done
• Unsupervised learning: find patterns in data without predefined labels

One common way is cluster analysis

• Group similar data points together

But...

Uncertainty

• Model assumptions
• Parameter estimation
• Measurement errors

Ways to address uncertainty

• Cross-validation: resampling
• Bootstrapping: simulating sampling
• Bayesian methods: prior knowledge about the thing

Today's activity

Clustering methods [~40 mins]

• Quality of clustering
  • Silhouette method
  • Elbow method
• K-means clustering
• Hierarchical clustering

Validation and evaluation [~30 mins]

• Cross-validation
• Bootstrapping