PRACTICAL 9: CLUSTERING (K-MEANS AND HIERARCHICAL)

Dataset: Wine dataset (load_wine from scikit-learn)

STEPS:
1. Install the libraries:  pip install numpy pandas matplotlib scipy scikit-learn
2. Load the Wine dataset into a DataFrame and print its summary.
3. Scale the data with StandardScaler.
4. Use the Elbow method (K = 1 to 10) to find the best number of clusters.
5. Apply K-Means with K = 3 and calculate the silhouette score.
6. Reduce the data to 2 dimensions with PCA and plot the K-Means clusters.
7. Draw the dendrogram (ward linkage) for hierarchical clustering.
8. Apply Agglomerative clustering with 3 clusters, calculate its silhouette score and plot the clusters (alcohol vs malic acid).

CODE:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from scipy.cluster.hierarchy import dendrogram, linkage
from sklearn.cluster import AgglomerativeClustering, KMeans
from sklearn.datasets import load_wine
from sklearn.decomposition import PCA
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler
wine = load_wine()
wine_df = pd.DataFrame(data=wine.data, columns=wine.feature_names)
print("Data Summary:")
print(wine_df.describe())
scaler = StandardScaler()
X_scaled = scaler.fit_transform(wine_df)
wss = []
for k in range(1, 11):
    kmeans = KMeans(n_clusters=k, random_state=42, n_init=10)
    kmeans.fit(X_scaled)
    wss.append(kmeans.inertia_)
plt.figure(figsize=(8, 4))
plt.plot(range(1, 11), wss, marker="o")
plt.xlabel("Number of clusters k")
plt.ylabel("Total within-cluster sum of squares (Inertia)")
plt.title("Elbow Method for Optimal K")
plt.show()
kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
wine_df["kmeans_cluster"] = kmeans.fit_predict(X_scaled)
silhouette_avg_kmeans = silhouette_score(X_scaled, wine_df["kmeans_cluster"])
print(
    f"\nMean Silhouette Width for K-Means Clustering: {silhouette_avg_kmeans:.4f}"
)
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)
plt.figure(figsize=(8, 5))
plt.scatter(
    X_pca[:, 0], X_pca[:, 1], c=wine_df["kmeans_cluster"], cmap="viridis", s=50
)
plt.title("K-Means Clustering (PCA Reduced)")
plt.xlabel("Principal Component 1")
plt.ylabel("Principal Component 2")
plt.show()
linkage_matrix = linkage(X_scaled, method="ward")
plt.figure(figsize=(10, 5))
dendrogram(linkage_matrix)
plt.title("Hierarchical Clustering Dendrogram")
plt.xlabel("Sample Index")
plt.ylabel("Distance")
plt.show()
agglomerative = AgglomerativeClustering(n_clusters=3)
wine_df["hierarchical_cluster"] = agglomerative.fit_predict(X_scaled)
silhouette_avg_hierarchical = silhouette_score(
    X_scaled, wine_df["hierarchical_cluster"]
)
print(
    f"Mean Silhouette Width for Hierarchical Clustering: {silhouette_avg_hierarchical:.4f}"
)
plt.figure(figsize=(8, 5))
plt.scatter(
    wine_df["alcohol"],
    wine_df["malic_acid"],
    c=wine_df["hierarchical_cluster"],
    cmap="viridis",
    alpha=0.8,
    s=50,
)
plt.xlabel("Alcohol")
plt.ylabel("Malic Acid")
plt.title("Hierarchical Clustering (Alcohol vs Malic Acid)")
plt.show()

EXPLANATION:
- Clustering is unsupervised: it groups similar wines together without using any class labels.
- The elbow plot shows the inertia for K = 1 to 10, and the bend of the curve gives the best K (here 3).
- K-Means puts each wine in the nearest of 3 centres, and PCA reduces the data to 2D only to draw the clusters.
- The dendrogram shows how hierarchical (ward) clustering merges the samples step by step, and Agglomerative clustering cuts it into 3 clusters.
- The silhouette score (closer to 1 is better) is used to compare K-Means and hierarchical clustering.
