PRACTICAL 4: DATA PREPROCESSING - DATA INTEGRATION

Dataset: Social Media Engagement Dataset (sme.csv)

1. CORRELATION

CODE-

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

df = pd.read_csv("sme.csv")
num_df = df[['likes', 'comments', 'shares']]

correlation_matrix = num_df.corr()
print("Correlation matrix:")
print(correlation_matrix)

sns.heatmap(correlation_matrix,
            annot=True,
            cmap="coolwarm",
            linewidths=0.5)

plt.title("Correlation heatmap")
plt.show()

EXPLANATION-
The correlation matrix shows the strength and direction of relationships between likes, comments, and shares. The heatmap displays these correlations visually.


2. CHI SQUARE

Platform and sentiment_score are categorical variables.
Does the platform on which a post is published have any association with its sentiment?

CODE-

import pandas as pd
from scipy.stats import chi2_contingency

df = pd.read_csv("sme.csv")
table = pd.crosstab(df["platform"], df["sentiment_score"])

chi2, p, dof, expected = chi2_contingency(table)
print("\nChi square statistic:", chi2)
print("DOF:", dof)
print("P value:", p)
print("\nExpected frequencies:", expected)

if p < 0.05:
    print("\nReject H0. Platform and Sentiment are dependent.")
else:
    print("\nFail to Reject H0. Platform and Sentiment are independent.")

EXPLANATION-
The Chi-square test checks whether platform and sentiment_score are associated. If the p-value is less than 0.05, reject the null hypothesis; otherwise, fail to reject it.
