Metadata-Version: 2.4
Name: mdr-readability
Version: 1.0.0
Summary: Multi-Dimensional Readability (MDR) score and 42 linguistic features for English text
Home-page: https://github.com/jacktanhua/MDR
Author: Hua Tan
Author-email: Hua TAN <jacktanhua@ccnu.edu.cn>
License: MIT
Project-URL: Homepage, https://github.com/jacktanhua/MDR
Project-URL: Issues, https://github.com/jacktanhua/MDR/issues
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Dynamic: author
Dynamic: home-page
Dynamic: requires-python

# mdr-readability

A Python package for computing the **MDR (Multi-Dimensional Readability)** score and 42 linguistic features from English text.

MDR is a regression-based readability index that combines lexical, syntactic, and semantic features to predict text difficulty. It achieves R² = 0.9249 on the calibration corpus.

---

## Installation

### 1. Install the package

```bash
pip install mdr-readability
```

Or, for development (editable install):

```bash
git clone https://github.com/jacktanhua/MDR.git
cd mdr-readability
pip install -e .
```

### 2. Install the spaCy model

```bash
python -m spacy download en_core_web_lg
```

### 3. Install NLTK data (first time only)

```python
import nltk
nltk.download("punkt")
nltk.download("wordnet")
nltk.download("averaged_perceptron_tagger")
```

### 4. Place the vocabulary data files

Two external data files are required:

| File | Description |
|------|-------------|
| `The New Dale-Chall Familiar Words List_2950.txt` | Dale-Chall familiar word list |
| `cefrj-vocabulary-profile-1.5.csv` | CEFR vocabulary profile (must have `headword` and `CEFR` columns) |

**Optional (for norm scores):**

| File | Description |
|------|-------------|
| `MDR_level_features_norm.csv` | Level-specific norm values (must have `Code` column + `L1`–`L12` columns) |

By default the package looks for these files in `<package_root>/data/`.  
You can point it to any directory:

```python
import mdr_readability
mdr_readability.set_data_dir("/path/to/your/data")
```

---

## Quick Start

```python
import mdr_readability

# Point to your data directory (skip if files are in the default location)
mdr_readability.set_data_dir("D:/MDR/data")

text = "The cat sat on the mat. It was a very small cat."

# --- Option 1: MDR score + 12 classic indices ---
df = mdr_readability.compute_mdr(text)
print(df)

# --- Option 2: MDR score + all 42 raw features + norm values ---
df_norm = mdr_readability.compute_mdr_with_norm(text)
print(df_norm.T)  # Transposing makes it easier to read

# --- Option 3: Step by step ---
features_df = mdr_readability.calculate_features(text)
features_df = mdr_readability.calculate_mdr_readability(features_df)
print(features_df["MDR"].iloc[0])

# --- Classic readability only ---
scores = mdr_readability.calculate_classic_readability(text)
labels = [
    "Flesch Reading Ease", "Flesch Kincaid Grade", "Gunning Fog",
    "SMOG Index", "Automated Readability", "Coleman Liau",
    "Linsear Write", "Dale Chall", "Spache", "Rix", "Lix", "Text Standard"
]
for label, score in zip(labels, scores):
    print(f"{label}: {score}")
```

---

## API Reference

### `mdr_readability.set_data_dir(path)`

Set the directory from which vocabulary data files are loaded.  
Call once before any computation when your data files are not in `<package_root>/data/`.

---

### `mdr_readability.calculate_features(text) → pd.DataFrame`

Extract all 42 linguistic features from `text`.  
Returns a single-row DataFrame with columns listed in `mdr_readability.COLUMN_NAMES`.

Feature categories:

| Category | Features (count) |
|----------|-----------------|
| Syllable | 7 |
| Word length & characters | 4 |
| Lexical difficulty | 6 |
| Type-token ratio / frequency | 3 |
| Sentence length | 2 |
| Dependency / syntax | 7 |
| Passive voice | 2 |
| Semantic | 7 |
| Referencing / conjunction | 4 |

---

### `mdr_readability.calculate_mdr_readability(df) → pd.DataFrame`

Apply the MDR linear regression formula to a features DataFrame.  
Returns a copy of `df` with an extra `"MDR"` column (rounded to 4 d.p.).

---

### `mdr_readability.calculate_classic_readability(text) → list`

Return a list of 12 classic readability scores in this order:

```
[Flesch Reading Ease, Flesch Kincaid Grade, Gunning Fog, SMOG Index,
 Automated Readability Index, Coleman Liau Index, Linsear Write Formula,
 Dale Chall Readability Score, Spache Readability, RIX, LIX, Text Standard]
```

---

### `mdr_readability.compute_mdr(text) → pd.DataFrame`

One-step convenience function.  
Returns a single-row DataFrame with `MDR` plus all 12 classic indices.

---

### `mdr_readability.compute_mdr_with_norm(text) → pd.DataFrame`

One-step convenience function with norm comparison.  
Returns a single-row DataFrame with:
- `MDR_value`
- All 42 raw feature values (`<feature_name>`)
- Corresponding norm values (`<feature_name>_norm`) — `None` if norm file is absent

---

## Feature Codes

Each feature has a short code used in the norm CSV:

| Code | Feature name |
|------|-------------|
| MWLS | avg_syllables_per_word_spacy |
| WO2S | words_over_2_syllables |
| W2SR | words_over_2_syllables_ratio |
| W2SE | words_over_2_syllables_entropy |
| W2SS | words_over_2_syllables_per_30_sentences |
| OSW1 | one_syllable_words_per_150 |
| OSW2 | one_syllable_words_per_100 |
| MWLL | Mean Word Length Refined |
| MLW  | average_letters_per_100_words |
| MSW  | Mean Sentence per Word |
| WLE  | Word Length Entropy |
| DWR  | Difficult Words Ratio |
| DWE  | difficult_words_entropy |
| WLEC | word_level_entropy_CERF |
| MLF  | Mean Lexical Frequency |
| LR   | Lexical Richness Entropy |
| STTR | Standard Type Token Ratio |
| WZE  | Word Zipf Entropy |
| MSL  | Mean Sentence Length |
| SLE  | Sentence Length Entropy |
| MDD  | Mean Dependency Distance |
| DDE  | Dependency Distance Entropy |
| DTE  | Dependency Distribution Entropy |
| SED  | Syntax Entropy Dependency |
| SEP  | Syntax Entropy POS |
| SEC  | syntax_entropy_component |
| PSR  | Passive Sentence Ratio |
| PDE  | Passive Dependency Entropy |
| TE   | Topic Entropy |
| SE   | Semantic Entropy |
| SR   | Semantic Richness |
| SAN  | Semantic Accuracy Noun |
| SAV  | Semantic Accuracy Verb |
| SANV | Semantic Accuracy Noun_Verb |
| SACW | Semantic Accuracy Content Words |
| SC   | Semantic Clarity |
| DSE  | Descriptive Style Entropy |
| POE  | POS Entropy |
| RE_I | Referencing Entropy I |
| RE_II| Referencing Entropy II |
| RE_III| Referencing Entropy III |
| CE   | Conjunction Entropy I |

---

## Dependencies

- `spacy >= 3.0` + `en_core_web_lg` model
- `spacy-syllables`
- `nltk` (punkt, wordnet, averaged_perceptron_tagger)
- `textstat`
- `wordfreq`
- `numpy`, `pandas`

---

## License

MIT
