Metadata-Version: 2.4
Name: mctn
Version: 0.1.0
Summary: Multi-Channel Text Normalizer for Chinese NLP
Author-email: xushuaike <3300536132@qq.com>
License: MIT
Project-URL: Homepage, https://github.com/xushuaike/MCTN
Project-URL: Repository, https://github.com/xushuaike/MCTN
Keywords: nlp,chinese,text-classification,pytorch,mctn
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=1.12
Requires-Dist: numpy
Requires-Dist: jieba
Requires-Dist: pypinyin
Requires-Dist: scikit-learn
Requires-Dist: matplotlib
Provides-Extra: ddparser
Requires-Dist: ddparser; extra == "ddparser"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

# MCTN-NLP: Multi-Channel Text Normalizer

[![PyPI version](https://badge.fury.io/py/mctn-nlp.svg)](https://pypi.org/project/mctn-nlp/)

MCTN（多通道文本归一化器）是一个专为中文 NLP 设计的**特征增强模块**。它从字形、语音、语义、句法 7 个维度提取语言学特征，输出与下游模型维度对齐的密集向量，可无缝替换任何现有的 Embedding 输入。

## 核心特性

- **7 通道特征增强**：字面语义、语义聚类、POS 句法、句子骨架、局部上下文、拼音、字形
- **架构无关**：输出 `[B, L, d]`，可搭配 Transformer / BiLSTM / CNN 任意下游
- **低资源友好**：数据越少，MCTN 的知识补偿效应越明显
- **工程落地**：支持 Engine 序列化、笔画离线缓存、延迟基准拆分

## 快速开始

```bash
pip install mctn-nlp
```

```python
from mctn import DynamicEngine, MCTNFeatureExtractor, DownstreamTransformer
from mctn.data import Vocab, MCDataset

# 1. 准备数据
texts = ["示例文本", "更多文本"]
labels = [0, 1]

# 2. 构建引擎（只需执行一次，可保存复用）
engine = DynamicEngine(n_sem=30)
engine.fit(texts, use_ddparser=False)
engine.save("engine.pkl")

# 3. 构建词表
vocab = Vocab()
vocab.build(texts)

# 4. 组装模型
extractor = MCTNFeatureExtractor(
    vs=vocab.size, d=128, np_=len(engine.POS2ID),
    sem_dim=engine.sem_dim, svs=engine.sim_vocab_size,
    n_ini=engine.n_ini, n_medial=engine.n_medial,
    n_nucleus=engine.n_nucleus, n_coda=engine.n_coda,
    n_cb=engine.n_cjk_block, n_cm=engine.n_complexity_bins,
    n_rd=engine.n_radical,
    sem_table=engine.build_semantic_table(vocab.c2i),
    glyph_tables=engine.build_glyph_table(vocab.c2i),
)
model = DownstreamTransformer(d=128, nh=4, nl=2, nc=2, ml=64)

# 5. 前向
import torch
token_ids = torch.tensor([vocab.encode(texts[0], 64)])
x = extractor(token_ids)  # [1, 64, 128]
logits = model(x)
```

## 完整训练示例

```python
from torch.utils.data import DataLoader
from mctn.training import run_training

# 构建数据集
dataset = MCDataset(list(zip(texts, labels)), vocab, engine, ml=64)
loader = DataLoader(dataset, batch_size=32, shuffle=True)

# 训练
hist = run_training(model, loader, loader, epochs=10, lr=3e-4)
```

## 依赖

- torch >= 1.12
- numpy
- jieba
- pypinyin
- scikit-learn

可选依赖：
- `ddparser`：启用依存句法分析（默认关闭，较重）

## 引用

如果本工作对你的研究有帮助，请引用：

```bibtex
@software{mctn_nlp,
  title = {MCTN-NLP: Multi-Channel Text Normalizer},
  author = {Your Name},
  year = {2026},
  url = {https://github.com/yourusername/mctn-nlp}
}
```

## License

MIT License
