Metadata-Version: 2.4
Name: arclm
Version: 1.0.0
Summary: An ArcLM-first self-driving framework for causal language-model workflows.
Author: Ahmad Al Dibo
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/ahmad-al-dibo/arclm
Project-URL: Documentation, https://github.com/ahmad-al-dibo/arclm#readme
Project-URL: Source, https://github.com/ahmad-al-dibo/arclm
Project-URL: Issues, https://github.com/ahmad-al-dibo/arclm/issues
Keywords: arclm,self-driving,framework,workflow,sft,llm,language-model,causal-language-model,transformer,model-artifacts,pytorch,training,fine-tuning,instruction-tuning,nlp
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <3.13,>=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch<3,>=2.1
Requires-Dist: safetensors<1,>=0.4
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: build<2,>=1.2; extra == "dev"
Requires-Dist: twine<7,>=5; extra == "dev"
Requires-Dist: mkdocs<2,>=1.6; extra == "dev"
Requires-Dist: mkdocs-material<10,>=9; extra == "dev"
Requires-Dist: ruff<1,>=0.8; extra == "dev"
Requires-Dist: mypy<2,>=1.10; extra == "dev"
Requires-Dist: pip-audit<3,>=2.7; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs<2,>=1.6; extra == "docs"
Requires-Dist: mkdocs-material<10,>=9; extra == "docs"
Provides-Extra: tokenizers
Requires-Dist: sentencepiece<0.3,>=0.2; extra == "tokenizers"
Provides-Extra: all
Requires-Dist: torch<3,>=2.1; extra == "all"
Requires-Dist: sentencepiece<0.3,>=0.2; extra == "all"
Requires-Dist: safetensors<1,>=0.4; extra == "all"
Dynamic: license-file

# ArcLM

ArcLM is a compact Python framework for causal language-model workflows. The
vNext direction is ArcLM-owned: model architecture, tokenizer contracts, model
loading, training, fine-tuning, runtime behavior, and `.arcmodel` artifacts are
defined by ArcLM rather than by Hugging Face Transformers.

PyTorch is currently used as the tensor/autograd/runtime backend. `safetensors`
is used for model and adapter tensor storage.

## Install

```bash
pip install -e .
```

For development and documentation work:

```bash
pip install -e ".[dev]"
```

ArcLM supports Python `>=3.9,<3.13`.

## Student / Lab API

```python
from arclm import Lab

lab = Lab()
dataset = lab.dataset(text="alpha beta gamma delta alpha beta gamma delta")
model = lab.model(data=dataset, size="tiny")
history = lab.train(epochs=1, steps=1, shuffle=False)
```

## Professional API

```python
from arclm import Dataset, Model, Runtime, Tokenizer, Trainer
from arclm.training import FineTuningConfig, TrainingConfig

dataset = Dataset.load([{"text": "alpha beta gamma delta alpha beta gamma delta"}])
tokenizer = Tokenizer(strategy="word", max_vocab=16).build(dataset.text())
runtime = Runtime.auto(prefer="cpu")
model = Model.create(
    architecture="arclm-native",
    tokenizer=tokenizer,
    runtime=runtime,
    embed_dim=8,
    block_size=4,
    num_blocks=1,
)

config = TrainingConfig(epochs=1, steps_per_epoch=1, save_artifact=True, artifact_path="model.arcmodel")
history = Trainer(model=model, dataset=dataset, config=config, fine_tuning=FineTuningConfig(method="pretrain")).train()
```

## Research API

```python
from arclm.research import BaseStrategy, NextTokenLoss, TrainingConfig, Trainer

class MyLoss:
    def __call__(self, *, model, batch, engine):
        return NextTokenLoss()(model=model, batch=batch, engine=engine) * 0.5

strategy = BaseStrategy(name="half_loss", loss_function=MyLoss())
history = Trainer(model=model, dataset=dataset, config=TrainingConfig(epochs=1, steps_per_epoch=1), strategy=strategy).train()
```

## Current Support

- Native compact causal language model architecture: `arclm-native-causal-lm`.
- Unified `Tokenizer` facade with native word and character engines.
- Optional lazy SentencePiece tokenizer engine.
- ArcLM-owned training engine with deterministic per-epoch steps, validation,
  checkpoint/resume, callbacks, metrics, progress display, gradient
  accumulation, and early stopping.
- Full fine-tuning and native LoRA-style adapter fine-tuning.
- `.arcmodel` model artifacts and `.arcadapter` adapter artifacts.
- Three API levels over the same engine: Student, Professional, Research.

## Planned

- Optional Hugging Face compatibility adapters.
- Additional ArcLM-native model families.
- ArcLM-native BPE, WordPiece, and Unigram tokenizer engines.
- Richer artifact migration/version policies.

## Examples

The files in `examples/` are executable and covered by tests:

- `01_student_lab.py`
- `02_professional_training.py`
- `03_lora_finetuning.py`
- `04_save_load_arcmodel.py`
- `05_research_custom_loss.py`

## Development

```bash
python -m pytest tests
python -m mkdocs build --strict
```

## License

ArcLM is released under the [Apache License 2.0](LICENSE).
