Metadata-Version: 2.4
Name: mblt-model-zoo
Version: 2.4.0
Summary: A codebase for pre-quantized AI models for Mobilint NPUs.
Author-email: "Mobilint Inc." <tech-support@mobilint.com>
License: BSD-3-Clause
Project-URL: Home, https://www.mobilint.com/
Project-URL: Repository, https://github.com/mobilint/mblt-model-zoo
Keywords: quantization,NPU,model zoo,pre-quantized models,inference,mobilint,mblt,aries,regulus,qb
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <3.13,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mobilint-qb-runtime>=1.2.0
Requires-Dist: numpy>=1.26.0
Requires-Dist: torch>=2.4.1
Requires-Dist: opencv-python>=4.11.0.86
Requires-Dist: pillow>=11.1.0
Requires-Dist: faster-coco-eval
Requires-Dist: huggingface-hub
Requires-Dist: scipy
Requires-Dist: requests>=2.32.0
Requires-Dist: gdown>=5.2.0
Requires-Dist: tqdm
Requires-Dist: mblt-tracker>=1.0.1
Requires-Dist: PyYAML
Requires-Dist: matplotlib
Requires-Dist: cityscapesScripts
Requires-Dist: mblt-npu-python
Requires-Dist: mblt-vision-python>=0.0.2
Provides-Extra: transformers
Requires-Dist: transformers[serving]<=5.12.1,>=4.54.0; extra == "transformers"
Requires-Dist: datasets<=3.6.0,>=3.2.0; extra == "transformers"
Requires-Dist: sentencepiece; extra == "transformers"
Requires-Dist: torchvision; extra == "transformers"
Requires-Dist: torchcodec; extra == "transformers"
Provides-Extra: melotts
Requires-Dist: transformers<=5.12.1,>=4.54.0; extra == "melotts"
Requires-Dist: g2p_en>=2.1.0; extra == "melotts"
Requires-Dist: anyascii>=0.3.2; extra == "melotts"
Requires-Dist: jamo>=0.4.1; extra == "melotts"
Requires-Dist: g2pkk>=0.1.1; extra == "melotts"
Requires-Dist: unidic>=1.1.0; extra == "melotts"
Requires-Dist: python-mecab-ko>=1.3.7; extra == "melotts"
Requires-Dist: soundfile; extra == "melotts"
Requires-Dist: gradio; extra == "melotts"
Requires-Dist: click; extra == "melotts"
Provides-Extra: onnxruntime
Requires-Dist: onnx; extra == "onnxruntime"
Requires-Dist: onnxruntime; extra == "onnxruntime"
Provides-Extra: onnxruntime-gpu
Requires-Dist: onnx; sys_platform != "darwin" and extra == "onnxruntime-gpu"
Requires-Dist: onnxruntime-gpu; sys_platform != "darwin" and extra == "onnxruntime-gpu"
Provides-Extra: qwen-asr
Requires-Dist: qwen-asr; extra == "qwen-asr"
Provides-Extra: qbcompiler
Requires-Dist: qbcompiler>=1.2.0; extra == "qbcompiler"
Requires-Dist: onnxruntime; extra == "qbcompiler"
Dynamic: license-file

# Mobilint Model Zoo

<!-- markdownlint-disable MD033 -->
<div align="center">
<p>
<a href="https://www.mobilint.com/" target="_blank">
<img src="https://raw.githubusercontent.com/mobilint/.github/main/assets/Mobilint_Logo_Primary.png" alt="Mobilint Logo" width="60%">
</a>
</p>
</div>
<!-- markdownlint-enable MD033 -->

**mblt-model-zoo** is a curated collection of AI models optimized by [Mobilint](https://www.mobilint.com/)’s Neural Processing Units (NPUs).

Designed to help developers accelerate deployment, Mobilint's Model Zoo offers access to public, pre-trained, and pre-quantized models for vision, language, and multimodal tasks. Along with performance results, we provide pre- and post-processing tools to help developers evaluate, fine-tune, and integrate the models with ease.

## Installation

[![PyPI - Version](https://img.shields.io/pypi/v/mblt-model-zoo?logo=pypi&logoColor=white)](https://pypi.org/project/mblt-model-zoo/)
[![PyPI Downloads](https://static.pepy.tech/badge/mblt-model-zoo?period=total&units=INTERNATIONAL_SYSTEM&left_color=BLACK&right_color=GREEN&left_text=downloads)](https://clickpy.clickhouse.com/dashboard/mblt-model-zoo)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/mblt-model-zoo?logo=python&logoColor=gold)](https://pypi.org/project/mblt-model-zoo/)

- Prepare environment equipped with Mobilint's NPU. In case you are not a Mobilint customer, please contact [us](mailto:tech-support@mobilint.com).
- Install **mblt-model-zoo** using pip:

```bash
pip install mblt-model-zoo
```

- If you want to install the latest version from the source, clone the repository and install it:

```bash
git clone https://github.com/mobilint/mblt-model-zoo.git
cd mblt-model-zoo
pip install -e .
```

Release notes are tracked in [CHANGELOG.md](CHANGELOG.md).

## Vision

Vision models, model metadata, preprocessing, postprocessing, datasets, evaluation, and Python API
documentation are maintained in
[mblt-vision-python](https://github.com/mobilint/mblt-vision-python). Install it
and import from `mblt_vision` for new applications. `mblt_model_zoo.vision`
remains available as a forwarding-only compatibility facade for existing code.

Model Zoo retains compatibility bridges for the Vision CLI and compilation.
Vision benchmarks and dataset-management workflows are maintained in
[mblt-vision-python](https://github.com/mobilint/mblt-vision-python/tree/main/benchmark).

## Optional Extras

When working with tasks other than vision, extra dependencies may be required. Those options can be installed via `pip install mblt-model-zoo[NAME]` or `pip install -e .[NAME]`.

Currently, these optional functions are only available on environment equipped with Mobilint's [ARIES](https://www.mobilint.com/aries).

|Name|Use|Details|
|-------|------|------|
|transformers|For using Hugging Face Transformers related models|[README.md](mblt_model_zoo/hf_transformers/README.md)|
|MeloTTS|For using MeloTTS models|[README.md](mblt_model_zoo/MeloTTS/README.md)|
|qbcompiler|For generating mxq files with custom setting|[README.md](compile/README.md)|

The `qbcompiler` extra is strictly isolated from ordinary package use. qbcompiler is loaded only
when `compile_vision_model()` or `mblt-model-zoo compile` actually starts compilation. The base
package, vision APIs, compilation module import, and non-compile CLI commands continue to work
without qbcompiler installed; only a compilation request reports the installation error.

For the `transformers` extra, the repository also includes:

- functional test instructions in [tests/transformers/TEST.md](tests/transformers/TEST.md)
- benchmark script usage in [benchmark/transformers/README.md](benchmark/transformers/README.md)

> Note: The `MeloTTS` extra includes `unidic`, which requires an additional dictionary download step. Python packaging (PEP 517/518) does not support running arbitrary post-install commands automatically, so run `mblt-unidic-download` (or `python -m unidic download`) after installing the extra when needed.

## Command Line Interface

Installing this package exposes the `mblt-model-zoo` console command:

```bash
mblt-model-zoo -h
```

The CLI provides Mobilint-specific helper commands and delegates selected upstream Hugging Face
Transformers commands to the installed `transformers` package.

The built-in command surface shown by `mblt-model-zoo -h` is:

- `predict` — run classification, depth estimation, object or face detection, instance or semantic
  segmentation, OBB, and pose inference.
- `val` — validate a vision model on its benchmark dataset.
- `compile` — compile a configured vision ONNX model to MXQ.
- `tps measure` and `tps sweep` — run Transformers token-per-second benchmarks.
- `melo` — run the MeloTTS CLI; `melotts` is an alias.
- `melo-ui` — launch the MeloTTS Gradio WebUI.

Run `mblt-model-zoo <command> -h` for argparse-based commands. `melo` and `melotts` are
Click-based and use `--help`.

Compile a configured vision ONNX model with the optional compiler dependency:

```bash
mblt-model-zoo compile --model-cls alexnet
```

The matching Python API is `mblt_model_zoo.compile.vision.compile_vision_model`. New
applications should use `mblt_vision.compile.compile_vision_model`; see the
[standalone Vision compilation guide](https://github.com/mobilint/mblt-vision-python/tree/main/compile/vision)
for calibration datasets and options.
Invoking this API or command is the only point where qbcompiler is imported.

When paths are omitted, compilation stores downloaded ONNX models and compiled MXQ outputs under
`~/.mblt_model_zoo`, and uses registry datasets under `~/.mblt_model_zoo/datasets`. It does not
derive these defaults from the current checkout or working directory.

Compilation accepts one of three data entry levels: `--data-path` for a full organized image
dataset, `--subset-path` for already-sampled images, or `--calib-data-path` for ready preprocessed
`.npy` tensors. Later-stage input skips all earlier preparation stages.
`--model-path` also accepts the `--onnx-path` compatibility alias, and `--calib-data-path` accepts
the `--calib-data-dir` alias.

### Vision Prediction And Validation

The vision CLI runs the same preprocess, NPU inference, postprocess, and plotting pipeline used by
the Python API. Use `predict` with a source image and a model name; the task is inferred from the
model configuration. It supports image classification, depth estimation, object and face detection,
instance and semantic segmentation, oriented bounding boxes (OBB), and pose estimation.
`classify`, `detect`, `pose`, and `segment` are also accepted as aliases.

```bash
mblt-model-zoo predict --source ./cat.png --model resnet50
mblt-model-zoo predict --source ./street.jpg --model yolo11m --output ./result_detect.jpg
```

Vision commands accept a shared `--model-path` for local MXQ and local ONNX files. When
`--framework` is omitted, the CLI infers `.mxq` and `.onnx` suffixes and otherwise falls back to
MXQ. If the explicit framework conflicts with the local file suffix, the command fails with a
clear error. The compatibility flags `--mxq-path` and `--onnx-path` stay separate from
`--model-path`, so framework-specific resolution still works when both a local MXQ artifact and an
explicit ONNX runtime are involved.

ONNX inference uses ONNX Runtime's `CPUExecutionProvider` by default. This avoids probing TensorRT,
CUDA, or other accelerators during normal package use. Python callers that intentionally need another
provider can pass its ordered provider list through `MBLT_Engine(onnx_providers=...)`.

```bash
mblt-model-zoo predict --source ./cat.png --model resnet50 --model-path ./resnet50.mxq
mblt-model-zoo predict --source ./cat.png --model resnet50 --model-path ./resnet50.onnx
mblt-model-zoo predict --source ./cat.png --model resnet50 --framework onnx
mblt-model-zoo predict --source ./cat.png --model resnet50 --framework onnx --mxq-path ./resnet50.mxq
mblt-model-zoo predict --source ./cat.png --model resnet50 --framework onnx --onnx-path ./resnet50.onnx
```

Prediction results are saved under `runs/vision/predict/` by default. Pass `--output` or
`--save-path` to choose a specific output file. Classification models accept `--topk`; object
detection, instance segmentation, and pose estimation models accept `--conf-thres` and
`--iou-thres`. Depth and semantic segmentation save colorized overlays without detection thresholds.
Both `predict` and `val` accept `--e2e` to enable end-to-end YOLO postprocessing; provide
`true` or `false`, or pass the bare flag to enable it. `predict --raw-output PATH` saves the
export-style model output when end-to-end postprocessing is disabled.

```bash
mblt-model-zoo predict --source ./cat.png --model resnet50 --topk 5
mblt-model-zoo predict --source ./street.jpg --model yolo11m --conf-thres 0.5 --iou-thres 0.5
```

Use `val` to validate a supported vision model on its benchmark dataset. Classification models use
ImageNet, object detection, instance segmentation, and pose estimation models use COCO. YOLO26 `*-sem` models use
Cityscapes, while `*-sem-ade20k` models keep their independent ADE20K pipeline.
Validation also supports `--framework onnx`, the shared `--model-path` override, and the
framework-specific compatibility aliases. Pass `--data-path` for an already organized validation
dataset; otherwise the CLI uses the default cache location. `--force-organize` (also `--force` or
`--reload`) rebuilds an organized dataset, while `--image-dir`, `--xml-dir`, and `--annotation-dir`
override the local archive paths or download URLs used by automatic organization.

```bash
mblt-model-zoo val --model resnet50
mblt-model-zoo val --model yolo11m --batch-size 8 --conf-thres 0.001 --iou-thres 0.7
mblt-model-zoo val --model resnet50 --model-path ./resnet50.mxq
mblt-model-zoo val --model resnet50 --model-path ./resnet50.onnx
mblt-model-zoo val --model resnet50 --framework onnx
mblt-model-zoo val --model resnet50 --framework onnx --mxq-path ./resnet50.mxq
mblt-model-zoo val --model resnet50 --framework onnx --onnx-path ./resnet50.onnx
mblt-model-zoo val --model yolo26n-sem-ade20k --framework onnx \
  --data-path ~/.mblt_model_zoo/datasets/ADEChallengeData2016
mblt-model-zoo val --model yolo26n-sem --framework onnx \
  --data-path ~/.mblt_model_zoo/datasets/cityscapes
```

Common NPU and artifact options are shared by the vision commands:

```bash
mblt-model-zoo predict \
  --source ./cat.png \
  --model resnet50 \
  --model-type DEFAULT \
  --model-path /path/to/model.mxq \
  --core-mode global8 \
  --dev-no 0
```

Use `--core-mode single`, `multi`, `global4`, or `global8` to select the NPU execution mode. For
manual placement, pass semicolon-separated values with `--target-cores`, such as `0:0;0:1`, or
`--target-clusters`, such as `0;1`. Full vision CLI details and supported model names are available
in [mblt_model_zoo/vision/README.md](mblt_model_zoo/vision/README.md).

The canonical wire form for NPU targets is fully-qualified: `--target-cores 0:0:0;0:0:1;1:0:0`
(device : cluster : core) and `--target-clusters 0:0;1:0` (device : cluster). Legacy 2-part
`c:k` cores and bare integers still work — they are migrated to the canonical form using
`--dev-no` as the device prefix. Passing `--dev-no` as a scalar keeps single-device behavior;
passing it as a list (e.g. `[0, 1]` when embedded in a config) spreads slots across those
devices without listing every core by hand.

### TPS Benchmark Helpers

The `tps` command measures token-per-second performance for Transformers-based text-generation and
image-text-to-text pipelines. It requires the `transformers` extra.

```bash
pip install "mblt-model-zoo[transformers]"
mblt-model-zoo tps measure --help
mblt-model-zoo tps sweep --help
```

`tps measure` accepts `--temperature FLOAT` (default `0.0`) to sample instead of greedy-decoding.
A value of `0.0` keeps the current greedy behavior; any value greater than zero enables
`do_sample=True` with that temperature. `tps sweep` remains greedy so its numbers stay comparable.

On VLM (`--task image-text-to-text`) pipelines whose language model uses the fake-prefill decode
path, `tps measure` decode TPS is measured with a greedy `torch.argmax` and the CLI rejects
`--temperature > 0` with a clear error. Use `--temperature 0` (default) for VLM decode TPS.

`--decode N` forces exactly `N` new tokens on non-speculative models. For EAGLE-3 speculative
decode, `N` is an upper bound: generation stops at the configured EOS and reported TPS is
computed over the tokens actually produced.

`--batch-size B` sets the aggregate batch capacity. `B` maps to the model's `max_batch_size`,
which the runtime resolves to `N` `qbruntime.Model` slots so that `N * K >= B`, where `K` is the
compiled MXQ batch axis. A non-batch MXQ (`K == 1`) with `B > 1` therefore fans out into `N = B`
Model slots that dispatch in parallel across the target device set (see `--dev-no`,
`--target-cores`, `--target-clusters`); a batched MXQ (`K > 1`) reuses hardware batching until
`N * K >= B`. Beam search paths remain `N = 1`. Legacy configs that store the older 2-part
`target_cores` / bare-int `target_clusters` are silently upgraded to the canonical form on load,
so no explicit migration step is required.

Batched MXQ execution (`K > 1`) is only supported under `--core-mode single`; other core modes
are rejected at runtime. The text-generation and VLM benchmark scripts enforce this by exiting
with `SystemExit("batch benchmark only supports --core-mode single")` when a batch run is paired
with any other explicit `--core-mode`, and the batch text-generation test suite is likewise
pinned to `single` (see [`mblt_model_zoo/hf_transformers/README.md`](mblt_model_zoo/hf_transformers/README.md)).
(In batch mode the benchmark scripts also skip their non-batch default `--target-cores 0:0`
injection, so batched runs rely on the config's default `target_cores` or an explicit
`--target-cores`.)

The text-generation benchmark script also accepts `--batch-size` and `--dev-no` on both `measure`
and `sweep`. `--batch-size N` overrides `config.max_batch_size` for the effective input batch
dim, and, on Mobilint targets only, forwards the same value as the backend `max_batch_size`
kwarg; on upstream/original Hugging Face targets it stays a measurement-only override. Passing
`--batch --original-models --batch-size N` with `N > 1` therefore admits an upstream target whose
config reports `max_batch_size == 1`. `--dev-no` on non-Mobilint targets is a silent no-op, so a
mixed Mobilint-vs-GPU sweep can share one CLI. See
[`benchmark/transformers/README.md`](benchmark/transformers/README.md) for the full example.

`tps measure --print-output` is a diagnostic flag that decodes and prints the tokens actually
generated by the last measured run in two versions (special tokens preserved, then cleaned). Use it
to visually confirm whether an EOS token terminated decoding before the `--decode` budget. The
trailing footer separates the TTFT sample from decode tokens using the same convention as
`decode_tps`: it reports `X decode tokens (+ 1 TTFT sample = Y emitted; --decode N max)` so the
count matches the measured throughput.

For thinking-capable models (e.g., Qwen3), `tps measure` exposes the mutually exclusive
`--enable-thinking` and `--disable-thinking` flags to override the `enable_thinking` argument
passed to `tokenizer.apply_chat_template`. When neither is set the tokenizer default is used, so
existing runs are unaffected. Use `--disable-thinking` to prevent a small `--decode` budget from
being consumed entirely by the `<think>` block; use `--enable-thinking` to force the block on.

Detailed TPS benchmark examples are available in
[benchmark/transformers/README.md](benchmark/transformers/README.md).

### MeloTTS Helpers

The `melo` command, also available as `melotts`, forwards arguments to the MeloTTS Click CLI. The
`melo-ui` command launches the MeloTTS Gradio WebUI. These commands require the `MeloTTS` extra.
`melo-ui` accepts `--share`, `--host`, and `--port`; use `melo --help` to see the MeloTTS text,
language, speaker, speed, device, and local-file options.

```bash
pip install "mblt-model-zoo[MeloTTS]"
mblt-model-zoo melo --help
mblt-model-zoo melotts --help
mblt-model-zoo melo-ui --help
```

### Delegated Transformers Commands

When the first argument is one of `add-fast-image-processor`, `add-new-model-like`, `chat`,
`convert`, `download`, `env`, `run`, `serve`, or `version`, `mblt-model-zoo` delegates execution to
the installed Transformers CLI. For `chat` and `serve`, the CLI installs Mobilint model registration
hooks when the delegated Transformers backend loads models through the local serve command path.

## Verbose Option

By default, model initialization stays quiet. To print the model file size and MD5 hash whenever an MXQ model loads, set the environment variable `MBLT_MODEL_ZOO_VERBOSE` to a truthy value before running your script:

```bash
export MBLT_MODEL_ZOO_VERBOSE=true  # accepted values: true/1/yes/on (case-insensitive)
python your_script.py
```

### Example Verbose Output

```bash
Model Initialized
Model Size: 216.94 MB
Model Hash: 23c262c43b4c1c453dd0326e249480a0
Device Number: 0
Core Mode: single
Target Cores: [CoreId(cluster=Cluster.Cluster0, core=Core.Core0)]
Model Variant 0
        Input Shape: [(1, 200, 96), (1, 200, 96), (2, 200, 200)]
        Output Shape: [(1, 102400, 1)]
Model Variant 1
        Input Shape: [(1, 300, 96), (1, 300, 96), (2, 300, 300)]
        Output Shape: [(1, 153600, 1)]
Model Variant 2
        Input Shape: [(1, 400, 96), (1, 400, 96), (2, 400, 400)]
        Output Shape: [(1, 204800, 1)]
Model Variant 3
        Input Shape: [(1, 500, 96), (1, 500, 96), (2, 500, 500)]
        Output Shape: [(1, 256000, 1)]
Model Variant 4
        Input Shape: [(1, 600, 96), (1, 600, 96), (2, 600, 600)]
        Output Shape: [(1, 307200, 1)]
Model Variant 5
        Input Shape: [(1, 900, 96), (1, 900, 96), (2, 900, 900)]
        Output Shape: [(1, 460800, 1)]
```

Unset or set the variable to any other value to suppress these messages.

## License

The Mobilint Model Zoo is released under BSD 3-Clause License. Please see the [LICENSE](https://github.com/mobilint/mblt-model-zoo/blob/master/LICENSE) file for more details.

Additionally, the license for each model provided in this package follows the terms specified in the source link provided with it.

## Support & Issues

If you encounter any problems with this package, please feel free to contact [us](https://discuss.mobilint.com/).
