Metadata-Version: 2.4
Name: llmready
Version: 0.1.0
Summary: A CLI tool to check if your local hardware can run a Hugging Face LLM.
Author: AI Hardware Architect
License: MIT
Project-URL: Homepage, https://github.com/shubham/llmready
Project-URL: Repository, https://github.com/shubham/llmready
Project-URL: Issues, https://github.com/shubham/llmready/issues
Keywords: llm,huggingface,hardware,memory,vram,checker,cuda,mac,apple-silicon
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: psutil>=5.9.0
Requires-Dist: huggingface-hub>=0.19.0
Requires-Dist: rich>=13.0.0

# llmready

[![PyPI version](https://img.shields.io/pypi/v/llmready.svg)](https://pypi.org/project/llmready/)
[![Python versions](https://img.shields.io/pypi/pyversions/llmready.svg)](https://pypi.org/project/llmready/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)


`llmready` is a CLI tool and Python library that analyzes your local system hardware (CPU RAM, Apple Silicon Unified Memory, and NVIDIA GPU VRAM) and accepts a Hugging Face model ID. It calculates the memory required to run that model at various quantizations (FP16, INT8, INT4) and outputs a clear "Go/No-Go" verdict.

It does **not** download the massive model weights to check memory; it smartly fetches only the metadata via the Hugging Face API!

## Features
- **Hardware Profiling:** Detects Apple Silicon Unified Memory, NVIDIA VRAM, and standard System RAM.
- **Fast Metadata Fetching:** Retrieves model parameter counts quickly without downloading weights.
- **Accurate Memory Math:** Includes KV Cache / context window overheads (20%).
- **Beautiful Output:** Uses `rich` to print colorful terminal tables with clear Go/No-Go hardware verdicts.

## Installation

You can install this package locally using `pip`:

```bash
# Clone the repository and cd into it
cd llmready

# Install the package and its dependencies in editable mode
pip install -e .
```

This will automatically install dependencies like `psutil`, `huggingface-hub`, and `rich`, and will register the `llmready` (and legacy `llm-check`) CLI command.

## Usage

Simply run the command with any Hugging Face model repository ID:

```bash
llmready meta-llama/Llama-3.2-1B
```

### Advanced Features

#### 1. Context Window Size
By default, the tool calculates the KV Cache memory footprint for an 8192 token context window using the model's exact architecture. You can change this using `--context-length` or `-c`:

```bash
llmready meta-llama/Llama-2-7b-chat-hf -c 32000
```

#### 2. Training / Fine-Tuning Memory
If you want to know if you can train a model (Full Fine-Tuning or LoRA/QLoRA), use the `--train` flag. This calculates the memory required for optimizer states and gradients rather than just inference.

#### 3. Hardware Model Recommendations
Not sure what to run? Ask the tool to scan your hardware and recommend popular models that will fit locally:

```bash
llmready --recommend
```

### Private & Gated Models

If you are trying to check a gated model (like `meta-llama/Llama-3-8b`) or a private repo, you must have access to it on Hugging Face and provide your access token.

You can set the standard Hugging Face token environment variable before running the tool:

```bash
export HF_TOKEN="hf_your_access_token_here"
llmready meta-llama/Llama-3-8b
```

## How It Works
1. **Hardware:** Uses `psutil` and `nvidia-smi` (if applicable) to see what compute you have available.
2. **Parameters:** Uses `huggingface_hub` to read `safetensors.total` metadata (or fallback size approximation).
3. **Calculation:** Memory = Parameters * Bytes per Parameter (FP16=2, INT8=1, INT4=0.6). It also adds a 20% overhead to account for KV caching and memory footprint.
4. **Verdict:** Compares the required memory to your available VRAM and RAM to issue a 🟢 Pass, 🟡 Warning, or 🔴 Fail.
