Metadata-Version: 2.4
Name: llm-velocity
Version: 2.0.0
Summary: llm_velocity is a lightweight framework designed to build high-performance, cost-effective AI agents. It drastically reduces LLM call expenses and accelerates response times by intelligently optimizing how the language model is utilized.
Author-email: Aditya Raj Singh Chouhan <adityarajschouhan@gmail.com>
License: MIT License
        
        Copyright (c) 2026 Aditya Raj Singh Chouhan
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
Project-URL: Homepage, https://github.com/ChouhanAdityaRaj/llm_velocity
Project-URL: Bug Tracker, https://github.com/ChouhanAdityaRaj/llm_velocity/issues
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Operating System :: OS Independent
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: faiss-cpu==1.13.2
Requires-Dist: numpy==2.4.4
Requires-Dist: platformdirs==4.11.2
Requires-Dist: rapidfuzz==3.14.5
Requires-Dist: requests==2.33.1
Requires-Dist: sentence-transformers==5.4.1
Dynamic: license-file

<div align="center">
<h2>माँ आशापुरा</h2>
<img src="static/llm_velocity.png" alt="llm_velocity logo" />

#  LLM Velocity

### Lightweight Framework for Building High-Performance, Cost-Effective AI Agents

<br/>

[![PyPI](https://img.shields.io/pypi/v/llm_velocity?style=for-the-badge\&logo=pypi\&logoColor=white)](https://pypi.org/project/llm-velocity/)
[![Python](https://img.shields.io/pypi/pyversions/llm_velocity?style=for-the-badge\&logo=python\&logoColor=white)](https://www.python.org/)
[![License](https://img.shields.io/badge/License-MIT-yellow?style=for-the-badge)](LICENSE)

</div>

<br/>

---

# ⚡ Introduction

`llm_velocity` is a lightweight framework designed to build high-performance, cost-effective AI agents. It drastically reduces LLM call expenses and accelerates response times by intelligently optimizing how the language model is utilized.

<br/>

---



# ⏱️ Token Usage Benchmark

A simple flight-booking Prompt-based agent benchmark shows how `llm_velocity` can significantly reduce token usage in tool-calling workflows.

| Agent                         | Approx. Token Usage |
| ----------------------------- | ------------------: |
| Standard Flight Booking Prompt-based Agent |       ~1,500 tokens |
| With `llm_velocity`           |         ~450 tokens |
| **Reduction**                 |            **~70%** |

In this benchmark, the same flight-booking workflow used approximately **1,500 tokens** with a conventional agent approach, while the `llm_velocity` implementation used approximately **450 tokens**.

<br/>


---

# ✨ Features


### 🛠️ Native Tool Calling

> **Reduce LLM token consumption by up to 70% in tool-calling workflows.**

The framework automatically handles multi-turn conversations and tool invocation, minimizing LLM calls and unnecessary token consumption while enabling faster, more efficient, and deterministic agent execution.

### 🚀 Smart Response Recall

> **Never compute the same informational or knowledge-seeking prompts twice.**

`llm_velocity` intelligently memorizes your previous requests, returning instant responses instead of making redundant llm calls.

<br/>

---

# 🤔 Why Use LLM Velocity?
- **LLM-Free Intent Detection** — Identify the appropriate tool without requiring an LLM call for every request.
- **Lower Token Consumption** — Reduce unnecessary LLM usage and overall token consumption.
- **Customizable Error Messages** — Let developers define their own user-friendly error messages for validation and tool-related errors.
- **Avoid Repeated LLM Calls** — Reuse results for repeated informational and knowledge-seeking prompts.
- **Easy to Learn** — A simple API that can be understood within **10–20 minutes**.
- **Quick and Easy Setup** — Minimal configuration required to get started.

<br/>


---

# 📦 Installation

Install `llm_velocity` directly from PyPI:

```bash
pip install llm_velocity
```

---

# 🚀 Quickstart

The following example demonstrates a complete tool-calling workflow using LLM Velocity.

### Workflow

| Step  | Description                                         |
| ----- | --------------------------------------------------- |
| **1** | Define validation callbacks.                        |
| **2** | Define the available tools and their parameters.    |
| **3** | Initialize `VelocityEngine`.                        |
| **4** | Initialize the internal setup.                      |
| **5** | Provide the previous conversation turns.            |
| **6** | Execute the conversation with tool calling enabled. |
| **7** | Receive the resolved tool parameters.               |

### Complete Example

```python
from llm_velocity import VelocityEngine
from llm_velocity.models import models
from typing import Any

# 1. Define Local Validation Callbacks

def validate_from_location(value: Any) -> str | None:
    if not value or not str(value).strip():
        return "Please provide where you are flying from."

    available_locations = ["USA", "India", "Japan"]

    if value not in available_locations:
        return f"Please provide a valid departure location. We only provide flights from: {', '.join(available_locations)}."

    return None


def validate_date(value: Any) -> str | None:
    if not value or not str(value).strip():
        return "Please provide a booking date."

    return None


# 2. Define Tools with Validation

tools_definition: dict[str, models.ToolInfo] = {
    "search_flights": models.ToolInfo(
        name="search_flights",
        description="Search for available flights.",
        parameters=[
            models.ToolParameters(
                name="from_location",
                description="The departure city or airport.",
                validationCallback=validate_from_location,
            ),
            models.ToolParameters(
                name="date",
                description="The date of travel.",
                validationCallback=validate_date,
            ),
        ],
    ),
}


# 3. Initialize the LookUp Store

velocity_engine = VelocityEngine(
    llm_provider=models.LLMProvider.GEMINI.value,
    llm_model="gemini-3.5-flash",
    tools_definition=tools_definition,
    api_key="YOUR_API_KEY"
)


try:
    velocity_engine.init_setup()
except Exception as e:
    import logging
    logging.error(f"Failed to initialize: {e}")


# 4. Execute Multi-Turn Conversation

user_context = [
    models.UserChatReq(
        user_message="Find flights from the UK",
        llm_response="Please provide a valid departure location. We only provide flights from: USA, India, Japan.\nPlease provide a booking date."
    ),
    models.UserChatReq(
        user_message="I am flying from Japan",
        llm_response="Please provide a booking date."
    ),
    models.UserChatReq(
        user_message="December 2nd 2026"
    ),
]


try:
    res = velocity_engine.call(user_context, True)
    print(f"Response: {res}") 
    #OUTPUT: Response: StoreResp(llm_response='', tool_name='search_flights', arguments={'from_location': 'Japan', 'date': 'December 2nd 2026'}, error=None, metadata=None)
except Exception as e:
    print(f"Execution Error: {e}")
```

---

# 🎓 Detailed Usage Guide

> Follow the steps below to configure `llm_velocity` for your own agent workflow.

---

# 🧩 Step 1: Define Validation Callbacks

Validation callbacks allow you to validate tool parameters locally before the tool is executed.

A validation callback must follow this structure:

```python
def callback_name(value: Any) -> str | None:
    if VALIDATION_CONDITION:
        return "MESSAGE_TO_SEND_TO_USER"

    return None
```

### 💡 Example

```python
from typing import Any


def validate_from_location(value: Any) -> str | None:
    if not value or not str(value).strip():
        return "Please provide your departure location."

    available_locations = ["USA", "India", "Japan"]

    if value not in available_locations:
        return (
            "Please provide a valid departure location. "
            f"Available locations: {', '.join(available_locations)}."
        )

    return None
```

### 🔍 Validation behavior

A validation callback should:

* ✅ Return `None` when the value is valid.
* ❌ Return a `str` containing the user-facing validation message when the value is invalid.
* 🧩 Handle missing values when the parameter is required.
* ⚙️ Perform application-specific validation locally.

For example:

```python
def validate_amount(value: Any) -> str | None:
    if value is None:
        return "Please provide an amount."

    if value <= 0:
        return "Amount must be greater than 0."

    if value > 20000:
        return "Amount cannot exceed 20,000."

    return None
```

---

# 🛠️ Step 2: Define Your Tools

Tools are defined using `models.ToolInfo`.

Each tool contains:

* 🔹 A tool name
* 🔹 A tool description
* 🔹 A list of parameters
* 🔹 A validation callback for each parameter

### 📐 Basic Structure

```python
from llm_velocity.models import models

tools_definition: dict[str, models.ToolInfo] = {
    "TOOL_NAME": models.ToolInfo(
        name="TOOL_NAME",
        description="DESCRIPTION_OF_THE_TOOL",
        parameters=[
            models.ToolParameters(
                name="PARAMETER_NAME_1",
                description="DESCRIPTION_OF_PARAMETER",
                validationCallback=VALIDATION_CALLBACK,
            ),
            models.ToolParameters(
                name="PARAMETER_NAME_2",
                description="DESCRIPTION_OF_PARAMETER",
                validationCallback=VALIDATION_CALLBACK,
            ),
        ],
    ),
}
```

### 💡 Example

```python
tools_definition: dict[str, models.ToolInfo] = {
    "search_flights": models.ToolInfo(
        name="search_flights",
        description="Search for available flights.",
        parameters=[
            models.ToolParameters(
                name="from_location",
                description="The departure city or airport.",
                validationCallback=validate_from_location,
            ),
            models.ToolParameters(
                name="date",
                description="The date of travel.",
                validationCallback=validate_date,
            ),
        ],
    ),
}
```

> 💡 The tool definition tells LLM Velocity what information is required before a tool can be executed.

---

# ⚙️ Step 3: Initialize `VelocityEngine`

Import and initialize `VelocityEngine`:

```python
from llm_velocity import VelocityEngine

velocity_engine = VelocityEngine(
    llm_provider="YOUR_LLM_PROVIDER",
    llm_model="YOUR_LLM_MODEL",
    endpoint_url="OLLAMA_LLM_URL",
    tools_definition=tools_definition,
    api_key="YOUR_API_KEY",
    llm_temperature=0.7,
)
```

## 🔧 Configuration Parameters

### `llm_provider`

Specifies which LLM provider should be used.

Currently supported providers:

```text
openai
gemini
anthropic
ollama
```

Example:

```python
llm_provider="gemini"
```

---

### `llm_model`

Specifies the model that should process the requests.

Example:

```python
llm_model="gemini-1.5-flash"
```

The model name must be supported by the selected provider.

---

### `endpoint_url`

Specifies the LLM endpoint URL.

This is primarily required when using Ollama.

Example:

```python
endpoint_url="http://localhost:11434"
```

For other providers, this parameter can generally be omitted unless a custom endpoint is supported.

---

### `tools_definition`

The tool definitions used by the engine.

Example:

```python
tools_definition=tools_definition
```

This tells LLM Velocity which tools are available and which parameters are required.

---

### `api_key`

API key used to authenticate with the selected LLM provider.

Example:

```python
api_key="YOUR_API_KEY"
```

For Ollama, an API key is generally not required.

---

### `llm_temperature`

Controls the LLM's response temperature.

Example:

```python
llm_temperature=0.5
```

The default value is:

```text
0.7
```

For more deterministic tool-parameter extraction, a lower temperature can be useful.

---

# 🔌 Step 4: Initialize the Engine

After creating the `VelocityEngine`, initialize its setup:

```python
try:
    velocity_engine.init_setup()
except Exception as e:
    import logging
    logging.error(f"Failed to initialize: {e}")
```

`init_setup()` prepares the engine before processing conversations.

It should be called before using:

```python
velocity_engine.call(...)
```

---

# 💬 Step 5: Build the Conversation Context

LLM Velocity supports multi-turn conversations.

Conversation turns are represented using `models.UserChatReq`.

```python
user_context = [
    models.UserChatReq(
        user_message="Find flights from Japan",
        llm_response="Please provide a booking date.",
    ),
    models.UserChatReq(
        user_message="December 2nd 2026",
        llm_response="{'from_location': 'Japan', 'date': '2026-12-02'}",
    ),
]
```

Each conversation entry contains:

### `user_message`

The message sent by the user.

```python
user_message="Find flights from Japan"
```

### `llm_response`

The response generated for that user turn.

```python
llm_response="Please provide a booking date."
```

> 💡 The conversation context allows LLM Velocity to use information provided across multiple turns when resolving tool parameters.

---

# ▶️ Step 6: Execute the Conversation

Call the engine using:

```python
response = velocity_engine.call(
    user_context,
    True,
)
```

The second argument determines whether the request should be processed as a tool-calling workflow.

```python
velocity_engine.call(user_context, tool_calling)
```

### 🟢 Tool calling enabled

```python
velocity_engine.call(
    user_context,
    True,
)
```

When `True`, LLM Velocity processes the conversation for tool parameter resolution and tool calling.

### ⚪ Tool calling disabled

```python
velocity_engine.call(
    user_context,
    False,
)
```

When `False`, the request is processed without tool-calling behavior.

The default value is:

```python
False
```

---

# 📄 License

This project is licensed under the [MIT License](LICENSE) - see the LICENSE file for details.

Copyright (c) 2026 [Aditya Raj Singh Chouhan]

<br/>

<div align="center">

**⚡ LLM Velocity — Faster, Efficient & Deterministic LLM Workflows**

</div>
