Metadata-Version: 2.4
Name: pplx-cli
Version: 1.1.1
Summary: A command-line notes and RAG interface for Perplexity AI API
License-File: LICENSE
Keywords: perplexity,ai,cli,llm
Author: xerexcoded
Author-email: arnavbanerjee61@gmail.com
Requires-Python: >=3.12,<4.0
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: click (>=8.0.0,<8.2.0)
Requires-Dist: mcp (>=1.0.0,<2.0.0)
Requires-Dist: numpy (>=1.26.0,<2.0.0)
Requires-Dist: openpyxl (>=3.1.5,<4.0.0)
Requires-Dist: pandas (>=2.1.0,<3.0.0)
Requires-Dist: pypdf (>=5.0.0,<7.0.0)
Requires-Dist: python-dotenv (>=1.0.1,<2.0.0)
Requires-Dist: requests (>=2.31.0,<3.0.0)
Requires-Dist: sentence-transformers (>=3.0.0,<4.0.0)
Requires-Dist: sqlite-vec (>=0.1.6,<0.2.0)
Requires-Dist: toml (>=0.10.2,<0.11.0)
Requires-Dist: torch (>=2.0.0,<3.0.0)
Requires-Dist: trafilatura (>=2.0.0,<3.0.0)
Requires-Dist: transformers (>=4.40.0,<5.0.0)
Requires-Dist: typer (>=0.15.0,<0.16.0)
Requires-Dist: watchdog (>=5.0.0,<7.0.0)
Project-URL: Documentation, https://github.com/xerexcoded/pplx-cli#readme
Project-URL: Homepage, https://github.com/xerexcoded/pplx-cli
Project-URL: Repository, https://github.com/xerexcoded/pplx-cli
Description-Content-Type: text/markdown

# Perplexity CLI

A command-line interface for Perplexity, NVIDIA NIM, OpenRouter, and local Ollama models with local notes, evidence-first RAG, and a Markdown-compatible LLM Wiki. Search notes, chat history, and curated research sources without giving up source provenance.

## 🚀 Key Features

- 🔍 **Evidence-First RAG**: Semantic and FTS retrieval across local content using BGE embeddings by default
- 🧠 **Hybrid Search**: Combines vector similarity and keyword search with Reciprocal Rank Fusion
- 📚 **Unified Knowledge Base**: Search notes and chat history together in a single, powerful interface
- 🔄 **Seamless Migration**: Automatically migrate your existing data to the new RAG system
- 🤖 **Multi-Provider AI Chat**: Use Perplexity Sonar, NVIDIA NIM, or OpenRouter models with one CLI
- 📝 **Advanced Note Management**: Local storage with AI-powered semantic search
- 💬 **Complete Chat History**: Track, analyze, and export all conversations
- 📊 **Rich Analytics**: Detailed statistics and insights about your usage
- 🕸️ **Knowledge Graph**: Visualize Obsidian-style markdown vaults as interactive D3.js force graphs
- 📚 **Native LLM Wiki**: Compile Markdown, text PDFs, and captured web pages into cited, linked Markdown pages
- 🖥️ **Cross-Platform**: Works on macOS, Linux, WSL2, and Windows

## Installation

### From PyPI
```bash
pip install pplx-cli
```

### From Source (Development)
```bash
git clone https://github.com/xerexcoded/pplx-cli.git
cd pplx-cli

# Using Poetry (recommended)
poetry install
poetry run perplexity --help

# Using pip
pip install -e .
```

## Quick Start

1. **Choose and set up your default AI provider:**
```bash
perplexity setup
```

The setup prompt lets you choose `perplexity`, `nvidia`, `openrouter`, or `ollama`. Hosted providers save an API key; Ollama setup selects an installed local model and asks whether to change the default.

2. **Migrate your existing data to RAG (recommended):**
```bash
perplexity rag-migrate
```

3. **Start searching through all your content:**
```bash
perplexity rag "machine learning concepts"
```

## Local research with Ollama

Start Ollama explicitly with cloud features disabled, then configure the CLI:

```sh
OLLAMA_NO_CLOUD=1 ollama serve
# In another terminal, with an already installed chat model:
perplexity setup --provider ollama
```

Setup checks the daemon, lists installed local chat models, and saves your selected endpoint, model, context and timeout. Choose whether to make Ollama the default; existing provider defaults otherwise stay unchanged. A compatible daemon must report its cloud-disabled state and model context metadata. Older or unverifiable versions fail with an actionable error. Ollama 0.35.1 was used for local verification.

Prepare retrieval models as an explicit download step. The `small` BGE model is useful for a first local workspace:

```sh
perplexity rag-config --embedding-provider local --model small
perplexity prepare-models
# Optional, explicitly download the reranker too:
perplexity prepare-models --include-reranker
```

With Ollama selected as the default, local indexing and note creation also require cached models. Queries never download missing models, start services, switch embedding models, or fall back to a hosted provider. An unavailable optional reranker visibly falls back to hybrid retrieval.

```sh
perplexity wiki init --dir ~/research
perplexity wiki sync --dir ~/research
perplexity wiki query "Why did we choose SQLite?" --dir ~/research --provider ollama
perplexity ask-notes "What did we decide?" --provider ollama
perplexity wiki compile --dir ~/research --provider ollama
perplexity ask "Explain SQLite WAL briefly" --provider ollama --no-save-history
```

Wiki inputs include `.md`, `.markdown`, `.txt`, text PDFs, and previously captured web sources. Notes and wiki collections remain separate. Source-backed local answers show inline references and supplied excerpts; omitted evidence is disclosed. Invalid references get one correction attempt, then an error with evidence. Reference validation does not prove the semantic correctness of every claim.

Ollama mode rejects live web research, remote embeddings, remote model routing, and non-loopback endpoints. Live URL capture is rejected while Ollama is the default; previously captured snapshots remain usable. Context and output are bounded; an incomplete response raises an explicit error. The application verifies daemon-reported policy, with OS-level network isolation outside its scope.

See the [development setup](docs/development.md), [agreed design](docs/design/ollama-local-research.md), and explicit real-model check in `scripts/verify-local-research.py`.

## 🔍 Fast RAG System - Your Personal Knowledge Assistant

The RAG system transforms your CLI into a **powerful personal knowledge assistant** that can instantly search through all your notes and conversations. This is the **most important feature** of the CLI - it makes all your accumulated knowledge instantly searchable and actionable.

### Why RAG is Useful

- **Semantic Understanding**: Find content by meaning, not just keywords
- **Unified Search**: One command searches both notes and chat history
- **Inspectable Evidence**: Results retain source and chunk metadata
- **Always Available**: Works completely offline once set up

Vector retrieval uses `sqlite-vec` when the local Python/SQLite build permits extensions. If it cannot load, the CLI makes an exact in-process similarity search instead and reports that fallback in diagnostics. Benchmark latency depends on corpus size, model, and hardware; measure it against your own vault before setting an SLO.

### 🎯 Essential RAG Commands

#### Search All Your Content
```bash
# Smart hybrid search (combines semantic + keyword)
perplexity rag "machine learning concepts"

# Find similar concepts semantically
perplexity rag "neural networks" --mode vector

# Search for exact terms
perplexity rag "specific function name" --mode keyword

# Search only your notes
perplexity rag "project ideas" --source notes

# Search only chat history
perplexity rag "debugging help" --source chats

# Get detailed results with metadata
perplexity rag "python programming" --verbose --limit 10
```

#### Migration (Essential First Step)
```bash
# See what will be migrated and estimated time
perplexity rag-migrate --estimate

# Migrate all your existing data
perplexity rag-migrate

# Migrate only specific content
perplexity rag-migrate --source notes     # Notes only
perplexity rag-migrate --source chats     # Chat history only

# Start fresh (clears existing RAG data)
perplexity rag-migrate --clear
```

#### System Management
```bash
# View your knowledge base statistics
perplexity rag-stats

# Re-index content (after model upgrades)
perplexity rag-index

# Configure embedding models and performance
perplexity rag-config --show
perplexity rag-config --model large --device cuda
```

### 🧠 Advanced RAG Features

#### Search Modes
- **Hybrid** (default): Best of both worlds - semantic understanding + exact matches
- **Vector**: Pure semantic search - finds conceptually similar content
- **Keyword**: Traditional search - finds exact term matches

#### Content Filtering
- **All**: Search across notes and chat history together
- **Notes**: Focus on your documented knowledge
- **Chats**: Find past conversations and solutions

#### Performance Tuning
- **Similarity Threshold**: Filter low-quality matches
- **Batch Processing**: Efficient handling of large datasets
- **Model Selection**: Choose between speed (small) and quality (large)

### 📊 Real-World RAG Use Cases

**For Developers:**
```bash
perplexity rag "error handling patterns"
perplexity rag "API integration examples" --source chats
perplexity rag "code review feedback" --verbose
```

**For Researchers:**
```bash
perplexity rag "methodology discussion" --mode vector
perplexity rag "literature review notes" --source notes
perplexity rag "experiment results" --threshold 0.7
```

**For Knowledge Workers:**
```bash
perplexity rag "meeting outcomes" --source chats
perplexity rag "project requirements" --mode hybrid
perplexity rag "client feedback" --limit 15
```

### Technical design

The RAG system uses local embeddings, SQLite FTS5, and an optional native vector extension:

- **BGE Embeddings**: State-of-the-art BAAI General Embedding models
  - `small`: 33M params, 62.17 MTEB score
  - `base`: 109M params, 63.55 MTEB score (default)
  - `large`: 335M params, 64.23 MTEB score
- **sqlite-vec**: Native KNN when the extension can be loaded; otherwise an explicit exact fallback
- **Hybrid Search**: Reciprocal Rank Fusion combines vector and FTS5 candidates
- **Inspectable chunking**: Chunk and source identifiers are retained with every result
- **Configurable embeddings**: local BGE is the default; Perplexity embeddings are opt-in with `rag-config --embedding-provider perplexity`

### 💡 Migration Benefits

**Why migrate to RAG?**
- **Broader Recall**: Hybrid retrieval finds semantic and exact-term matches
- **Safer Results**: Source IDs and locators make retrieval inspectable
- **Unified Interface**: No more separate commands for notes vs chat history
- **Future-Proof**: Built on modern RAG architecture used by leading AI companies
- **Offline First**: All processing happens locally for privacy and speed

**Migration is Safe:**
- Non-destructive: Original data remains untouched
- Progress tracking with ETA estimates
- Detailed error reporting and retry logic
- Can re-run anytime to sync new content

## 📚 Native LLM Wiki — persistent, cited research

`perplexity wiki` turns a directory of sources into an open Markdown wiki. Your local source files are never moved or edited. The CLI creates only:

```text
research/
├── papers/                  # Your Markdown and text PDFs stay in place
├── wiki/                    # Generated, editable Markdown pages
│   ├── overview.md
│   └── sources/
└── .pplx/                   # Rebuildable SQLite index and captured web snapshots
```

Start a workspace, index local files and a URL, then compile pages explicitly:

```bash
perplexity wiki init --dir ~/research
perplexity wiki sync --dir ~/research
perplexity wiki ingest https://example.com/article --dir ~/research --tag reading
perplexity wiki compile --dir ~/research
perplexity wiki lint --dir ~/research
```

`wiki sync` is idempotent: it adds new files, reindexes changed files, and deactivates deleted files. `wiki watch --dir ~/research` performs only that synchronization in the foreground; it never silently changes generated pages. `wiki compile` replaces only marked generated sections, preserving content under **Manual notes**.

### Search and ask from evidence

```bash
# Inspect the authoritative chunks and their file/page/URL locators
perplexity wiki search "retrieval citations" --dir ~/research --source-type pdf

# Compose an answer from local evidence; [L…] references identify local sources
perplexity wiki query "What do my sources say about RAG evaluation?" --dir ~/research

# Keep current research visibly separate from local evidence
perplexity wiki query "What changed this month?" --dir ~/research --web

# Equivalent shortcut that also keeps normal chat-history behavior
perplexity ask --wiki ~/research "What do I know about retrieval?"
```

The default answer path retrieves raw source chunks, not generated wiki prose, and prompts the writer to report insufficient evidence rather than guess. `--web` needs a Perplexity API key even when the local-answer writer is NVIDIA NIM or OpenRouter; it prints Perplexity URLs as separate `[W…]` references.

### Agent access and retrieval evaluation

```bash
# Read-only MCP tools: wiki_search, wiki_read_page, wiki_read_source, wiki_status
perplexity wiki mcp --dir ~/research

# JSONL entries: {"query":"...", "relevant_source_ids":[1, 4]}
perplexity wiki eval --dir ~/research --dataset retrieval-eval.jsonl
# Nested RAG form (a top-level `rag-eval` alias also remains available)
perplexity rag eval --dir ~/research --dataset retrieval-eval.jsonl
```

The evaluation report includes Recall@k, MRR, nDCG, and queries with no relevant source in the retrieved set.

## 🕸️ Knowledge Graph — Visualize Your Markdown Vault

Turn any folder of interlinked markdown files into an interactive, Obsidian-style knowledge graph in your browser. Perfect for visualizing your personal wiki, Zettelkasten, or project documentation.

![Knowledge Graph Screenshot](screenshot.png)

### How It Works

Parses `[[wikilinks]]` (with aliases via `|`), `[text](url)` markdown links, relative paths, and external URLs. Renders as an interactive D3.js force-directed graph with drag, zoom, tooltips, and node sizing by connection count.

### Usage

```bash
# Visualize a markdown vault
perplexity knowledge-graph --dir ~/my-vault

# Or your Obsidian vault
perplexity knowledge-graph --dir ~/Documents/ObsidianVault

# Save HTML for offline viewing
perplexity knowledge-graph --dir ~/my-vault --output ~/Desktop/graph.html

# With a custom title
perplexity knowledge-graph --dir ~/my-vault --title "My Knowledge Base"
```

### Demo Script

Seed a sample vault and launch the graph:

```bash
./scripts/demo-knowledge-graph.sh
```

### Graph Features

- **Force-directed layout**: Nodes repel, links attract — clusters form naturally
- **Color-coded nodes**: Blue for internal files, red for external/absent references
- **Weighted nodes**: Larger circles for highly-connected files
- **Drag & zoom**: Pan, zoom, and rearrange nodes interactively
- **Tooltips**: Hover any node to see its file path
- **Wikilink parsing**: Supports `[[link]]`, `[[link|Alias]]`, `[[link#heading]]`
- **Markdown link parsing**: Handles `[text](url)`, relative paths, bare URLs

## Traditional Commands

### Direct AI Chat

Ask your configured provider directly:
```bash
perplexity ask "What is the capital of France?"

# Configure a provider non-interactively
perplexity setup --provider nvidia
perplexity setup --provider openrouter

# Override the saved provider for one request
perplexity ask --provider nvidia "Explain retrieval augmented generation"
perplexity ask --provider openrouter "Summarize the CAP theorem"

# Perplexity aliases
perplexity ask "Complex reasoning question" --model large
perplexity ask "Quick search query" --model small

# NVIDIA NIM and OpenRouter accept provider-native model IDs
perplexity ask --provider nvidia --model meta/llama-3.3-70b-instruct "Write a haiku"
perplexity ask --provider openrouter --model openrouter/free "Explain recursion"

# With conversation topics
perplexity ask "Explain quantum computing" --topic physics
```

### Provider configuration

Environment variables override keys saved by `perplexity setup`, which is useful for CI and shell-specific credentials:

```bash
export PERPLEXITY_API_KEY="..."
export NVIDIA_API_KEY="..."
export OPENROUTER_API_KEY="..."
```

| Provider | Default model | `--model` behavior |
|----------|---------------|--------------------|
| Perplexity | `sonar` | Use `small`, `large`, or `huge` aliases |
| NVIDIA NIM | `meta/llama-3.3-70b-instruct` | Pass any supported NVIDIA NIM model ID |
| OpenRouter | `openrouter/free` | Pass any supported OpenRouter model slug |

Run `perplexity list-models --provider <provider>` to see Perplexity aliases or the native-model usage for NVIDIA NIM and OpenRouter.

**Perplexity aliases:**
- `small`: Lightweight, fast responses (maps to sonar)
- `large`: Deep reasoning and analysis (maps to sonar-reasoning)
- `huge`: Comprehensive research reports (maps to sonar-deep-research)

### Chat History Management

View chat history:
```bash
perplexity history
```

Show detailed chat statistics:
```bash
perplexity chat-stats
```

View a specific conversation:
```bash
perplexity show-chat <conversation-id>
```

Export a conversation:
```bash
# Export to markdown
perplexity export-chat <conversation-id> --format markdown --output conversation.md

# Export to JSON
perplexity export-chat <conversation-id> --format json --output conversation.json
```

Export all conversations:
```bash
# Export to Excel
perplexity export-all --format excel --output chat_history.xlsx

# Export to JSON
perplexity export-all --format json --output chat_history.json
```

### Legacy Note Management

> **💡 Pro Tip**: Use `perplexity rag` for much faster and better search results!

Create and manage individual notes:
```bash
# Create a note
perplexity note --title "My Note" --content "Note content" --tag research

# List all notes
perplexity list-notes

# View specific note
perplexity view-note <note-id>

# Search notes (legacy - use RAG instead!)
perplexity ask-notes "What did I write about machine learning?"

# Use another provider for the generated answer
perplexity ask-notes --provider openrouter --model openrouter/free "What did I write about machine learning?"
```

## 🎯 Command Reference

### RAG Commands (Recommended)
| Command | Purpose | Example |
|---------|---------|---------|
| `rag` | Search all content | `perplexity rag "python functions"` |
| `rag-migrate` | Migrate existing data | `perplexity rag-migrate` |
| `rag-stats` | Show database stats | `perplexity rag-stats` |
| `rag-index` | Re-index content | `perplexity rag-index` |
| `rag-config` | Configure RAG | `perplexity rag-config --show` |

### Knowledge Graph Commands
| Command | Purpose | Example |
|---------|---------|---------|
| `knowledge-graph` | Visualize markdown vault | `perplexity knowledge-graph --dir ~/vault` |

### Traditional Commands
| Command | Purpose | Example |
|---------|---------|---------|
| `ask` | Chat with AI | `perplexity ask "question"` |
| `history` | View chat history | `perplexity history` |
| `export-chat` | Export conversation | `perplexity export-chat 123 --output file.md` |
| `note` | Create note | `perplexity note --title "My Note" --content "..."` |
| `list-notes` | List notes | `perplexity list-notes` |

### CLI Options

Most commands support long-form options. Boolean flags (like `--verbose`) also support short forms (e.g., `-v`):

```bash
# Boolean flags support short forms
perplexity ask "question" --verbose    # or -v
perplexity rag "query" --verbose       # or -v

# Non-boolean options use long form
perplexity ask "question" --model large --topic physics
perplexity rag "query" --mode vector --source notes --limit 10
perplexity export-chat 123 --format json --output file.json
```

## 🚀 Performance & Capabilities

### Speed Benchmarks
- **Search 10,000 documents**: < 100ms
- **Index 1,000 documents**: < 5 seconds
- **Memory usage**: < 512MB for typical workloads
- **Embedding generation**: 200+ texts/second

### Storage Efficiency
- **Unified database**: Single file for all content
- **Smart compression**: Optimized vector storage
- **Incremental updates**: Only process new/changed content
- **Portable**: Entire knowledge base in one SQLite file

### Advanced Features
- **Multi-modal search**: Combine vector similarity + keyword matching
- **Content-aware chunking**: Intelligent text segmentation
- **Relevance tuning**: Configurable similarity thresholds
- **Batch processing**: Efficient bulk operations
- **Progress tracking**: Real-time migration status

## 🎯 Getting Started Checklist

**New Users:**
1. ✅ Install: `pip install pplx-cli`
2. ✅ Setup: `perplexity setup`
3. ✅ Try it: `perplexity ask "Hello world"`
4. 🚀 **Migrate to RAG**: `perplexity rag-migrate`
5. 🔍 **Start searching**: `perplexity rag "your first search"`

**Existing Users:**
1. 🚀 **Essential**: `perplexity rag-migrate` (unlock fast search!)
2. 📊 Check stats: `perplexity rag-stats`
3. 🔍 Try search: `perplexity rag "something you remember discussing"`
4. 🎉 Enjoy 10x faster, better search results!

## Development

### Setup Development Environment
```bash
git clone https://github.com/xerexcoded/pplx-cli.git
cd pplx-cli

# Using Poetry (recommended)
poetry install
poetry run pytest  # Run tests
poetry run perplexity --help  # Test CLI

# Or using pip
pip install -e .
```

### Testing RAG Features
```bash
# Test migration with sample data
poetry run python -m pplx_cli.migrations.migrate_to_rag --dry-run

# Test RAG search
poetry run perplexity rag "test query" --explain

# Run full test suite
poetry run pytest -v

# Measure project-wide test coverage
poetry run coverage run -m pytest
poetry run coverage report -m
```

### Architecture
The project uses modern Python practices:
- **Poetry**: Dependency management
- **Typer**: CLI framework
- **SQLite + sqlite-vec**: Vector database
- **BGE Embeddings**: State-of-the-art semantic search
- **Hybrid Search**: Best of vector + keyword search

## 💪 Why Choose Perplexity CLI?

**Compared to other solutions:**
- ✅ **Completely local**: Your data never leaves your machine
- ✅ **Ultra-fast search**: 10x faster than text-based search
- ✅ **Production-ready**: Built with enterprise-grade architecture
- ✅ **Easy migration**: Automatically import existing data
- ✅ **Active development**: Regular updates with latest AI technology
- ✅ **Open source**: Full transparency and customization

**Perfect for:**
- 👨‍💻 **Developers**: Search code discussions, debug sessions, learning notes
- 🔬 **Researchers**: Find methodology notes, literature reviews, experiment logs
- 💼 **Knowledge workers**: Locate project info, meeting notes, decisions
- 📚 **Students**: Search study materials, lecture notes, research papers
- 🧠 **Anyone**: Who wants their accumulated knowledge to be instantly searchable

## 🚀 Ready to Transform Your Workflow?

**Don't let your valuable knowledge get lost in countless notes and conversations.**

The RAG system makes **every piece of information you've ever saved instantly discoverable**. Whether it's a solution you found months ago, a concept you learned last week, or a conversation from yesterday - find it in seconds, not minutes.

**Start your knowledge transformation today:**
```bash
pip install pplx-cli
perplexity setup
perplexity rag-migrate  # 🔥 This changes everything
perplexity rag "anything you want to find"
```

## Contributing

We welcome contributions! The RAG system opens up many possibilities for enhancements:
- New embedding models
- Advanced search algorithms
- Export integrations
- Performance optimizations

Please submit Pull Requests and join our mission to make personal knowledge instantly accessible.

## License

MIT License - see LICENSE file for details.

---

**⭐ Star this repo if Perplexity CLI's RAG system helps you find information faster!**

