Metadata-Version: 2.4
Name: parasort
Version: 5.1.0
Summary: URL Parameter Categorization and Extraction Tool for Penetration Testing
Author-email: Ev3rPalestine <Ev3rPalestine@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/Ev3rPalestine/Parasort
Project-URL: Repository, https://github.com/Ev3rPalestine/Parasort
Project-URL: Issues, https://github.com/Ev3rPalestine/Parasort/issues
Keywords: pentesting,bug-bounty,security,url,parameters,sqli,xss
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Security
Classifier: Topic :: Internet :: WWW/HTTP
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: colorama>=0.4.6
Provides-Extra: test
Requires-Dist: pytest>=7; extra == "test"
Dynamic: license-file

# Parasort - URL Parameter Categorization & Extraction Tool

![PARASORT Logo](https://img.shields.io/badge/PARASORT-URL%20Parameter%20Categorizer-blue)
![Python](https://img.shields.io/badge/Python-3.11%2B-green)
![License](https://img.shields.io/badge/License-MIT-yellow)
![Author](https://img.shields.io/badge/Author-Ev3rPalestine-orange)

```text
██████╗  █████╗ ██████╗  █████╗ ███████╗ ██████╗ ██████╗ ████████╗
██╔══██╗██╔══██╗██╔══██╗██╔══██╗██╔════╝██╔═══██╗██╔══██╗╚══██╔══╝
██████╔╝███████║██████╔╝███████║███████╗██║   ██║██████╔╝   ██║
██╔═══╝ ██╔══██║██╔══██╗██╔══██║╚════██║██║   ██║██╔══██╗   ██║
██║     ██║  ██║██║  ██║██║  ██║███████║╚██████╔╝██║  ██║   ██║
╚═╝     ╚═╝  ╚═╝╚═╝  ╚═╝╚═╝  ╚═╝╚══════╝ ╚═════╝ ╚═╝  ╚═╝   ╚═╝
URL Parameter Categorizer & Extraction
by Ev3rPalestine
```

#### A powerful Python tool for automatically categorizing URLs by their parameters & extracting them to streamline penetration testing and vulnerability assessment workflows.

## Features

- **Automatic URL Categorization**: Sort URLs by vulnerability type (SQLi, XSS, SSRF, LFI, etc.)
- **Domain-Based Organization**: Output organized by domain folders
- **Parameters Extraction**: Extract found parameters in urls for fuzzing or other use
- **Custom Parameter Search**: Support for custom parameters via command line or file
- **Parameter Value Clearing**: Clean URLs by keeping only parameter names
- **Flexible Category Selection**: Choose specific vulnerability types, or use `web` / `api` presets
- **Dedup & Unique-Params**: Drop exact duplicates (`-dd`), normalized duplicates ignoring order/case/values (`-ddn`), or collapse same-param-set URLs (`-uq`)
- **Filters**: Exclude domains (`-ed`, supports `*` wildcards) and noisy params (`-xp`)
- **Top-N Focus**: Keep only the first INT URLs per category per domain (`-tp`)
- **Input Guard**: Abort oversized runs with `--max-urls`
- **Live-Host Check**: Keep only live URLs (`-hc`, HEAD-first with GET fallback, `--retries`, `--include-status`, persistent `--live-cache`)
- **Multiple Output Formats**: Classic `txt`, machine-readable `json`, flat `csv` (single or per-domain via `--csv-split`), or pipe-friendly `jsonl` / stdout (`-o -`, `--stdout`)
- **Self-Updating**: Upgrade to the latest release with `parasort --update`
- **Configurable Parameters**: External JSON configuration (`--init-config`, `--reset-config`), per-category validation warnings
- **Tested**: pytest suite (36 tests) + CI matrix across Python 3.11–3.14
- **Colorful Output**: Visual feedback with color-coded categories

## Installation

### Prerequisites
- Python 3.11 or higher
- pip package manager

### From PyPI (recommended)

```bash
pip install parasort
```

Isolated install with [pipx](https://pipx.pypa.io/):

```bash
pipx install parasort
```

Upgrade to the latest release:

```bash
pip install --upgrade parasort
# or: parasort --update
```

### From source

```bash
git clone https://github.com/Ev3rPalestine/Parasort.git
cd Parasort
pip install .
```
## Example

```bash
# parasort -i urls.txt


██████╗  █████╗ ██████╗  █████╗ ███████╗ ██████╗ ██████╗ ████████╗
██╔══██╗██╔══██╗██╔══██╗██╔══██╗██╔════╝██╔═══██╗██╔══██╗╚══██╔══╝
██████╔╝███████║██████╔╝███████║███████╗██║   ██║██████╔╝   ██║   
██╔═══╝ ██╔══██║██╔══██╗██╔══██║╚════██║██║   ██║██╔══██╗   ██║   
██║     ██║  ██║██║  ██║██║  ██║███████║╚██████╔╝██║  ██║   ██║   
╚═╝     ╚═╝  ╚═╝╚═╝  ╚═╝╚═╝  ╚═╝╚══════╝ ╚═════╝ ╚═╝  ╚═╝   ╚═╝   
                    URL Parameter Categorizer & Extraction
                         by Ev3rPalestine
    
Processing 858 URLs using 9 categories...
Processing: example.com
Processing: example.org
Processing: example2.com

==================================================
PROCESSING COMPLETE
==================================================
URLs processed: 858
Domains found: 22

GLOBAL CATEGORY SUMMARY
--------------------------------------------------
  uncategorized      :   641 URLs
  xss                :   122 URLs| Parameters: AdvertiserID, _charset_, _gl, belboon, dclid ... (+33 more)
  info_disclosure    :    66 URLs| Parameters: v
  sqli               :    21 URLs| Parameters: _charset_, cnt, groupId, id, k ... (+13 more)
  business_logic     :     8 URLs| Parameters: actionId, advancePaymentId, backLink, calcType, carModel ... (+25 more)
  ssrf               :     7 URLs| Parameters: id, image, k, limit, offset ... (+5 more)
  lfi                :     6 URLs| Parameters: ANAME, FNAME, LAF_PARTNER, Page, action ... (+9 more)
  open_redirect      :     6 URLs| Parameters: id, k, limit, offset, p_l_id ... (+4 more)
==================================================
Output directory: results/

```

## Available Arguments
```text
INPUT / OUTPUT:
  -i, --input FILE      Input file containing URLs (optional if using stdin)
  -o, --output DIR      Output directory (default: results); use - for stdout JSONL
  -f, --format FORMAT   Output format: txt (per-domain *-urls.txt),
                        json (per-domain .json + summary.json),
                        csv (urls.csv, see -cs) or
                        jsonl (single urls.jsonl) (default: txt)
  -st, --stdout         Same as -o -: emit JSONL records to stdout (pipe-friendly)
  -mx, --max-urls INT     Abort with an error if input exceeds INT URLs (no limit by default)

PROCESSING OPTIONS:
  -c, --clear           Clear parameter values, keep names only (e.g., -c)
  -v, --verbose         Show detailed processing information (e.g., -v)
  -s, --silent          Minimal output for scripting (e.g., -s)
  -nc, --no-color       Disable colored output (e.g., -nc)
  -ep, --extract-params Extract all parameters to parameters.txt files
  -dd, --dedup          Drop exact duplicate URLs before processing
    -ddn, --dedup-normalized
                        Drop URLs with identical param sets, ignoring
                        order/case/values (e.g., -ddn)
  -uq, --unique-params  Keep one URL per param-name set per domain
  -tp, --top INT          Keep only the first INT URLs per category per domain

VULNERABILITY CATEGORIES:
  -vl, --vuln CATEGORY [CATEGORY ...]
                        Vulnerability categories (e.g., -vl sqli xss)
  -pf, --profile PROFILE
                        Preset set: web or api (explicit -vl overrides -pf)

CUSTOM PARAMETERS:
  -cp, --custom-params PARAM [PARAM ...]
                        Custom parameters (comma or space separated,
                        e.g., -cp "id,user,cmd" or -cp id user cmd)
  -cpf, --custom-params-file FILE
                        File with custom parameters (e.g., -cpf params.txt)

FILTERS:
  -ed, --exclude-domain DOMAIN [DOMAIN ...]
                        Skip domains (exact or * wildcard, e.g.,
                        -ed static.example.com "*.cdn.com")
  -xp, --exclude-param PARAM [PARAM ...]
                        Ignore params for matching/extraction
                        (e.g., -xp "utm_source,fbclid")
  -cs, --csv-split MODE  CSV layout with -f csv: single writes one top-level
                        urls.csv (domain,category,url); per-domain writes one
                        urls.csv per domain folder (category,url) (default: single)

NETWORK:
  -hc, --status-check   Keep only live URLs (concurrent probes; 2xx-3xx count
                        as live, see -is) (e.g., -hc)
  -m, --method METHOD   Live-probe method for -hc: auto is HEAD first with GET
                        fallback (fastest), head or get (default: auto)
  -rt, --retries INT      Extra attempts per URL when a probe fails with a network
                        error, 0-10 (default: 0)
  -is, --include-status CODE [CODE ...]
                        Extra HTTP status codes counted as live on top of
                        2xx-3xx (e.g., -is 403 404)
  -lc, --live-cache FILE
                        JSON file caching live-check results across runs;
                        entries younger than -ct are reused without probing
                        (e.g., -lc ~/.parasort/live_cache.json)
  -ct, --cache-ttl SEC  Reuse -lc cache entries younger than SEC seconds
                        (default: 86400 = 24h)

INFORMATION:
  -h, --help            Show this help message and exit
  -sc, --show-categories
                        List available vulnerability categories
  -lp, --list-params [CATEGORY]
                        List parameters of a category (e.g., -lp sqli)
  -V, --version         Show version and exit
  -ic, --init-config   Create ~/.parasort/parameter_categories.json with built-in
                        defaults if missing, print its path, and exit (e.g., -ic)
  -rc, --reset-config  Overwrite the config file with built-in defaults,
                        discarding customizations, and exit (e.g., -rc)
  -up, --update         Update parasort to the latest release via pip

EXAMPLES:
  Basic usage:
    parasort -i urls.txt
    parasort -i urls.txt -o results -f json
    cat urls.txt | parasort -o results

  Specific vulnerabilities / profiles:
    parasort -i urls.txt -o results -vl sqli xss
    parasort -i urls.txt -o results -pf web

  Custom parameters:
    parasort -i urls.txt -o results -cp "id,user,cmd"
    parasort -i urls.txt -o results -cpf my_params.txt

  Clean + focus output:
    parasort -i urls.txt -o results -c -dd -uq -tp 20
    parasort -i urls.txt -o results -c -ddn -mx 50000
    parasort -i urls.txt -o results -ed static.example.com "*.cdn.com" -xp "utm_source,fbclid"

  Machine-readable piping:
    parasort -i urls.txt -o - | nuclei -l /dev/stdin
    parasort -i urls.txt -f jsonl -o results
    parasort -i urls.txt -f csv -cs per-domain

  Live hosts only:
    parasort -i urls.txt -o results -hc -to 8
    parasort -i urls.txt -o results -hc -m head -rt 2 -is 403 404
    parasort -i urls.txt -o results -hc -lc ~/.parasort/live_cache.json

  Config:
    parasort -ic
    parasort -rc

  Advanced options:
    parasort -i urls.txt -o results --clear -v
```

## Basic Usage

```bash
# Show help with all examples
parasort -h

# Show categories / version
parasort -sc
parasort -V

# Process custom parameters and clearing their values
parasort -i urls.txt -o results -cp "id,token,api_key,session" -c

# Process with specific vulnerabilities only (or a preset profile)
parasort -i urls.txt -o results -vl sqli xss -v
parasort -i urls.txt -o results -pf web

# Clean duplicates and focus output
parasort -i urls.txt -o results -dd -uq -tp 20
parasort -i urls.txt -o results -ddn -mx 50000

# Exclude noise, keep machine-readable output
parasort -i urls.txt -o results -ed "*.cdn.com" -xp "utm_source,fbclid" -f json

# Pipe JSONL straight into other tools
parasort -i urls.txt -o - | nuclei -l /dev/stdin

# Live hosts with HEAD-first probing and a persistent cache
parasort -i urls.txt -o results -hc -m auto -rt 2 -lc ~/.parasort/live_cache.json

# Config management
parasort -ic
parasort -rc

# Silent mode
parasort -i urls.txt -o results -s
```

## Command Line Options
#### - Input / Output

    -i, --input FILE - Input file containing URLs (optional if using stdin)

    -o, --output DIR - Output directory (default: results); use - for stdout JSONL (pipe-friendly)

    -f, --format FORMAT - Output format: txt, json, csv or jsonl (default: txt)

    -st, --stdout - Same as -o -: emit JSONL records to stdout; human output goes to stderr (pipe-friendly)

    -mx, --max-urls INT - Abort with an error if total input (stdin + file) exceeds INT URLs (no limit by default)

#### - Processing Options

    -c, --clear - Clear parameter values, keep names only

    -v, --verbose - Show detailed processing information

    -s, --silent - Minimal output (for scripting)

    -nc, --no-color - Disable colored output

    -ep, --extract-params - Extract all parameters to parameters.txt files

    -dd, --dedup - Drop exact duplicate URLs before processing

    -ddn, --dedup-normalized - Drop URLs sharing (domain, path, param names), ignoring param order, case and values

    -uq, --unique-params - Keep one URL per param-name set per domain

    -tp, --top INT - Keep only the first INT URLs per category per domain

#### - Vulnerability Categories

    -vl, --vuln CATEGORY - Vulnerability categories (default: all)

        Available: all, sqli, xss, ssrf, lfi, open_redirect, command_injection, auth_bypass, business_logic, info_disclosure
        (plus any custom categories added via parameter_categories.json)

    -pf, --profile PROFILE - Preset set: web or api (explicit -vl overrides -pf)

#### - Custom Parameters

    -cp, --custom-params PARAM - Custom parameters to search for (comma or space separated)

    -cpf, --custom-params-file FILE - File with custom parameters (one per line)

#### - Filters

    -ed, --exclude-domain DOMAIN - Skip domains (exact or * wildcard, case-insensitive)

    -xp, --exclude-param PARAM - Ignore params for matching/extraction (comma or space separated)

    -cs, --csv-split MODE - CSV layout with -f csv: single writes one top-level urls.csv (domain,category,url); per-domain writes one urls.csv per domain folder (category,url) (default: single)

#### - Network

    -hc, --status-check - Keep only live URLs (concurrent probes; 2xx-3xx count as live, see -is)

    -m, --method METHOD - Live-probe method for -hc: auto is HEAD first with GET fallback (fastest), head or get (default: auto)

    -rt, --retries INT - Extra attempts per URL when a probe fails with a network error, 0-10 (default: 0)

    -is, --include-status CODE - Extra HTTP status codes counted as live on top of 2xx-3xx (e.g., -is 403 404)

    -lc, --live-cache FILE - JSON file caching live-check results across runs; entries younger than -ct are reused without probing

    -ct, --cache-ttl SEC - Reuse -lc cache entries younger than SEC seconds (default: 86400 = 24h)

    -to, --timeout SEC - HTTP timeout in seconds for -hc and -up (default: 10)

#### - Information

    -h, --help - Show help message and exit
    -sc, --show-categories - List available vulnerability categories
    -lp, --list-params [CATEGORY] - List parameters of a category (e.g., -lp sqli)
    -V, --version - Show version and exit
    -ic, --init-config - Create ~/.parasort/parameter_categories.json with built-in defaults if missing, print its path, and exit
    -rc, --reset-config - Overwrite the config file with built-in defaults, discarding customizations, and exit
    -up, --update - Update parasort to the latest release via pip


## Output Structure

The tool creates domain-based folders with categorized URL files (txt format):

```bash
results/
├── example.com/
│   ├── sqli-urls.txt
│   ├── xss-urls.txt
│   ├── ssrf-urls.txt
│   ├── custom-params-urls.txt
│   ├── parameters.txt          
│   └── uncategorized-urls.txt
├── target.org/
│   ├── sqli-urls.txt
│   ├── xss-urls.txt
│   ├── parameters.txt          
│   └── ...
└── all-parameters.txt   
```

With `-f json`: each domain gets `<domain>.json` (category → URL list) plus a
top-level `summary.json` with per-domain and total counts.
With `-f csv`: a single top-level `urls.csv` (`domain,category,url`), or per-domain
`urls.csv` files (`category,url`) with `--csv-split per-domain`.
With `-f jsonl`: a single top-level `urls.jsonl` (one `{"domain","category","url"}`
record per line). With `-o -` / `--stdout`, those JSONL records go to stdout
(human output goes to stderr) for piping into other tools.
`parameters.txt` / `all-parameters.txt` (`-ep`) are always plain-text lists
(skipped in stdout mode).

## Configuration
#### - Parameter Categories File

The parameter_categories.json file is created and located in directory named ".parasort" in the home directory of the user, it defines which parameters belong to each vulnerability category. You can customize this file to add or remove parameters:

```json
{
  "sqli": ["id", "user_id", "product_id"],
  "xss": ["q", "search", "query"],
  "ssrf": ["url", "redirect", "image"],
  ...
}
```

* The file is automatically created on first run if it doesn't exist.
* The `parameter_categories.json` in this repo is a reference copy of the built-in
  defaults; PyPI wheels rely on the embedded defaults and create `~/.parasort/parameter_categories.json` at runtime.


#### - Adding New Vulnerability Categories
* Edit parameter_categories.json
* Add your new category with relevant parameters
* The tool will automatically detect the new category

#### - Using Custom Parameter Files

Create a text file with one parameter per line:

```text
api_key
session_token
user_id
admin
debug
```

### - License

```text
This project is licensed under the MIT License - see the LICENSE file for details.
Author

Note: This tool is intended for legitimate security testing and educational purposes only. Always ensure you have proper authorization before testing any systems.
```

## Changelog

See [CHANGELOG.md](CHANGELOG.md) for release history. Current version: **5.1.0**.

