Metadata-Version: 2.4
Name: privatext
Version: 0.1.0
Summary: Privacy text masking and unmasking tool for protecting PII before sending to LLMs
Project-URL: Homepage, https://github.com/pandeng-001/privateText
Project-URL: Repository, https://github.com/pandeng-001/privateText
Project-URL: Issues, https://github.com/pandeng-001/privateText/issues
Author-email: PrivaText Team <1912936908@qq.com>
License: MIT
License-File: LICENSE
Keywords: llm,masking,pii,privacy,security,text-processing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Text Processing
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# PrivaText

🔐 A privacy-focused Python tool for masking and unmasking personally identifiable information (PII) in text before sending to language models (LLMs) and restoring it after receiving responses.

## Features

- ✅ **Automatic PII Detection**: Recognize phone numbers, emails, ID cards, API keys, and more
- ✅ **Manual Tagging**: Use `[[[ ]]]` to explicitly mark sensitive content
- ✅ **Flexible Configuration**: Enable/disable specific PII types as needed
- ✅ **Robust Unmask**: Handle case variations from LLM responses
- ✅ **Escape Support**: Use `\[\[\[` to escape manual tags
- ✅ **Easy Integration**: Simple API for mask/unmask operations
- ✅ **Production Ready**: Comprehensive test coverage with 20+ unit tests

## Installation

### Using pip

```bash
pip install privatext
```

### Using uv (recommended for faster installation)

```bash
uv pip install privatext
```

### From source

```bash
git clone https://github.com/yourusername/privatext.git
cd privatext
pip install .
# Or with uv:
uv pip install .
```

## Quick Start

```python
from privatext import PrivaText

# Initialize
pt = PrivaText()

# Mask sensitive information
raw_text = "拨打13800138000，或者发邮件到 test@me.com，我的暗号是 [[[天王盖地虎]]]"
masked = pt.mask(raw_text)
print("Masked:", masked)
# Output: 拨打__PRIV_PHONE_1__，或者发邮件到 __PRIV_EMAIL_1__，我的暗号是 __PRIV_CUSTOM_1__

# View vault mapping
print("Vault:", pt.get_vault())
# Output: {
#   '__PRIV_PHONE_1__': '13800138000',
#   '__PRIV_EMAIL_1__': 'test@me.com',
#   '__PRIV_CUSTOM_1__': '[[[天王盖地虎]]]'  # Note: stores complete original form
# }

# Send masked text to LLM...
# Then unmask the response
llm_response = "您可以拨打__PRIV_PHONE_1__联系我们"
unmasked = pt.unmask(llm_response)
print("Unmasked:", unmasked)
# Output: 您可以拨打13800138000联系我们
```

## Configuration

### View Current Configuration

```python
from privatext import PrivaText

pt = PrivaText()
print(pt.config)
# Output: {
#   'phone': True,
#   'email': True,
#   'id_card': True,
#   'address': False,
#   'secret_key': True,
#   'custom_tag_only': False
# }
```

### Modify Configuration

```python
# Modify individual config items
pt.config['email'] = False          # Disable email detection
pt.config['address'] = True         # Enable address detection

# Or batch update
pt.update_config({
    'email': False,
    'address': True,
    'phone': False
})
```

### Configuration Options

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `phone` | bool | `True` | Detect Chinese mobile numbers (1[3-9]xxxxxxxxx) |
| `email` | bool | `True` | Detect email addresses |
| `id_card` | bool | `True` | Detect ID card numbers (15 or 18 digits) |
| `address` | bool | `False` | Detect addresses (disabled by default, regex matching is imprecise) |
| `secret_key` | bool | `True` | Detect API keys and tokens |
| `custom_tag_only` | bool | `False` | If enabled, only mask manually tagged `[[[ ]]]` content |

## Advanced Usage

### Manual Tagging with Priority

Manually marked content using `[[[ ]]]` has the highest priority and won't be double-masked:

```python
pt = PrivaText()
text = "Contact: admin@company.com or secret: [[[api_key_12345]]]"
masked = pt.mask(text)
# Result: Contact: __PRIV_EMAIL_1__ or secret: __PRIV_CUSTOM_1__
# Vault: {
#   '__PRIV_EMAIL_1__': 'admin@company.com',
#   '__PRIV_CUSTOM_1__': '[[[api_key_12345]]]'  # Complete form stored
# }
```

### Custom-Tag-Only Mode

Enable to mask only manually tagged content and ignore automatic detection:

```python
pt = PrivaText()
pt.config['custom_tag_only'] = True

text = "Phone: 13800138000, Secret: [[[password123]]]"
masked = pt.mask(text)
print(masked)
# Output: Phone: 13800138000, Secret: __PRIV_CUSTOM_1__
# (Phone is NOT masked because custom_tag_only is enabled)
```

### Escaping Manual Tags

Use `\[\[\[` to escape the manual tag syntax if you have literal `[[[` in your text:

```python
pt = PrivaText()
text = r"Example code: \[\[\[my_secret\]\]\]"
masked = pt.mask(text)
print(masked)
# Output: Example code: [[[my_secret]]]
# (The escape sequence is converted to literal brackets)
```

### Case-Insensitive Unmask

The unmask function handles case variations from LLM responses:

```python
pt = PrivaText()
text = "Call 13800138000"
masked = pt.mask(text)
# masked = "Call __PRIV_PHONE_1__"

# LLM might change case
modified = masked.lower()  # "call __priv_phone_1__"
unmasked = pt.unmask(modified)
print(unmasked)  # "Call 13800138000"
```

## Placeholder Format

Placeholders follow the format: `__PRIV_{TYPE}_{INDEX}__`

Examples:

- `__PRIV_PHONE_1__`: First detected phone number
- `__PRIV_EMAIL_2__`: Second detected email
- `__PRIV_ID_CARD_1__`: First detected ID card
- `__PRIV_CUSTOM_1__`: First manually tagged content
- `__PRIV_SECRET_KEY_1__`: First detected API key

## Regular Expressions Used

| Type | Pattern | Description |
|------|---------|-------------|
| `phone` | `1[3-9]\d{9}` | Chinese mobile numbers |
| `email` | RFC-like pattern | Standard email format |
| `id_card` | `\d{15}(?:\d{2}[0-9Xx])?` | 15 or 18 digit ID cards |
| `secret_key` | Multiple patterns | API keys, tokens, with common prefixes |
| `address` | Chinese cities | Major Chinese cities (requires more work for accuracy) |

## API Reference

### `PrivaText()`

Initialize a new PrivaText instance.

### `mask(text: str) -> str`

Mask PII in text based on configuration. Returns masked text with placeholders.

### `unmask(text: str) -> str`

Restore original values from placeholders. Supports case-insensitive matching.

### `get_vault() -> Dict[str, str]`

Get a copy of the current vault mapping (placeholder -> original value).

### `update_config(new_config: Dict[str, Any]) -> None`

Batch update configuration items.

### `config: Dict[str, Any]`

Dictionary containing current configuration. Can be modified directly.

## Testing

Run the comprehensive test suite:

```bash
# Using unittest
python -m unittest discover tests

# Using pytest (if installed)
pytest tests

# Specific test file
python -m unittest tests.test_privatext
```

The test suite includes:

- Configuration functionality tests
- Masking accuracy tests
- Priority handling (manual tags override auto-detection)
- Unmasking and restoration tests
- Vault management tests
- Integration and roundtrip tests

## Real-World Example: Using with LLMs

```python
from privatext import PrivaText
import anthropic

pt = PrivaText()

# Original query with sensitive info
user_query = """
I need help with my account.
Phone: 13800138000
Email: john@company.com
Account ID: 110101199003076019
Secret code: [[[XXXXXXXX]]]
"""

# Mask before sending to LLM
masked_query = pt.mask(user_query)

# Send to LLM
client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    messages=[{"role": "user", "content": masked_query}]
)

# Unmask the response
unmasked_response = pt.unmask(response.content[0].text)
print(unmasked_response)
```

## Security Considerations

⚠️ **Important**: This library is designed to reduce the risk of accidentally leaking PII to LLM services, but it is not a complete security solution.

- **Regex Limitations**: Regular expressions may not catch all variations of PII formats
- **LLM Processing**: LLMs may still infer sensitive information from context
- **Manual Tags Only**: For maximum security, use only manual `[[[ ]]]` tagging instead of auto-detection
- **Regular Updates**: Keep the library updated for improved patterns and security features

For highly sensitive applications, consider:

1. Using a private/self-hosted LLM
2. Combining with additional encryption
3. Regular audits of LLM responses
4. Using `custom_tag_only=True` mode for explicit control

## Development

### Setup Development Environment

```bash
git clone https://github.com/yourusername/privatext.git
cd privatext

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install in editable mode
pip install -e .

# Run tests
python -m unittest discover tests
```

### Project Structure

```bash
privatext/
├── privatext/
│   ├── __init__.py          # Package exports
│   └── core.py              # Core PrivaText class
├── tests/
│   ├── __init__.py
│   └── test_privatext.py    # Comprehensive test suite
├── pyproject.toml           # Project configuration
├── README.md                # This file
└── LICENSE                  # MIT License
```

## License

MIT License - see LICENSE file for details

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## Changelog

### v0.1.0 (2026)

- Initial release
- Core masking and unmasking functionality
- Support for multiple PII types
- Comprehensive test suite
- Full documentation

## Roadmap

- [ ] Support for more PII types (credit cards, bank accounts)
- [ ] Multi-language support
- [ ] Streaming unmask for large texts
- [ ] Web UI dashboard
- [ ] Async interface for high-throughput scenarios
- [ ] Performance optimizations for large documents

## FAQ

**Q: Will this completely protect my privacy?**
A: No. This tool reduces accidental leakage but is not foolproof. Use with other security measures for sensitive data.

**Q: Does it support custom PII patterns?**
A: Currently, you can access `pt.patterns` to understand the regex patterns used. Custom patterns support is planned for v0.2.

**Q: How does it handle nested/complex formats?**
A: The current implementation handles most common cases. Complex nested structures may require pre-processing.

**Q: Can I use this with non-English text?**
A: Yes! The library works with any UTF-8 text, including Chinese, and includes patterns optimized for Chinese phone numbers and ID cards.

## Support

For issues, questions, or suggestions, please visit: [this](https://github.com/pandeng-001/privateText/issues)
