Metadata-Version: 2.4
Name: tango-keystore
Version: 1.0.0
Summary: Type-preserving key-value store for Tango database
Author-email: Ezequiel Panepucci <ezequiel.panepucci@maxiv.lu.se>
License-Expression: GPL-3.0-or-later
Project-URL: Homepage, https://gitlab.com/maxiv/tools/tango-keystore
Project-URL: Repository, https://gitlab.com/maxiv/tools/tango-keystore/
Project-URL: Issues, https://gitlab.com/maxiv/tools/tango-keystore/issues
Keywords: tango,database,keystore,controls,maxiv
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pytango>=9.3.0
Requires-Dist: rich>=13.0.0
Provides-Extra: tests
Requires-Dist: pytest; extra == "tests"
Requires-Dist: pytest-cov; extra == "tests"
Requires-Dist: pytango-db; extra == "tests"
Provides-Extra: dev
Requires-Dist: ipython; extra == "dev"
Dynamic: license-file

# Tango Free Properties with Type Preservation

A Python module for managing key-value pairs in Tango database's free property section while preserving Python types.

Properties go into the **TangoKeystore** namespace by default.

<img alt="Jive free properties" src="docs/jive-free-properties.png" width="400">

## Problem

Tango database stores all properties as strings. When you store a float like `1.2`, it gets stored as the string `'1.2'`. This module solves this by storing type metadata alongside the value, enabling proper type restoration upon retrieval.

## Features

- ✅ Preserves original Python types (int, float, str, bool, list, dict, None)
- ✅ Automatic serialization/deserialization using JSON
- ✅ Simple, intuitive API
- ✅ Bulk operations (get_all, clear)
- ✅ Key existence checking
- ✅ No default values, keys must exist before being accessed
- ✅ Keys must be explicitly created before being accessed
  - either with `must_exist=False` or by using the `create` method.
- ✅ Key name validation: `r"^[a-zA-Z_][a-zA-Z0-9_]*$"`, at most 255 characters
  - this means keys are also valid python identifiers
- ✅ Optional per-key descriptions (up to 500 characters)

The command line tool takes input in the **JSON format** whereas the python module uses **Python types**.

## Value Type Parsing

The CLI automatically detects and converts value types:

| Input | Parsed Type | Example |
|-------|-------------|---------|
| `42` | int | `count=42` |
| `-10` | int | `offset=-10` |
| `3.14` | float | `pi=3.14` |
| `-2.5` | float | `delta=-2.5` |
| `true`, `false` | bool | `enabled=true` |
| `none`, `null` | NoneType | `optional=none` |
| `[1,2,3]` | list | `values=[1,2,3]` |
| `{"a":1}` | dict | `config={"timeout":30}` |
| `hello` | str | `name=hello` |


## Installation

`tango-keystore` is on [PyPI](https://pypi.org/project/tango-keystore/) and [conda-forge](https://github.com/conda-forge/tango-keystore-feedstock).
Install it with your preferred package manager: `pip`, `uv`, `conda` or `pixi`.

```bash
pip install tango-keystore
```

## Usage


### Basic Example

```python
from tango_keystore import TangoKeystore

# Initialize
keystore = TangoKeystore()

# Store values with automatic type preservation
keystore.put("omega_reference", 1.2, must_exist=False)
keystore.put("iteration_count", 42, must_exist=False)
keystore.put("is_enabled", True, must_exist=False)

# Retrieve with correct types
omega = keystore.get("omega_reference")  # Returns float 1.2
count = keystore.get("iteration_count")  # Returns int 42
enabled = keystore.get("is_enabled")    # Returns bool True

print(type(omega))  # <class 'float'>
```

### Complex Types

```python
# Store lists and dictionaries
keystore.put("coordinates", [10.5, 20.3, 30.1])
keystore.put("config", {"timeout": 30, "retry": 3})

# Retrieve with proper types
coords = keystore.get("coordinates")  # Returns list
config = keystore.get("config")       # Returns dict
```

### utilities
```python
# get history for a Key (timestamp is epoch)
history = keystore.get_history("omega_reference")
[{'key': 'omega_reference',
  'value': 1.2,
  'type': 'float',
  'timestamp': 1763123518.0},
 {'key': 'omega_reference',
  'value': 4.2,
  'type': 'float',
  'timestamp': 1763123535.0},
 {'key': 'omega_reference',
  'value': 4.2,
  'type': 'float',
  'timestamp': 1763123740.0}]

# feature flags
keystore.put("room_temperature", "on", must_exist=False)
keystore.is_active("room_temperature")
True
keystore.is_on("room_temperature")
True
keystore.is_active("room_temperature")
True
keystore.is_inactive("room_temperature")
False
keystore.is_off("room_temperature")
False
```

### Values Inside Dicts and Lists (jq-style paths)

Values inside a dict or list can be read and set with a [jq](https://jqlang.org/)-style path,
without rewriting the whole value.

```python
keystore.put("limits", {"min": -2.0, "max": 3.2, "items": [1.0, 2.0]}, must_exist=False)

keystore.get_path(".limits.max")          # 3.2
keystore.put_path(".limits.max", 4.56)
keystore.put_path(".limits.items[0]", 1.5)
keystore.put_path(".limits.step", 0.1, create=True)   # add a new key to the dict

# several values at once: validated together, each key written once
keystore.put_paths([(".limits.min", -3.0), (".limits.max", 5.0)])
```

Supported syntax (a subset of jq): `.key`, `."key with spaces"`, `.key["other key"]`,
`.list[0]`, `.list[-1]`.

- Key names are matched ignoring case, at every level, and keep their original spelling:
  `.LIMITS.MAX` updates `limits.max`. If a dict has keys differing only in case, an exact
  match is required.
- A value must keep its type; an int is accepted for a float (`.limits.max=4` stores `4.0`),
  and a `null` can be replaced by anything.
- `create=True` (`-c` on the command line) adds a missing key to an existing dict; it never
  creates intermediate dicts or new list items.
- The paths printed by `--find` can be used directly.

### Key Descriptions

Descriptions are optional. A key's description is kept when its value changes.

```python
# describe a key when creating (or updating) it
keystore.put("omega_reference", 1.2, must_exist=False, description="Omega offset after realignment")

# describe an existing key without changing its value
keystore.describe("omega_reference", "Omega offset measured on 2026-09-30")

keystore.get_description("omega_reference")
'Omega offset measured on 2026-09-30'

# remove the description
keystore.describe("omega_reference", None)  # or ""
```

Saving a tagged snapshot (`--save-tag`) copies the descriptions; loading one (`--load-tag`)
restores values only and keeps the current descriptions.

### Tagged Snapshots

A snapshot copies (some of) the keys into a separate namespace, `{namespace}_{tag}`,
so they can be restored later. The CLI's `--save-tag`, `--load-tag`, `--list-tags`,
`--compare-tags` and `--delete-tag` options use these methods.

```python
# save some keys (all keys if keyset is omitted)
keystore.save_tag("hve", keyset=["ssx_mode", "md3_omega_limit"], description="HVE parameters")

# add a key to the snapshot, or refresh all its keys with the current values
keystore.save_tag("hve", keyset=["panda_device"])
keystore.save_tag("hve")

keystore.list_tags()             # [{'tag': 'hve', 'description': ..., 'keyset': [...], 'timestamp': ...}]
keystore.get_tag_info("hve")
keystore.compare_tags("hve")     # {key: (snapshot value, current value)} for keys that differ
keystore.load_tag("hve")         # restore the values
keystore.last_loaded_tag()       # 'hve'
keystore.delete_keys_from_tag("hve", ["panda_device"])
keystore.delete_tag("hve")
```

### Additional Operations

```python
# Check if key exists
if keystore.exists("omega_reference"):
    print("Key exists!")

# get with metadata
keystore.get_with_metadata("temperature")
{'value': 35.0, 'type': 'float'}

# Get all keys
all_keys = keystore.get_all_key_names()

# Get all properties as dictionary
all_props = keystore.get_all()

# Delete a property
keystore.delete("omega_reference")

# Clear all properties, requires confirmation code.
# Freja+ coming soon.
keystore.clear()
Type confirmation code to continue => 18398  : 18398
```

### Using Custom Database Connection

```python
import tango

# Use specific database
db = tango.Database("host", "port")
keystore = TangoKeystore(db=db)
```

## API Reference

### `TangoKeystore(namespace="FunkyKeystore)`
Initialize the manager with optional namespace.

### `put(key, value, must_exist=True, description=None)`
Store a value with type preservation.
- **key**: Property name (str)
- **value**: Value to store (must be JSON-serializable)
- **description**: Optional description (str, max 500 characters); `None` keeps the current one

### `describe(key, description)`
Set the description of an existing key without changing its value; `None` or `""` removes it.

### `get_description(key)`
- **Returns**: the key's description, or `None`

### `get(key)`
Retrieve a value with type restoration.
- **key**: Property name (str)
- **Returns**: Original value with proper type

### `delete(key)`
Delete a property.

### `exists(key)`
Check if a property exists.
- **Returns**: bool

### `get_all_keys()`
Get all property keys.
- **Returns**: list of keys

### `get_all()`
Get all properties as a dictionary.
- **Returns**: dict mapping keys to typed values

### `clear()`
Delete all properties (use with caution!).

## How It Works

The module wraps each value in a JSON object containing both the value and its type:

```python
# When you call: keystore.put("omega_reference", 1.2)
# It stores: '{"value": 1.2, "type": "float"}'

# A key with a description also stores it:
# '{"value": 1.2, "type": "float", "description": "Omega offset"}'

# When you call: keystore.get("omega_reference")
# It retrieves the JSON, extracts the value and type, and returns: 1.2 (as float)
```

## Supported Types

- `int`
- `float`
- `str`
- `bool`
- `list`
- `dict`
- `NoneType` (None)

## Error Handling

- Raises `TypeError` for unsupported types
- Raises `ValueError` for corrupted or invalid data
- Returns default value if key doesn't exist (instead of raising exception)

## License

See `LICENSE` file.

## Disclaimer

This package was initially produced with AI tools from Claude by Anthropic.
