Metadata-Version: 2.1
Name: wexample-wex-addon-services-db
Version: 12.1.2
Summary: Provides wex with dump, restore and connect commands for PostgreSQL, MySQL, MongoDB, Redis and other database containers
Author-Email: weeger <contact@wexample.com>
License: MIT
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Requires-Dist: wexample-wex-addon-app>=30.0.0
Requires-Dist: wexample-wex-core>=30.0.0
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Description-Content-Type: text/markdown

# wex_addon_services_db

Version: 12.1.2

`wex-addon-services-db` extends [wex](https://wexample.com) with database-management commands — dump, restore, connect, exec, and destroy — that run against Docker containers for PostgreSQL, MySQL, MariaDB, MongoDB, SQL Server, Redis, Elasticsearch, ClickHouse, and Neo4j. It is aimed at developers who manage multi-service applications through wex and need a uniform CLI interface for routine database operations across engines, without writing engine-specific shell scripts.

## Table of Contents

- [Installation](#installation)
- [Quickstart](#quickstart)
- [Tests](#tests)
- [Architecture](#architecture)
- [Integration in the Suite](#integration-in-the-suite)
- [Dependencies](#dependencies)
- [Versioning & Compatibility Policy](#versioning--compatibility-policy)
- [License](#license)
- [About us](#about-us)
- [Known Limitations & Roadmap](#known-limitations--roadmap)
- [Status & Compatibility](#status--compatibility)
- [Useful Links](#useful-links)
- [Migration Notes](#migration-notes)

## Installation

```bash
pip install wexample-wex-addon-services-db
```

Requires Python >=3.10.

## Quickstart

Install the package into the same Python environment as wex:

```bash
pip install wexample-wex-addon-services-db
```

The public entry point is `ServicesDbAddonManager`, exported from src/wexample_wex_addon_services_db/services_db_addon_manager.py:

```python
from wexample_wex_addon_services_db import ServicesDbAddonManager
```

Pass it to the kernel at startup alongside the other addon managers your wex instance loads:

```python
from wexample_wex_core.common.kernel import Kernel
from wexample_wex_core.addons.core.core_addon_manager import CoreAddonManager
from wexample_wex_addon_services_db import ServicesDbAddonManager

kernel = Kernel(entrypoint_path=wex_dir)
kernel.setup(addons=[CoreAddonManager, ServicesDbAddonManager])
```

Once the kernel is running, configure a postgres service from the app's root directory:

```bash
wex @postgres::service/install
```

The command is implemented in src/wexample_wex_addon_services_db/services/postgres/commands/service/install.py. It writes `service.postgres.host`, `service.postgres.name`, `service.postgres.user`, `service.postgres.port`, and a randomly-generated `service.postgres.password` into the app's config file, then rebuilds the runtime config. It logs:

```
Configured postgres service for app '<name>'
```

After `wex app/start` brings the container up, verify it is accepting connections:

```bash
wex @postgres::service/ready
```

That runs `SELECT 1` inside the container and returns `True` when the exit code is 0. To dump the current database to a file:

```bash
wex @postgres::db/dump -f backup
```

The dump lands in `.wex/setup/postgres/dumps/backup.sql` inside the app workdir. To get the connection string for use in application config or a client tool:

```bash
wex @postgres::db/connect
# postgresql://postgres:"<password>"@localhost/<db>
```

The same `service/install`, `service/ready`, `db/dump`, `db/restore`, and `db/connect` commands exist for every engine the addon ships: `maria`, `mysql`, `mongo`, `sqlserver`, `redis`, `elasticsearch`, `clickhouse`, and `neo4j`. Substitute the engine name for `postgres` in the `@<engine>::` prefix.

## Tests

This project uses `pytest` for testing and `pytest-cov` for code coverage analysis.

### Installation

First, install the required testing dependencies:
```bash
.venv/bin/python -m pip install pytest pytest-cov
```

### Basic Usage

Run all tests with coverage:
```bash
.venv/bin/python -m pytest --cov --cov-report=html
```

### Common Commands
```bash
# Run tests with coverage for a specific module
.venv/bin/python -m pytest --cov=your_module

# Show which lines are not covered
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing

# Generate an HTML coverage report
.venv/bin/python -m pytest --cov=your_module --cov-report=html

# Combine terminal and HTML reports
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing --cov-report=html

# Run specific test file with coverage
.venv/bin/python -m pytest tests/test_file.py --cov=your_module --cov-report=term-missing
```

### Viewing HTML Reports

After generating an HTML report, open `htmlcov/index.html` in your browser to view detailed line-by-line coverage information.

### Coverage Threshold

To enforce a minimum coverage percentage:
```bash
.venv/bin/python -m pytest --cov=your_module --cov-fail-under=80
```

This will cause the test suite to fail if coverage drops below 80%.

## Architecture

This addon extends the wex CLI with dump, restore, connect, and lifecycle commands for nine database engines running in Docker containers: PostgreSQL, MySQL, MariaDB, MongoDB, Redis, SQL Server, ClickHouse, Elasticsearch, and Neo4j. Every command it provides follows the same three-layer structure: an addon manager that registers the package, a per-engine service module, and individual command files grouped by concern.

### Addon registration

src/wexample_wex_addon_services_db/services_db_addon_manager.py is the single entry point:

```python
class ServicesDbAddonManager(AbstractAddonManager):
    @classmethod
    def get_package_module(cls) -> Any:
        import wexample_wex_addon_services_db
        return wexample_wex_addon_services_db
```

wex calls `get_package_module()` and then walks the returned module tree to discover every function decorated with `@command`. No explicit command registration is needed — placing a file in the right directory and decorating the function is sufficient.

### Service structure

Each engine lives under `src/wexample_wex_addon_services_db/services/<engine>/`. The layout is the same for every engine:

```
services/<engine>/
├── app_service.py          # filesystem requirements for the container
└── commands/
    ├── config/
    │   └── runtime.py      # writes db.main into the runtime config
    ├── db/
    │   ├── connect.py      # return a connection string or arguments
    │   ├── destroy.py      # drop (and optionally recreate) the database
    │   ├── dump.py         # dump to disk via docker exec
    │   ├── dumps_list.py   # list dump files on disk
    │   ├── exec.py         # execute a one-shot SQL/command
    │   ├── go.py           # open an interactive CLI session
    │   └── restore.py      # restore from a dump file
    └── service/
        ├── install.py      # write connection credentials into the app config
        ├── ready.py        # health-check the running container
        └── setup.py        # optional one-time setup (mongo only)
```

Not every engine has every command. Redis and the search engines (Elasticsearch, Neo4j, ClickHouse) omit the `db/` namespace; SQL Server omits `connect.py`; only `mongo` has `service/setup.py`.

### Workdir contributions

`app_service.py` for each engine extends `AppService` from `wexample-wex-addon-app` and overrides `get_workdir_contribution()` to declare filesystem state that must exist before the container starts — directories, files, ownership, and permissions. The framework enforces these on the host.

Examples from src/wexample_wex_addon_services_db/services/postgres/app_service.py: a `setup/postgres/logs/` directory owned `999:999` mode `750`.

src/wexample_wex_addon_services_db/services/mongo/app_service.py adds a `mongo-keyfile` (`999:999`, mode `400`) alongside the `logs/` directory. The file is populated by `service/setup.py` using `openssl rand -base64 756`.

ClickHouse uses uid/gid `101:101`, Elasticsearch `1000:1000`, Neo4j `7474:7474`. MySQL declares a `setup/local/mysql.cnf` with mode `600`.

### Command namespaces

**`config/runtime.py`** reads the app config, extracts the database name from `service.<name>.name`, and writes a runtime config file that exposes `db.main`. Subsequent commands read this key as a fallback when no database is named explicitly.

**`service/install.py`** writes the full set of connection credentials into the app config file and rebuilds the runtime config. For postgres it writes host, name, password (random token), port `5432`, and user `postgres`. SQL Server uses port `1433` and user `sa`. Neo4j writes two ports (`port_bolt` 7687, `port_http` 7474). See src/wexample_wex_addon_services_db/services/postgres/commands/service/install.py for the pattern all `install.py` files follow.

**`service/ready.py`** runs a health probe inside the container via `docker exec` and returns a `BooleanResponse`. The probe is engine-specific: postgres runs `SELECT 1` with `psql`, Redis sends `PING` with `redis-cli`, Elasticsearch calls `curl -sf http://localhost:9200/_cluster/health`. See src/wexample_wex_addon_services_db/services/postgres/commands/service/ready.py.

**`db/dump.py`** creates a `dumps/` directory under the workdir's setup path, then calls the engine's dump tool inside the container (`pg_dump`, `mysqldump`, `mongodump`) and returns the host-side path to the file. See src/wexample_wex_addon_services_db/services/postgres/commands/db/dump.py.

**`db/go.py`** returns an `InteractiveShellCommandResponse` containing the `docker exec -ti ...` command. The wex runner launches the subprocess and hands the terminal to the user. See src/wexample_wex_addon_services_db/services/postgres/commands/db/go.py.

**`db/exec.py`** returns a `ShellCommandResponse` for a non-interactive one-shot query.

**`config/runtime.py`** (postgres example at src/wexample_wex_addon_services_db/services/postgres/commands/config/runtime.py): merges `db.main` into the base runtime config and writes the result.

### Call path through a command

Taking `postgres db/dump` as the example (src/wexample_wex_addon_services_db/services/postgres/commands/db/dump.py):

1. wex discovers `postgres__db__dump` by walking the module tree from `ServicesDbAddonManager.get_package_module()`.
2. wex resolves the target app and service, injects `ExecutionContext` and the postgres `AppService`.
3. The function reads credentials from `service.app_workdir.get_runtime_config()` — `app.project_name`, `service.<name>.user`, `service.<name>.password`, and `db.main`.
4. It derives the container name as `{app_project_name}_{service_name}` (matching the Docker naming convention `[app]_[env]_[service]` after the env component is stripped by wex).
5. It calls `subprocess.run(["docker", "exec", ..., "pg_dump", ...])` and raises `RuntimeError` if the exit code is non-zero.
6. It returns the host-side path to the written `.sql` file.

Every other command follows this same sequence: read runtime config → derive container name → call `docker exec` → return a value or response object.

### MariaDB delegates to MySQL

MariaDB has no independent SQL implementation. Each command under `services/maria/commands/db/` imports and directly calls the corresponding MySQL function:

```python
# services/maria/commands/db/dump.py
def maria__db__dump(...) -> str:
    return mysql__db__dump(context=context, service=service, file_name=file_name)
```

See src/wexample_wex_addon_services_db/services/maria/commands/db/dump.py. The `maria` command set exists only to expose the MySQL implementation under the correct service name.

MySQL's `db/connect` command returns `"--defaults-extra-file=/tmp/mysql.cnf"` — a credential argument string rather than a full DSN, matching how `mysqldump` and `mysql` accept credentials via a mounted config file.

### SQL Server helper

SQL Server is the only engine with a helper module. src/wexample_wex_addon_services_db/services/sqlserver/helper/sqlcmd.py provides `build_sqlcmd_command()`, which probes three candidate binary paths inside the container (`/opt/mssql-tools18/bin/sqlcmd`, `/opt/mssql-tools/bin/sqlcmd`, then `command -v sqlcmd`) before assembling the `docker exec` invocation. This handles the path difference between mssql-tools versions. All SQL Server `db/` commands call this helper rather than building the shell command directly.

### Tags

src/wexample_wex_addon_services_db/const/tags.py defines `DomainTag` with one constant per engine (`DB_POSTGRES`, `DB_MYSQL`, `DB_MONGO`, etc.) plus `DB`, `CONFIG`, and `SERVICE`. Every `@command` declaration combines these with `AudienceTag`, `EffectTag`, and `ScopeTag` from `wexample_cli`. The tags serve two purposes: agent filtering (commands tagged `AudienceTag.DANGEROUS` are not called by automated agents; `AudienceTag.HUMAN_ONLY` covers interactive `go` commands) and effect annotation (`EffectTag.DESTRUCTIVE` on `destroy` and `restore`, `EffectTag.IDEMPOTENT` on `ready`).

## Integration in the Suite

This package is part of the Wexample Suite — a collection of high-quality, modular tools designed to work seamlessly together across multiple languages and environments.

### Related Packages

The suite includes packages for configuration management, file handling, prompts, and more. Each package can be used independently or as part of the integrated suite.

Visit the [Wexample Suite documentation](https://docs.wexample.com) for the complete package ecosystem.

## Dependencies

- wexample-wex-addon-app: >=30.0.0
- wexample-wex-core: >=30.0.0

## Versioning & Compatibility Policy

Wexample packages follow **Semantic Versioning** (SemVer):

- **MAJOR**: Breaking changes
- **MINOR**: New features, backward compatible
- **PATCH**: Bug fixes, backward compatible

We maintain backward compatibility within major versions and provide clear migration guides for breaking changes.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

Free to use in both personal and commercial projects.

## About us

[Wexample](https://wexample.com) stands as a cornerstone of the digital ecosystem — a collective of seasoned engineers, researchers, and creators driven by a relentless pursuit of technological excellence. More than a media platform, it has grown into a vibrant community where innovation meets craftsmanship, and where every line of code reflects a commitment to clarity, durability, and shared intelligence.

This packages suite embodies this spirit. Trusted by professionals and enthusiasts alike, it delivers a consistent, high-quality foundation for modern development — open, elegant, and battle-tested. Its reputation is built on years of collaboration, refinement, and rigorous attention to detail, making it a natural choice for those who demand both robustness and beauty in their tools.

Wexample cultivates a culture of mastery. Each package, each contribution carries the mark of a community that values precision, ethics, and innovation — a community proud to shape the future of digital craftsmanship.

## Known Limitations & Roadmap

Current limitations and planned features are tracked in the GitHub issues.

See the [project roadmap](https://github.com/wexample/python-wex_addon_services_db/issues) for upcoming features and improvements.

## Status & Compatibility

**Maturity**: Production-ready

**Python Support**: >=3.10

**OS Support**: Linux, macOS, Windows

**Status**: Actively maintained

## Useful Links

- **Homepage**: https://github.com/wexample/python-wex-addon-services-db
- **Documentation**: [docs.wexample.com](https://docs.wexample.com)
- **Issue Tracker**: https://github.com/wexample/python-wex-addon-services-db/issues
- **Discussions**: https://github.com/wexample/python-wex-addon-services-db/discussions
- **PyPI**: [pypi.org/project/wexample-wex-addon-services-db](https://pypi.org/project/wexample-wex-addon-services-db/)

## Migration Notes

When upgrading between major versions, refer to the migration guides in the documentation.

Breaking changes are clearly documented with upgrade paths and examples.
