Metadata-Version: 2.4
Name: hyperprobe-agent
Version: 1.2.30b6
Summary: Production-grade, non-breaking live debugger and telemetry agent for Python.
Home-page: https://www.hyperprobe.co
Author: Arif, Saksham, Karan
Author-email: arif@hypertest.co, saksham@hypertest.co, karan@hypertest.co
License: Hyperprobe Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: protobuf<4.25,>=4.24.4; python_version < "3.8"
Requires-Dist: protobuf<6.0.0,>=4.25.3; python_version >= "3.8"
Requires-Dist: grpcio<1.63,>=1.62.2; python_version < "3.8"
Requires-Dist: grpcio>=1.62.2; python_version >= "3.8"
Requires-Dist: importlib-metadata<7,>=6; python_version < "3.8"
Requires-Dist: bytecode==0.13.0; python_version < "3.8"
Requires-Dist: bytecode==0.16.2; python_version >= "3.8" and python_version < "3.12"
Provides-Extra: dev
Requires-Dist: grpcio>=1.50.0; extra == "dev"
Requires-Dist: grpcio-tools>=1.50.0; extra == "dev"
Requires-Dist: protobuf>=4.21.0; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# HyperProbe Python Agent

Production-grade, non-breaking live debugger and telemetry agent for Python.

HyperProbe captures stack frames, local variables, and logs at selected source
lines without stopping the application in a debugger. At agent initialization,
Python 3.12+ selects the `sys.monitoring` backend. CPython 3.7–3.11 selects the
bytecode backend. Both support snapshot, log, counter, metric, and duration
probes and use the same broker, evaluator, serializer, quotas, safety monitor,
and output pipeline.
Only the selected backend is imported; `bytecode` is not a dependency on 3.12+.

## Features

- **Non-Breaking Snapshot Probes:** Capture local variables and stack traces dynamically at any line of code.
- **Dynamic Log Probes:** Inject live log templates to print formatted output without redeploying code.
- **Counter Probes:** Emit a value of `1` whenever a configured source line is reached.
- **Metric Probes:** Evaluate a numerical Python expression and emit its value.
- **Duration Probes:** Measure elapsed monotonic time in milliseconds between two source lines, with optional request correlation.
- **Circular Reference & Deep Object Protection:** Memory-safe serialization of deeply nested or cyclic variables.
- **Safety Guard:** Suspends instrumentation when scheduler lag or the capture pause budget crosses its configured threshold.

## Bytecode backend (CPython 3.7–3.11)

The broker still supplies `runtime_location` and `runtime_line`. The backend
uses the existing path resolver, locates functions through their loaded modules,
and replaces only the selected functions' code objects. It does not use
`sys.settrace`, `threading.settrace`, or a heap-wide `gc.get_objects()` scan.
Pending probes can watch standard file imports; the normal loader executes the
module after the SDK prepares its code. Loaded factories can register and
instrument nested functions they create later. Conditions, watches, redaction,
trace IDs, hit limits, and telemetry use the shared services. Duration probes
index both source locations, including an end line in a later-imported module.

This backend also selects its own agent lifecycle controller at startup; the
Python 3.12+ lifecycle and `sys.monitoring` implementation remain separate.
Broker responses are compared against stable local definition records. An
unchanged response (including reordered probes) does no module discovery,
function planning, or bytecode rewriting. Condition, watch, and template changes
replace evaluation state without rebuilding bytecode. Location changes discover
the new target and replan affected functions; removal uses tracked functions.
Callback-site tokens are held weakly, so short-lived nested functions do not
accumulate permanent lookup entries.
Broker-confirmed completed probes stay disabled until the broker removes them.

On RED, the backend closes its capture gate and synchronously restores the
saved original code for functions it still owns, including factories. Pending
import work cannot install new probes while suspended. Restoration runs before
agent bookkeeping or logging, and recovery builds bytecode without holding the
agent lock. Capture delivery checks the original definition identity and
activation generation before queue admission, so an old capture cannot be
accepted under an updated definition or after RED/recovery. A frame already running
keeps its existing code, but its callbacks become no-ops. An expression already
running cannot be forcibly interrupted safely. Instrumentation failures are
reported through bounded engine diagnostics and best-effort error logging;
normal exceptions from the application's own function or module still propagate.

Discovery supports module/class namespaces and bounded built-in containers.
Arbitrary generated code, custom loaders, hidden pre-existing closures, and
automatic reload coverage are not guaranteed. Function prologues are not probe
locations. Performance, including a 2% overhead target, requires workload-based
measurement; the shared evaluator is not a security sandbox.

## Metric-family probes

Counter, metric, and duration probes are created through HyperProbe in the same way as snapshot and log probes. No application code changes or additional public SDK calls are required.

| Probe | Configuration | Emitted `metric_value` |
| --- | --- | --- |
| Counter | `metric_name` and one runtime location | `1` per matching hit |
| Metric | `metric_name`, `metric_expression`, and one runtime location | The finite numerical expression result |
| Duration | `metric_name`, primary and secondary runtime locations, and optional `correlation_expression` | Monotonic elapsed milliseconds |

Conditions are evaluated at every configured location. Metric expressions use the same restricted Python evaluator as conditions and watches. Numerical strings follow Node SDK `parseFloat` prefix behavior for compatibility; invalid or non-finite values are exported with `metric_value` set to `0` and a populated `capture_error`. The bytecode backend examines at most the first 1,024 characters of a metric string.

Duration correlation expressions must evaluate to a string or number. When omitted or when they evaluate to `None`, a singleton correlation key is used. Unmatched duration ends are ignored, and pending starts expire after 60 seconds. On CPython 3.7–3.11, correlation keys are limited to 256 characters and at most 4,096 starts are kept per process; the oldest start is evicted if that limit is reached. Duration starts never consume a probe hit. A matching end consumes the hit when its event is admitted to the telemetry queue.

## Installation

Install the package via `pip` (or `testpypi` for testing):

```bash
pip install hyperprobe-agent
```

## Usage

Start the agent programmatically at your application's entrypoint:

```python
import os
from hyperprobe import HyperProbe

agent = HyperProbe.start({
    "service_id": "<service-uuid-from-dashboard>",
    "environment": os.getenv("PYTHON_ENV"),
    "broker_url": "https://logger.app.hyperprobe.co",
    "commit_sha": os.getenv("GIT_COMMIT"),
})
```

Required identity values may also be supplied through the environment variables
below. Explicit options take precedence. Initialization failures are reported
and the application continues without instrumentation.

### Configuration Environment Variables

Configure the agent using the following environment variables:

| Variable | Description | Default |
| --- | --- | --- |
| `HYPERPROBE_BROKER_URL` | The URL of the HyperProbe Telemetry Broker. | *Required* |
| `HYPERPROBE_SERVICE_ID` | UUIDv4 service identifier provided by the HyperProbe dashboard. | *Required* |
| `HYPERPROBE_ALLOW_NON_UUID` | Set to `yes` only to allow a legacy non-UUID service identifier. | *(unset)* |
| `HYPERPROBE_ENVIRONMENT` | Deployment environment name (e.g., `production`, `staging`). | *Required* |
| `HYPERPROBE_COMMIT_SHA` or `GIT_COMMIT` | Git commit SHA of the running application version. | *Required* |
| `HYPERPROBE_DISABLED` | Set to `YES` to explicitly disable the agent. | *(unset)* |
| `HYPERPROBE_SYNC_INTERVAL_MS` | Rate at which the agent syncs probe definitions from the broker. | `60000` (1 min) |
| `HYPERPROBE_FLUSH_INTERVAL_MS` | Delay before sending telemetry events to the broker. | `1000` (1 sec) |
| `HYPERPROBE_MAX_QUEUE_SIZE` | Maximum number of telemetry events held before flush. | `100` |
| `HYPERPROBE_HITS_PER_SEC` | Max quota-charged probe evaluations per second. | `10` |
| `HYPERPROBE_BANDWIDTH_KB_PER_SEC` | Max bandwidth limit for telemetry transmissions (quota). | `1024` (1 MB) |
| `HYPERPROBE_RPC_TIMEOUT_SEC` | Deadline for broker RPCs. | `10` |
| `HYPERPROBE_COOLDOWN_SEC` | Tracing suspension duration after a RED safety state. | `10` |
| `HYPERPROBE_MAX_LAG_MS` | Per-reading scheduler-lag threshold. YELLOW requires 4/10 breaches; RED requires 7/10 breaches or 3/5 readings above 4x the threshold. | `50` |
| `HYPERPROBE_PAUSE_BUDGET_MS` | Per-second capture pause budget for the safety monitor. | `15` |
| `HYPERPROBE_REDACT_KEYS` | Comma-separated key patterns to redact. | `password,secret,token,authorization,cookie,key,signature` |
| `HYPERPROBE_REDACT_VALUES` | Comma-separated value patterns to redact. | *(empty)* |
| `HYPERPROBE_MAX_OBJECT_DEPTH` | Maximum serialized object depth. | `3` |
| `HYPERPROBE_MAX_ARRAY_LENGTH` | Maximum serialized array length. | `3` |
| `HYPERPROBE_STACK_FRAME_DEPTH` | Maximum captured stack-frame depth. | `3` |
| `HYPERPROBE_MAX_OBJECT_PROPERTIES` | Maximum serialized properties per object. | `50` |
| `HYPERPROBE_MAX_STRING_LENGTH` | Maximum serialized string length. | `1024` |
| `HYPERPROBE_DISABLE_SAFE_EVALUATION` | Set to `true` to explicitly disable safe evaluation restrictions and allow arbitrary method/function calls in probe expressions. | `false` |
| `HYPERPROBE_FORK_MODE` | Set to `worker` for preloaded, prefork servers. The parent defers agent initialization and every worker starts fresh process-local state. | `none` |

Programmatic integrations can set `allow_non_uuid=True` for the same legacy
bypass. Without an explicit bypass, a non-UUIDv4 service ID prevents agent
startup while the host application continues normally.

### Safe Evaluation & Security Considerations

Captured strings are checked against configured value-redaction patterns before
truncation. When value rules are configured, a string longer than 16,384
characters is masked rather than scanned partially. Log substitutions use the
bounded serializer, so nested sensitive keys are masked before an object becomes
log text. Telemetry errors carry categories rather than raw runtime values or
exception messages. These rules do not infer the origin of an arbitrary watch
result: a scalar watch expression such as `password` can still expose its value
when no value pattern matches it.

By default, **safe evaluation is enabled** (`HYPERPROBE_DISABLE_SAFE_EVALUATION=false`).
The existing evaluator checks function and method names against an allowlist.
This reduces available operations but is not a sandbox: properties, overloaded
operators, or allowed method names on custom objects can still execute
application code. There is no hard timeout that can interrupt such code safely.

**Disabling Safe Evaluation (`HYPERPROBE_DISABLE_SAFE_EVALUATION=true`):**
> ⚠️ **Security Warning**: Setting `HYPERPROBE_DISABLE_SAFE_EVALUATION=true` allows conditions, watch expressions, and log templates to execute arbitrary functions and methods in the host application process. This may lead to unintended side effects, state mutations, performance degradation, or security risks if expressions invoke mutating or blocking operations. Only disable safe evaluation in trusted environments where dynamic method evaluation is strictly required.

### Debug logging

Set `DEBUG` to a comma- or space-separated list of namespaces. Patterns accept
`*`, and exclusions begin with `-`:

```bash
DEBUG=hyperprobe:*                               # all HyperProbe SDK logs
DEBUG=hyperprobe:broker,hyperprobe:monitor      # selected components
DEBUG=hyperprobe:*,-hyperprobe:evaluator         # exclude a component
DEBUG_COLORS=1                                  # force ANSI colors
```

Without `*`, a selector is exact: `DEBUG=hyperprobe` does not match
`hyperprobe:agent`. Use `DEBUG=hyperprobe*` to enable every component.
Set `DEBUG_COLORS=0` to disable colors. Logging configuration is read during
SDK initialization, so set these variables before starting the process.

## Shutdown

Call `HyperProbe.shutdown()` when the application process stops to close
monitoring, workers, and the broker connection cleanly.

For servers that import the application before forking workers, such as
Gunicorn with `--preload`, enable worker fork mode:

```python
agent = HyperProbe.start({
    "service_id": "<service-uuid-from-dashboard>",
    "environment": os.getenv("PYTHON_ENV"),
    "broker_url": "https://logger.app.hyperprobe.co",
    "commit_sha": os.getenv("GIT_COMMIT"),
    "fork_mode": "worker",
})
```

The prefork parent remains agent-free. Every worker receives a unique agent ID,
telemetry queue, gRPC channel, monitoring engine, and set of background threads.
Application code running inside those workers must use the `spawn` process start
method rather than creating additional children with `fork`.
