Metadata-Version: 2.4
Name: gl-observability-binary
Version: 0.2.8.post1
Summary: SDK for Observability tools
Author-email: HansSeanNathanael <hans.s.nathanael@gdplabs.id>
Requires-Python: <3.14,>=3.11
Description-Content-Type: text/markdown
Requires-Dist: sentry-sdk<3.0.0,>=2.20.0
Requires-Dist: opentelemetry-api<2.0.0,>=1.11.0
Requires-Dist: opentelemetry-sdk<2.0.0,>=1.11.0
Requires-Dist: opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.11.0
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc<2.0.0,>=1.11.0
Requires-Dist: opentelemetry-instrumentation-logging<=0.62b1
Requires-Dist: opentelemetry-instrumentation-fastapi<=0.62b1
Requires-Dist: opentelemetry-instrumentation-langchain<=0.61
Requires-Dist: opentelemetry-instrumentation-requests<=0.62b1
Requires-Dist: opentelemetry-instrumentation-httpx<=0.62b1
Requires-Dist: opentelemetry-instrumentation-openai<=0.61
Requires-Dist: opentelemetry-instrumentation-anthropic<=0.61,>=0.53.2
Requires-Dist: opentelemetry-instrumentation-bedrock<=0.61
Requires-Dist: opentelemetry-instrumentation-google-generativeai<=0.61
Requires-Dist: wrapt<2.0.0,>=1.0.0
Requires-Dist: packaging<26.0,>=23.0
Requires-Dist: typing-extensions<5.0.0,>=4.5.0
Provides-Extra: fastapi
Requires-Dist: opentelemetry-instrumentation-fastapi[instruments]<=0.62b1; extra == "fastapi"
Provides-Extra: langchain
Requires-Dist: opentelemetry-instrumentation-langchain[instruments]<=0.61; extra == "langchain"
Provides-Extra: requests
Requires-Dist: opentelemetry-instrumentation-requests[instruments]<=0.62b1; extra == "requests"
Provides-Extra: httpx
Requires-Dist: opentelemetry-instrumentation-httpx[instruments]<=0.62b1; extra == "httpx"
Provides-Extra: llm
Requires-Dist: opentelemetry-instrumentation-openai[instruments]<=0.61; extra == "llm"
Requires-Dist: opentelemetry-instrumentation-anthropic[instruments]<=0.61,>=0.53.2; extra == "llm"
Requires-Dist: opentelemetry-instrumentation-bedrock[instruments]<=0.61; extra == "llm"
Requires-Dist: opentelemetry-instrumentation-google-generativeai[instruments]<=0.61; extra == "llm"
Provides-Extra: all
Requires-Dist: opentelemetry-instrumentation-fastapi[instruments]<=0.62b1; extra == "all"
Requires-Dist: opentelemetry-instrumentation-langchain[instruments]<=0.61; extra == "all"
Requires-Dist: opentelemetry-instrumentation-requests[instruments]<=0.62b1; extra == "all"
Requires-Dist: opentelemetry-instrumentation-httpx[instruments]<=0.62b1; extra == "all"
Requires-Dist: opentelemetry-instrumentation-openai[instruments]<=0.61; extra == "all"
Requires-Dist: opentelemetry-instrumentation-anthropic[instruments]<=0.61,>=0.53.2; extra == "all"
Requires-Dist: opentelemetry-instrumentation-bedrock[instruments]<=0.61; extra == "all"
Requires-Dist: opentelemetry-instrumentation-google-generativeai[instruments]<=0.61; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest<10.0.0,>=9.0.0; extra == "dev"
Requires-Dist: pre-commit<4.0.0,>=3.7.0; extra == "dev"
Requires-Dist: pytest-cov<6.0.0,>=5.0.0; extra == "dev"
Requires-Dist: pytest-asyncio<2.0.0,>=1.0.0; extra == "dev"
Requires-Dist: pytest-mock<4.0.0,>=3.14.0; extra == "dev"
Requires-Dist: coverage<8.0.0,>=7.6.10; extra == "dev"
Requires-Dist: mypy<2.0.0,>=1.11.2; extra == "dev"
Requires-Dist: ruff<1.0.0,>=0.11.12; extra == "dev"

# GL Observability

`gl-observability` is a comprehensive SDK for implementing observability in Python applications. It provides easy-to-use wrappers for OpenTelemetry, Sentry, and custom logging handlers with PII redaction capabilities.

## Key Features

- 📊 **OpenTelemetry Integration**: simplified initialization for tracing.
- 🛡️ **Sentry Support**: easy setup for error tracking and performance monitoring.
- 🕵️ **PII Redaction**: custom logging handlers to redact PII using Regex or NER (Named Entity Recognition).
- 🔌 **Framework Support**: built-in support for FastAPI, Langchain, HTTPX, Requests, OpenAI, Anthropic, Bedrock, and Google Generative AI instrumentation.

## Installation

### Prerequisites

- Python 3.11-3.13 - [Install here](https://www.python.org/downloads/)
- Pip (if using pip) - [Install here](https://pip.pypa.io/en/stable/installation/)
- Poetry 2.1.3+ (if using Poetry) - [Install here](https://python-poetry.org/docs/#installation)
- uv (if using uv) - [Install here](https://docs.astral.sh/uv/guides/install-python/)
- Git (if using Git) - [Install here](https://git-scm.com/downloads)
- For git installation, access to the [GDP Labs SDK github repository](https://github.com/GDP-ADMIN/gl-sdk)

### 1. Installation from Pypi

Choose one of the following methods to install the package:

#### Using pip

```bash
pip install gl-observability-binary[all]
```

#### Using Poetry

```bash
poetry add gl-observability-binary[all]
```

#### Using uv

```bash
uv add gl-observability-binary[all]
```

### 2. Development Installation (Git)

For development purposes, you can install directly from the Git repository:

```bash
poetry add "git+ssh://git@github.com/GDP-ADMIN/gl-sdk.git#subdirectory=libs/gl-observability" --extras all
```

### Optional Dependencies

The OTel instrumentor packages for FastAPI, Langchain, HTTPX, Requests, OpenAI, Anthropic, Bedrock, and Google Generative AI are bundled in core. HTTPX, Requests, and all LLM providers (OpenAI, Anthropic, Bedrock, Google Generative AI) are enabled by default — `use_httpx` and `use_requests` default to `True`, and `use_llm` (which covers all four LLM providers together) defaults to `True`. Set any flag to `False` to disable that integration. Langchain and FastAPI follow the same `use_*` pattern (`use_langchain` defaults to `False`; FastAPI runs when `fastapi_config` is set).

Each instrumentor must run against a compatible version of the library it traces. If you hit a `DependencyConflictError` at startup, it means your installed library version falls outside the range the bundled instrumentor supports.

The optional extras solve this by pulling in the instrumented library at a version that is known to be compatible with the bundled instrumentor. Use them as a version resolver, not as a feature flag.

| Extra            | Installs                                                              |
| ---------------- | --------------------------------------------------------------------- |
| `fastapi`        | `fastapi` at a compatible version                                     |
| `langchain`      | `langchain` at a compatible version                                   |
| `httpx`          | `httpx` at a compatible version                                       |
| `requests`       | `requests` at a compatible version                                    |
| `llm`            | `openai`, `anthropic`, `boto3`, `google-generativeai` at compatible versions |
| `all`            | All libraries above                                                   |

Install a specific extra via `gl-observability-binary`:

```bash
# pip
pip install "gl-observability-binary[langchain]"

# Poetry
poetry add "gl-observability-binary[langchain]"

# uv
uv add "gl-observability-binary[langchain]"
```

## Usage

### 1. Telemetry Initialization

The library uses a unified `init_telemetry` function that takes a `TelemetryConfig` object. You can configure it to send traces to OpenTelemetry (OTLP) or Sentry backend. Multiple backend configuration is supported.

#### OpenTelemetry Configuration

This setup sends traces to an external OTLP collector (e.g., Jaeger, Tempo).

```python
from fastapi import FastAPI
from gl_observability import init_telemetry, TelemetryConfig, OpenTelemetryBackendConfig, FastAPIConfig

# 1. Setup FastAPI Config (optional, if using FastAPI)
app = FastAPI()
fastapi_config = FastAPIConfig(app=app)

# 2. Configure OpenTelemetryBackendConfig
otel_backend_config = OpenTelemetryBackendConfig(
    endpoint="localhost:4318",                  # OTLP endpoint
    use_grpc=False,                             # Use gRPC or HTTP
    headers={"Authorization": "Bearer ..."},    # Optional headers
)

# 3. Configure TelemetryConfig
otel_config = TelemetryConfig(
    attributes={"service.name": "..."},         # Resource attributes
    backend_config=otel_backend_config,         # Backend configuration
    fastapi_config=fastapi_config,              # FastAPI Instrumentation
    use_langchain=True,                         # Enable Langchain instrumentation (default: False)
    # use_httpx, use_requests, and use_llm default to True; set to False to disable.
)

# 4. Initialize Telemetry
init_telemetry(otel_config)
```

#### Sentry Configuration

This setup sends errors and traces to Sentry.

```python
from fastapi import FastAPI
from gl_observability import init_telemetry, TelemetryConfig, SentryBackendConfig, FastAPIConfig

# 1. Setup FastAPI Config (optional, if using FastAPI)
app = FastAPI()
fastapi_config = FastAPIConfig(app=app)

# 2. Configure SentryBackendConfig
sentry_backend_config = SentryBackendConfig(
    dsn="https://...",
    environment="...",
    release="...",
    send_default_pii=True,
    disable_sentry_distributed_tracing=False
)

# 3. Configure TelemetryConfig
otel_config = TelemetryConfig(
    attributes={"service.name": "..."},         # Resource attributes
    backend_config=sentry_backend_config,       # Backend configuration
    fastapi_config=fastapi_config,              # FastAPI Instrumentation
    use_langchain=True,                         # Enable Langchain instrumentation (default: False)
    # use_httpx, use_requests, and use_llm default to True; set to False to disable.
)

# 4. Initialize Telemetry
init_telemetry(otel_config)
```

#### Multiple Backend Configuration

This setup the OpenTelemetry SDK used for tracing.

```python
from fastapi import FastAPI
from gl_observability import init_telemetry, TelemetryConfig, OpenTelemetryBackendConfig, SentryBackendConfig

jaeger_backend = OpenTelemetryBackendConfig(endpoint="jager...", ...)
init_telemetry(
    TelemetryConfig(
        attributes={"service.name": "..."},
        backend_config=jaeger_backend,
        fastapi_config=fastapi_config,
        use_langchain=True,
        use_httpx=True,
        use_requests=True,
        use_llm=True,
    )
)

langfuse_backend = OpenTelemetryBackendConfig(endpoint="langfuse...", ...)
init_telemetry(
    TelemetryConfig(
        backend_config=langfuse_backend
    )
)

sentry_backend = SentryBackendConfig(dsn="https://...", ...)
init_telemetry(
    TelemetryConfig(
        backend_config=sentry_backend
    )
)
```

#### Masking policy and failure reporting

Import the public masking API from `gl_observability`:

| Purpose | Public names |
| --- | --- |
| Configure scrubbing | `MaskingConfig`, `MaskingRule`, `BuiltInMaskingRule`, `DenyFields`, `MaskingMode` |
| Consume failure reports | `MaskingReport`, `MaskingFailureGroup`, `MaskingPatternSource`, `OnReportHook` |
| Reuse default reporting | `masking_config_default_on_report` |
| Reuse policy defaults | `DEFAULT_RULES`, `DEFAULT_MASKING_CONFIG`, `EMPTY_DENY_FIELDS` |

```python
from gl_observability import (
    DEFAULT_RULES,
    EMPTY_DENY_FIELDS,
    BuiltInMaskingRule,
    MaskingConfig,
    MaskingMode,
    MaskingRule,
)

masking_config = MaskingConfig(
    rules=DEFAULT_RULES + (MaskingRule(name="CUSTOM_ID", pattern=r"customer-\d+"),),
    deny_fields=EMPTY_DENY_FIELDS,
    mode=MaskingMode.FAIL_CLOSED,
)
# Select individual built-ins directly; an explicit tuple replaces the defaults.
email_only = MaskingConfig(rules=(BuiltInMaskingRule.EMAIL,))
# Pass masking_config to TelemetryConfig(masking_config=masking_config, ...).
```

`BuiltInMaskingRule.PRIVATE_KEY_BLOCK` masks from a recognized `BEGIN … PRIVATE KEY`
header through the first `END … PRIVATE KEY` delimiter, regardless of label. If no
closing delimiter exists, it masks the rest of that string, including any trailing
text or serialized JSON punctuation. Text after a complete block is preserved.

Masking is enabled by default. `TelemetryConfig(masking_config=None)` disables it.
A policy with both `rules=()` and `deny_fields.fields=()` also skips compilation and
installs no masking wrapper or stage: no masking traversal, reports, or `FAIL_OPEN`
warning occurs. Rule-only and deny-only policies remain active, including a deny
pattern of `""`, which matches every attribute key.

OTLP masks span names and span attributes, including string sequences. Sentry masks
child-span `data`, `op`, and `description`; root `contexts.trace.data`, `op`, and
`description`; root `contexts.otel.attributes`; the `transaction` name and its
`contexts.trace.dynamic_sampling_context.transaction` copy; and
recursive string leaves in transaction `request` and `user` regions. Names,
operations, and descriptions use the configured value rules only, without
attribute-key denial. SQL and URL text copied into descriptions is masked too.
The sampling transaction name is scrubbed before Sentry moves it into the envelope
header; other sampling metadata remains unchanged.

Both backends preserve resources. OTLP events, links, status and instrumentation
metadata, and Sentry other contexts, tags, extra, breadcrumbs and error events are
outside this masking stage. Independent SDK scrubbing and consumer hooks may
still change those fields.

**Behavior change:** Matching text in OTel span names and Sentry operations and
transaction names is now redacted; Sentry description masking is retained.
For example, both `data["db.statement"]` and its copied description become
`SELECT * FROM users WHERE email = '[EMAIL]'`. An HTTP description becomes
`GET api.internal /reset?[GENERIC_CREDENTIAL_KV]` when its target contains
`token=abc123456789`. Ordinary operation labels remain unchanged, but matching
names/operations may change dashboard grouping. These fields use the same
`FAIL_CLOSED` / `FAIL_OPEN` behavior as other covered values.

Sentry resources previously masked by this component now pass through unchanged.
Applications relying on resource masking can sanitize `contexts.otel.resource`
in a consumer `before_send_transaction` hook supplied through `SentryBackendConfig`.
The hook runs after this component's masking stage. Regex rules still have false
positives and false negatives; this coverage does not guarantee removal of every
sensitive value.

Deny fields apply only to attribute keys, before value rules, using Python
`re.match` prefix semantics. Matching is case-sensitive unless flags are supplied:
`email` does not deny `user.email`, while `user\.email$` does. Denied values keep the
configured deny mask without value scanning. Request/user keys are never deny-checked.

`FAIL_CLOSED` replaces a value whose scrubbing raises with `[Dropped]`. `FAIL_OPEN`
returns that value unmasked and is intended for non-production debugging. Both modes
continue scrubbing the other values normally. `EMPTY_DENY_FIELDS` disables key-based
denial; value rules still apply.

`on_report` receives one `MaskingReport` per item with masking failures. It defaults to
`masking_config_default_on_report`; a custom hook replaces it, and `None` disables reporting.
Hooks must return promptly. A hook exception is caught and diagnostic logging is attempted;
if diagnostic logging raises, that exception may propagate. During context-manager exit,
it may replace a traversal exception, which remains in the exception context chain.
Nested report delivery on the same thread is skipped while a hook runs; masking still applies
and later independent reports can be delivered.

Compiled policies, `MaskingSession`, `MaskingPatternError`, `DROPPED_VALUE`, exporters,
and chain/traversal helpers are internal implementation details. The supported masking API consists of the package-root
imports listed above; deep imports of implementation details have no compatibility guarantee.

### 2. Logging Handlers

The library provides logging handlers to automatically redact Personally Identifiable Information (PII) from logs.

#### Regex-based PII Redaction

Uses regular expressions to mask common PII patterns like KTP, NPWP, Phone Numbers, and Email.

```python
import logging
from gl_observability.logs.regex_pii_logger_handler import init_regex_pii_logging_handler

# Initialize the handler for a specific logger
init_regex_pii_logging_handler(
    logger_name="my_application_logger",
    pii_regex_process_enabled=True
)

logger = logging.getLogger("my_application_logger")
logger.info("User email is john.doe@example.com and phone is 08123456789")
# Output: User email is jo******om and phone is 0812******6789
```

#### NER-based PII Redaction (Named Entity Recognition)

Uses an external API to perform Named Entity Recognition for more advanced PII detection and redaction.

> [!WARNING]
> The NER logging handler makes synchronous API calls for each log record, which may impact performance.

```python
import logging
from gl_observability.logs.ner_pii_logger_handler import init_ner_pii_logging_handler

# Initialize the handler
init_ner_pii_logging_handler(
    logger_name="my_application_logger",
    api_url="https://your-ner-api.com/anonymize",
    api_field="text",  # The field name in API response containing the redacted text
    pii_ner_process_enabled=True
)

logger = logging.getLogger("my_application_logger")
logger.info("My KTP is 3525011212941001")
# Output will be redacted based on API response
```
