Metadata-Version: 2.5
Name: evidentia-collectors
Version: 0.13.0
Summary: Evidence collection agents for AWS, Databricks, GitHub, Okta, Google Workspace, Snowflake, SQL databases, and vendor-risk APIs (Vanta, Drata, BitSight, SecurityScorecard)
Project-URL: Homepage, https://github.com/polycentric-labs/evidentia
Project-URL: Repository, https://github.com/polycentric-labs/evidentia
Project-URL: Issues, https://github.com/polycentric-labs/evidentia/issues
Project-URL: Changelog, https://github.com/polycentric-labs/evidentia/blob/main/CHANGELOG.md
Author-email: Allen Byrd <allen@allenfbyrd.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: aws,bitsight,compliance,databricks,drata,evidence-collection,github,google-workspace,grc,okta,securityscorecard,snowflake,sql,tprm,vanta,vendor-risk
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: System :: Systems Administration
Classifier: Typing :: Typed
Requires-Python: <3.15,>=3.12
Requires-Dist: evidentia-core<0.14.0,>=0.13.0
Requires-Dist: httpcore==1.0.9
Requires-Dist: httpx>=0.27
Provides-Extra: all
Requires-Dist: boto3>=1.43.46; extra == 'all'
Requires-Dist: botocore==1.43.98; extra == 'all'
Requires-Dist: databricks-sdk>=0.30; extra == 'all'
Requires-Dist: defusedxml>=0.7.1; extra == 'all'
Requires-Dist: okta>=3.4.4; extra == 'all'
Requires-Dist: oracledb>=4.0.2; extra == 'all'
Requires-Dist: psycopg[binary]>=3.3.4; extra == 'all'
Requires-Dist: py-ocsf-models<0.11.0,>=0.9.0; extra == 'all'
Requires-Dist: pygithub>=2.4; extra == 'all'
Requires-Dist: pymysql>=1.1; extra == 'all'
Requires-Dist: pyodbc>=5.0; extra == 'all'
Requires-Dist: signxml==5.1.0; extra == 'all'
Requires-Dist: snowflake-connector-python>=3.10; extra == 'all'
Provides-Extra: aws
Requires-Dist: boto3>=1.43.46; extra == 'aws'
Provides-Extra: databricks
Requires-Dist: databricks-sdk>=0.30; extra == 'databricks'
Provides-Extra: github
Requires-Dist: pygithub>=2.4; extra == 'github'
Provides-Extra: ocsf
Requires-Dist: py-ocsf-models<0.11.0,>=0.9.0; extra == 'ocsf'
Provides-Extra: okta
Requires-Dist: okta>=3.4.4; extra == 'okta'
Provides-Extra: registries
Requires-Dist: defusedxml>=0.7.1; extra == 'registries'
Requires-Dist: signxml==5.1.0; extra == 'registries'
Provides-Extra: retention
Requires-Dist: botocore==1.43.98; extra == 'retention'
Requires-Dist: defusedxml>=0.7.1; extra == 'retention'
Provides-Extra: scan
Requires-Dist: defusedxml>=0.7.1; extra == 'scan'
Provides-Extra: snowflake
Requires-Dist: snowflake-connector-python>=3.10; extra == 'snowflake'
Provides-Extra: sql
Requires-Dist: oracledb>=4.0.2; extra == 'sql'
Requires-Dist: psycopg[binary]>=3.3.4; extra == 'sql'
Requires-Dist: pymysql>=1.1; extra == 'sql'
Requires-Dist: pyodbc>=5.0; extra == 'sql'
Provides-Extra: sql-mssql
Requires-Dist: pyodbc>=5.0; extra == 'sql-mssql'
Provides-Extra: sql-mysql
Requires-Dist: pymysql>=1.1; extra == 'sql-mysql'
Provides-Extra: sql-oracle
Requires-Dist: oracledb>=4.0.2; extra == 'sql-oracle'
Provides-Extra: sql-postgres
Requires-Dist: psycopg[binary]>=3.3.4; extra == 'sql-postgres'
Provides-Extra: sql-sqlite
Description-Content-Type: text/markdown

# evidentia-collectors

Evidence collectors for [Evidentia](https://github.com/Polycentric-Labs/evidentia), the open-source compliance-as-code engine. Collectors read source configuration and findings without changing the source system. Results include `SecurityFinding` records and a collection manifest; available control mappings and coverage depend on the collector. Configuration observations alone do not establish compliance.

## Collectors

| Collector | CLI leaf | Evidence | Extra |
|---|---|---|---|
| AWS | `collect aws` | Config rules, Security Hub findings, IAM Access Analyzer | `[aws]` |
| GitHub | `collect github` | Repository visibility, branch protection, CODEOWNERS, Dependabot alerts, OSPS Baseline helpers | `[github]` |
| Okta | `collect okta` | User inventory, inactive accounts, admin assignments, MFA enrollment, password and sign-on policies | none (`[okta]` adds the official SDK for your own code) |
| Google Workspace | `collect google-workspace` | Directory users, inactive accounts, admin accounts, 2-Step Verification enrollment, Reports login activity | none |
| Entra ID / Microsoft 365 | `collect entra-m365` | Conditional Access, authentication registration, sign-ins, activated roles, Intune devices, retention labels, supplied DLP configuration, Defender alerts and incidents | none |
| Enterprise retention | `collect enterprise-retention` | Selected Google Vault matters/holds, Splunk Enterprise index retention and Elasticsearch ILM configuration | none |
| Incident clock | `collect incident-clock` | One selected ServiceNow, Jira Cloud or PagerDuty record, declared workflow events and exact elapsed seconds | none |
| Release cadence | `collect release-cadence` | Public GitHub release facts, publication-time qualification, event identity and explicit local persistence | none |
| Public registries | `collect registry` | Eleven selectors for source facts, selected identities, traversal status and freshness | `[registries]` for InCommon XML verification; none for the other selectors |
| Storage retention | `collect retention` | Selected S3 Object Lock/versioning, Azure Blob policy/hold configuration with account/service context, and GCS bucket retention/versioning | `[retention]` for S3; none for Azure/GCS |
| PostgreSQL, MySQL, SQLite, MS SQL, Oracle | `collect sql` | User privileges, audit configuration, encryption posture | `[sql-postgres]`, `[sql-mysql]`, `[sql-sqlite]`, `[sql-mssql]`, `[sql-oracle]`, or `[sql]` for the whole family |
| Databricks | `collect databricks` | Token lifecycle, cluster configuration, service principals, secret scopes | `[databricks]` |
| Snowflake | `collect snowflake` | Login history, users and MFA, grants, network and masking policies | `[snowflake]` |
| Vanta, Drata, BitSight, SecurityScorecard | `collect vanta`, `collect drata`, `collect bitsight`, `collect securityscorecard` | Vendor inventory and security ratings | none |

Four importers read exports or feeds: `collect ocsf` for OCSF Compliance and Detection Findings (`[ocsf]`), `collect nessus` and `collect greenbone` for vulnerability-scan exports, and `collect scap` for selected XCCDF 1.2 or OVAL 5.8, 5.11.2 and 5.12.3 core results (`[scan]` for XML importers).

## Install

```bash
pip install evidentia-collectors            # the httpx-based collectors and importers
pip install 'evidentia-collectors[aws]'     # add one provider's driver
pip install 'evidentia-collectors[all]'     # every optional driver
```

Credentials come from environment references documented for each collector. Enterprise retention resolves the token reference in its authorized profile; its `--help` names the registry environment variable. No collector accepts a secret as a flag or in a request body. Outbound hosts pass an SSRF guard by default.

Entra/M365 returns a full result envelope with nine capability states, counts, source provenance and a manifest. Prefer `EntraM365Collector.collect_v2(request)` in Python; `collect(request)` returns findings only. Graph access uses pre-minted tokens from `ENTRA_M365_ACCESS_TOKEN` and a separate delegated `ENTRA_M365_RETENTION_ACCESS_TOKEN`. `ENTRA_M365_AUTH_MODE` declares the primary token mode. DLP-only ingestion needs no token. Partial or unavailable evidence is explicit and never proves tenant-wide compliance. Tests use labeled synthetic Graph responses and a selected recorded CISA DLP export; live-tenant acceptance remains unverified.

Operator guide: [Run evidence collectors](https://github.com/Polycentric-Labs/evidentia/blob/main/docs/wiki/2-guides/run-collectors.md).

Storage retention accepts a bounded JSON request selecting one provider and 1 through 20 resources. `StorageRetentionCollector.collect_v2(request)` and the CLI, API and console retain every resource and component, including partial and unavailable evidence. Native days, years, quoted seconds and source timestamp text remain distinct. Results describe configuration, with unknown compliance and no object-enforcement or recordset-completeness claim. Credentials use fixed server-side references, without SDK discovery or refresh. Tests use 24 labeled synthetic provider-response fixtures; live cloud acceptance is unverified. See the [storage design](https://github.com/Polycentric-Labs/evidentia/blob/main/docs/designs/storage-retention-collector-design.md).

Enterprise retention requires an administrator-configured profile and a separate explicit grant for each API principal or local CLI use. Full results preserve the unique read ledger, native values, partial/unavailable states and informational findings with unknown compliance. The console and API share the strict request and result contract; live-provider acceptance remains unverified. See the [enterprise design](https://github.com/Polycentric-Labs/evidentia/blob/main/docs/designs/enterprise-retention-collector-design.md) and the operator guide above.

Public registry requests select one typed target and an optional scope label. `RegistryCollector.collect_v2(request)` preserves the complete result; `collect(request)` returns findings only. TLS, RDAP, GLEIF and security.txt use fixed verified transport policies. SAM reads `EVIDENTIA_REGISTRY_SAM_API_KEY` on the server. FedRAMP, CMVP and FCC use packaged reviewed snapshots. InCommon requires `pip install 'evidentia-collectors[registries]'`; the GUI extra alone does not install its verifier. SSL Labs live access is disabled before all I/O. Source evidence does not establish eligibility, ownership, applicability or compliance. See the [registry design](https://github.com/Polycentric-Labs/evidentia/blob/main/docs/designs/registry-collector-design.md) for exact source, timestamp, trust and approved publisher-data preservation rules.

Incident clocks require a trusted profile with exact record, workflow and principal grants; local CLI permission is separate. `IncidentClockCollector.collect_v2(request)` uses an issued selection and returns the full source/clock result. Current ServiceNow date columns, Jira changelog transitions and PagerDuty log entries retain their native operational meanings. Complete, incomplete and unavailable sources remain distinct from computed, unresolved and reversed clocks. Tests and console examples are synthetic; live permissions, entitlement and workflow correctness require operator validation. See the [incident clock design](https://github.com/Polycentric-Labs/evidentia/blob/main/docs/designs/incident-clock-collector-design.md) and operator guide.

SCAP imports preserve the complete native graph, exact source hash, selected assessment and completion provenance. An artifact can be unavailable even when the full import succeeds. The CLI and console require explicit profile and assessment selection; import does not run a scanner or save evidence. Source URLs and embedded instructions remain inert. See the [SCAP operator guide](https://github.com/Polycentric-Labs/evidentia/blob/main/docs/scap-collectors.md) for completion assertions, finite limits, downloads and separate persistence.

Release cadence fixes the unauthenticated public GitHub Releases profile and returns a complete bounded `PollResult`. `collect_release_cadence(request)` is the Python entrypoint; `poll_release_bytes(request)` returns the accepted full JSON bytes. Polling does not open the evidence store unless `persist` is true. Core's `evaluate_release_series` reads saved publication families offline under an explicit window and spacing policy. Source observations preserve later fact changes without inflating publication counts. These results do not establish patch installation, remediation or compliance. See the [release-cadence guide](https://github.com/Polycentric-Labs/evidentia/blob/main/docs/wiki/2-guides/release-cadence.md) for authentication, source limits, storage outcomes and console downloads.

License: Apache 2.0
