Metadata-Version: 2.4
Name: fileroute
Version: 0.2.5.dev1
Summary: Convenient connectors to make interoperating and collaborating across multiple content management systems easier
License-Expression: MIT
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.10.6
Requires-Dist: pydantic-settings>=2.7.0
Requires-Dist: ruamel.yaml<0.20,>=0.19.1
Requires-Dist: requests>=2.32.3
Requires-Dist: python-dotenv>=1.0.1
Requires-Dist: typer>=0.24.1
Requires-Dist: boto3>=1.42.55
Requires-Dist: google-auth-oauthlib>=1.2.1
Requires-Dist: google-auth>=2.0
Requires-Dist: rich>=13.0
Requires-Dist: msal>=1.33.0
Provides-Extra: docs
Requires-Dist: mkdocs>=1.6.1; extra == "docs"
Requires-Dist: mkdocs-material>=9.7.1; extra == "docs"
Dynamic: license-file

# fileroute

Fileroute records where project artifacts come from, where they live locally,
and where they should be published. A YAML or JSON descriptor connects local
files to SharePoint, Google Drive, and S3 locations. Preview transfers without
credentials, visualize the relationships, pull remote inputs, and publish to
supported targets through the CLI or Python API. Python 3.11+ is required.

**Current transfer support:** SharePoint, Google Drive, and S3 can be pulled;
SharePoint and Google Drive can be pushed. S3 targets can be described and
diagrammed, but `push --dry-run` rejects them until upload support exists.
Fileroute does not transform files or transfer directly between cloud providers.

## Descriptor format change

Catalogs and resources are now keyed maps: `resources: {report: {path: report.csv}}`.
The map key is the registered name. Cross-file catalogs use
`catalogs: {archive: {descriptor: catalogs/archive.yaml}}`.
Run `fileroute migrate OLD_DESCRIPTOR NEW_DIRECTORY --dry-run` before
converting existing named lists and `$ref` links. See the
[migration guide](docs/descriptors.md#migrate-the-old-format).

## Get started

For a repeatable project workflow, add Fileroute as a dependency and commit the
descriptor and `uv.lock`:

```bash
uv add fileroute
uv run fileroute diagram config/fileroute.yaml
uv run fileroute pull config/fileroute.yaml --dry-run
```

For occasional CLI use, run the published tool in a separate environment:

```bash
uvx fileroute diagram config/fileroute.yaml
uvx fileroute pull config/fileroute.yaml --dry-run
```

`uv run` uses the project's dependencies and supports Python imports;
`uvx` does not install Fileroute into the project. Pin a version with
`uv add 'fileroute==X.Y.Z'` or
`uvx --from 'fileroute==X.Y.Z' fileroute --help`.
See uv's [project](https://docs.astral.sh/uv/concepts/projects/run/),
[dependency](https://docs.astral.sh/uv/concepts/projects/dependencies/), and
[tool](https://docs.astral.sh/uv/concepts/tools/) guides.

One supported workflow downloads a selected SharePoint file to a local artifact
and publishes it to two SharePoint destinations:

```yaml
resources:
  monthly-report:
    path: artifacts/monthly-report.csv
    sources:
      - path: https://contoso.sharepoint.com/sites/data/Shared%20Documents/monthly-report.csv
    targets:
      - path: https://contoso.sharepoint.com/sites/reports/Shared%20Documents/monthly-report.csv
      - path: https://contoso.sharepoint.com/sites/archive/Shared%20Documents/monthly-report.csv
  annual-report:
    path: artifacts/annual-report.csv
    sources:
      - path: https://contoso.sharepoint.com/sites/data/Shared%20Documents/annual-report.csv
    targets:
      - path: https://contoso.sharepoint.com/sites/reports/Shared%20Documents/annual-report.csv
```

Save this as `config/fileroute.yaml`, replace the example URLs, and run:

```bash
uv run fileroute resolve config/fileroute.yaml
uv run fileroute diagram config/fileroute.yaml
uv run fileroute list config/fileroute.yaml
uv run fileroute pull config/fileroute.yaml --select monthly-report --dry-run
uv run fileroute pull config/fileroute.yaml --select monthly-report
uv run fileroute push config/fileroute.yaml --select monthly-report --dry-run
uv run fileroute push config/fileroute.yaml --select monthly-report
```

Omit `--select` to process all eligible resources; selecting a catalog includes
its descendants. A selected pull uses its remote source, while a selected push
publishes to all effective targets.

`resolve` parses URLs and scoped paths offline; `resolve --online --write`
verifies remote locations and saves their IDs and entity types. `diagram` and
dry runs also work without provider access; `--online` and actual transfers
require credentials. Configure credentials using
[.env-sample](.env-sample); see [Authentication](docs/authentication.md) for
provider setup.

To publish a new file to Google Drive, target an existing folder; Fileroute
creates or replaces the file below it. An exact file URL instead replaces that
file by ID and preserves its existing name:

```yaml
resources:
  report:
    path: artifacts/report.csv
    targets:
      - path: https://drive.google.com/drive/folders/FOLDER_ID
      - path: https://drive.google.com/file/d/EXISTING_FILE_ID/view
```

The two targets receive separate copies. See [Transfers](docs/transfers.md)
for nested folders, shared drives, and ambiguous names.

## Documentation

- [Descriptor model, paths, references, and resolution](docs/descriptors.md)
- [More use cases, saved YAML, and generated diagrams](docs/use-cases.md)
- [Transfer behavior and provider support](docs/transfers.md)
- [Diagram formats and Python graph API](docs/diagram.md)
- [CLI reference](docs/cli.md) and [Python API](docs/api.md)
- [Development, migration, and package releases](docs/contributing.md)

The descriptor `path` is a local artifact for transfers. `sources` are
upstream inputs or provenance; `targets` are publication destinations.
For nested edits, `fileroute list` shows exact JSONPath selectors; the leading
`$` is optional when passing one to `list`, `update`, or `add --parent`.
Diagrams show intent, not a completed transfer. Use `fileroute --help` for
commands and options. To develop this repository, run `uv sync` and see the
[contributor guide](docs/contributing.md).

### HTTP retries and upload recovery

Fileroute retries transient Microsoft Graph reads and content PUTs up to three times,
respecting `Retry-After` when supplied. A content PUT reopens the local file on
each attempt. Folder-creation POSTs are not automatically replayed after an
uncertain result. Google Drive resumable uploads query the upload session after
a transient or rate-limit error and continue from the byte offset confirmed by
the server. Network and HTTP failures retain their provider-specific exception
types and expose `status_code`, `response_text`, `response_json`, and
`response_headers` for callers that need details.

## Root catalogs and profile schemas

Use `catalog.yaml` (also `.yml` or `.json`) as a project resource inventory.
Discovery checks an explicit argument, then the activated descriptor, then a root
catalog, then `resources/descriptor.*`. A stale activation remains an error;
Fileroute does not search parent directories.

`Catalog.resourcePathTemplate` describes an artifact naming pattern relative to `path`:

```yaml
profile: fileroute-catalog
targets: []
catalogs:
  surveys:
    path: data
    resourcePathTemplate: "{surveyid}/{env}/v{version}/schema.json"
```

Applications interpret the pattern. Transfers do not expand placeholders,
discover matching files, or replace explicit resource paths. Select publication
scope explicitly, for example `fileroute push catalog.yaml --select documentation`.

`poe schema-export` generates `dist/schemas/fileroute-catalog.schema.json` from
the models. Stable releases attach that exact version's schema as a GitHub asset.
Associate YAML with a released schema using this editor comment (replace VERSION):

```yaml
# yaml-language-server: $schema=https://github.com/mbkranz/fileroute/releases/download/vVERSION/fileroute-catalog.schema.json
profile: fileroute-catalog
```

Keep `profile` as document data; `$schema` is only in the editor comment.
Schema validation covers structural constraints. Runtime checks still validate
case-insensitive name collisions, descriptor paths and cycles, filesystem state,
and operational transfer requirements. Service aliases normalized at runtime may
need canonical spelling for editor validation.

Release preparation saves schemas under `dist/schemas/` alongside distributions;
PyPI receives only distributions. Retries reuse the saved release artifact and
refuse to replace an existing GitHub schema asset with different bytes.

## License

Fileroute's code and generated profile schemas are covered by the MIT license
in `LICENSE`. Resources referenced by user catalogs retain their own terms.
