Metadata-Version: 2.4
Name: fduplex-git-remote-s3
Version: 0.6.1
Summary: A git remote helper for Amazon S3
Author: FullDuplex Media
Author-email: FullDuplex Media <contact@fullduplex.media>
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Classifier: Topic :: Software Development :: Version Control
Classifier: Topic :: Software Development :: Version Control :: Git
Requires-Dist: boto3>=1.42.24,<2.0.0
Requires-Dist: botocore>=1.42.24,<2.0.0
Requires-Dist: urllib3>=2.7.0,<3.0.0
Requires-Dist: dnspython>=2.7.0,<3.0.0
Requires-Dist: aws-s3-access-grants-boto3-plugin>=1.3.0,<2.0.0
Requires-Python: >=3.10
Project-URL: Repository, https://github.com/fduplex/git-remote-s3
Description-Content-Type: text/markdown

# git-remote-s3

This library enables to use Amazon S3 as a git remote and LFS server. It provides an implementation of a
[git remote helper](https://git-scm.com/docs/gitremote-helpers) to use S3 as a serverless Git server, and an
implementation of the
[git-lfs custom transfer](https://github.com/git-lfs/git-lfs/blob/main/docs/custom-transfers.md) to enable pushing
LFS managed files to the same S3 bucket used as remote.

## About this fork

This is a fork of [awslabs/git-remote-s3](https://github.com/awslabs/git-remote-s3), licensed under the
Apache License 2.0 (unchanged). Not affiliated with or endorsed by Amazon Web Services.

What this fork adds on top of upstream:

- LFS fixes: per-remote scoping so an S3 LFS remote can coexist with non-S3 ones, auto-install of the transfer agent
  on first remote-helper run, correct temp-file paths when the repo is a submodule, and no ~10s stall per push on
  git-lfs's pure-SSH endpoint probe of the `s3://` URL
- DNS TXT bucket-alias resolution for `s3://` remote URIs
- S3 Access Grants support and region-aware S3 clients
- `git-s3 doctor`, a read-only auditor that never writes or deletes anything and is safe to run against nested
  remote prefixes (e.g. `s3://bucket/team/repo`)
- Pushing from a shallow clone is rejected with a clear error instead of silently uploading a truncated pack
- Partial clones (`git clone --filter=blob:none` / `--filter=tree:0`) are fully supported for push and fetch
- Push and fetch render live transfer progress, honoring `git push --quiet` / `--progress`
- `--force-with-lease` with real compare-and-swap semantics against the remote ref, not just a `+` force push

## Table of Contents

- [Installation](#installation)
- [Prerequisites](#prerequisites)
- [Security](#security)
  - [Data encryption](#data-encryption)
  - [Access control](#access-control)
- [Use S3 remotes](#use-s3-remotes)
  - [Create a new repo](#create-a-new-repo)
  - [Clone a repo](#clone-a-repo)
  - [Branches, etc.](#branches-etc)
  - [Using S3 remotes for submodules](#using-s3-remotes-for-submodules)
  - [DNS bucket aliases](#dns-bucket-aliases)
  - [S3 Access Grants](#s3-access-grants)
- [LFS](#lfs)
  - [Install the transfer agent](#install-the-transfer-agent)
  - [Creating the repo and pushing](#creating-the-repo-and-pushing)
  - [Clone the repo](#clone-the-repo)
  - [Coexisting with other LFS remotes](#coexisting-with-other-lfs-remotes)
  - [Fetching through a facade URL (uv and friends)](#fetching-through-a-facade-url-uv-and-friends)
- [Manage the Amazon S3 remote](#manage-the-amazon-s3-remote)
  - [Delete branches](#delete-branches)
  - [Protected branches](#protected-branches)
  - [Compact the remote](#compact-the-remote)
  - [Migrate a legacy repo](#migrate-a-legacy-repo)
  - [git-s3 doctor](#git-s3-doctor)
- [Notes about specific behaviors of Amazon S3 remotes](#notes-about-specific-behaviors-of-amazon-s3-remotes)
  - [Arbitrary Amazon S3 URIs](#arbitrary-amazon-s3-uris)
- [Under the hood](#under-the-hood)
  - [How S3 remotes work](#how-s3-remotes-work)
  - [Concurrency and locking](#concurrency-and-locking)
  - [Bucket region cache](#bucket-region-cache)
  - [DNS alias resolution](#dns-alias-resolution)
  - [Imported-entry high-water mark](#imported-entry-high-water-mark)
  - [Access Grants and IAM](#access-grants-and-iam)
  - [How LFS works](#how-lfs-works)
  - [Debugging](#debugging)
- [Credits](#credits)

## Installation

`git-remote-s3` is a Python script and works with any Python version >= 3.10.

Run:

```
pip install fduplex-git-remote-s3
```

This fork is published on PyPI as `fduplex-git-remote-s3`, but it installs the same `git-remote-s3`
command as upstream and therefore replaces it, so a given environment should install
`fduplex-git-remote-s3` or upstream `git-remote-s3`, not both.

## Prerequisites

Before you can use `git-remote-s3`, you must:

- Complete initial configuration:

  - Creating an AWS account
  - Configuring an IAM user or role

- Create an AWS S3 bucket (or have one already) in your AWS account.
- Attach a minimal policy to that user/role that allows access to the S3 bucket:

  ```json
  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "S3ObjectAccess",
        "Effect": "Allow",
        "Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject", "s3:AbortMultipartUpload"],
        "Resource": ["arn:aws:s3:::<BUCKET>/*"]
      },
      {
        "Sid": "S3ListAccess",
        "Effect": "Allow",
        "Action": ["s3:ListBucket"],
        "Resource": ["arn:aws:s3:::<BUCKET>"]
      }
    ]
  }
  ```

- Install Python and its package manager, pip, if they are not already installed. To download and install the latest version of Python, [visit the Python website](https://www.python.org/).
- Install Git on your Linux, macOS, Windows, or Unix computer.
- Install the latest version of the AWS CLI on your Linux, macOS, Windows, or Unix computer. You can find instructions [here](https://docs.aws.amazon.com/cli/latest/userguide/installing.html).

## Security

### Data encryption

All data is encrypted at rest and in transit by default. To add an additional layer of security you can use customer managed KMS keys to encrypt the data at rest on the S3 bucket. We recommend to use [Bucket keys](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html) to minimize the KMS costs.

Optional (but recommended) - use [SSE-KMS Bucket keys to encrypt the content of the bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html), ensuring the user/role created previously has the permission to access and use the key:

```json
{
  "Sid": "KMSAccess",
  "Effect": "Allow",
  "Action": ["kms:Decrypt", "kms:GenerateDataKey"],
  "Resource": ["arn:aws:kms:<REGION>:<ACCOUNT>:key/<KEY_ID>"]
}
```

### Access control

Access control to the remote is ensured via IAM permissions, and can be controlled at:

- bucket level
- prefix level (you can use prefixes to store multiple repos in the same S3 bucket thus minimizing the setup effort)
- KMS key level

If you store multiple repos in a single bucket but would like to separate permissions to access each repo, you can do so by modifying the resource definitions for the object related action to specify the repo prefix and by adding a condition to the ListBucket action to restrict the operation to matching prefixes (and by consequence the corresponding repo) :

```json
      {
        "Sid": "S3ObjectAccess",
        "Effect": "Allow",
        "Action": [
          "s3:PutObject",
          "s3:GetObject",
          "s3:DeleteObject",
          "s3:AbortMultipartUpload"
        ],
        "Resource": ["arn:aws:s3:::<BUCKET>/<REPO>/*"]
      },
      {
        "Sid": "S3ListObjects",
        "Effect": "Allow",
        "Action": [
          "s3:ListBucket",
        ],
        "Condition": {
          "StringLike": {
            "s3:prefix": "<REPO>/*"
          }
        },
        "Resource": ["arn:aws:s3:::<BUCKET>"]
      },
```

Using the condition key restricts the access operation to the content of the specific repo in the bucket. Every listing call this
tool makes goes through a helper that appends a trailing slash to the prefix (so `<REPO>` doesn't also match a sibling repo like
`<REPO>-other`), so the condition has to match `<REPO>/` and everything under it, not `<REPO>` alone: use `StringLike` with
`<REPO>/*`, not `StringEquals` with `<REPO>`.

## Use S3 remotes

### Create a new repo

S3 remotes are identified by the prefix `s3://` and at the bare minimum specify the name of the bucket. You can also provide a key prefix as in `s3://my-git-bucket/my-repo` and a profile `s3://my-profile@my-git-bucket/myrepo`.

```bash
mkdir my-repo
cd my-repo
git init
git remote add origin s3://my-git-bucket/my-repo
```

You can then add a file, commit and push the changes to the remote:

```bash
echo "Hello" > hello.txt
git add -A
git commit -a -m "hello"
git push --set-upstream origin main
```

The remote HEAD is set to track the branch that has been pushed first to the remote repo. To change the remote HEAD branch, run `git-s3 head <remote> <branch>`.

`s3+zip://` is accepted for back-compat and behaves exactly like `s3://`: no `repo.zip` archive is written, so use
`s3://` for new remotes. Because PyPI does not permit the `+` character in an installed command name, the
`git-remote-s3+zip` helper binary is not installed; if you still have `s3+zip://` remotes, create the helper once as
an alias of the `s3` helper, e.g. `ln -s "$(command -v git-remote-s3)" ~/.local/bin/git-remote-s3+zip` (or copy it
to a directory on your `PATH` on Windows).

### Clone a repo

To clone the repo to another folder just use the normal git syntax using the s3 URI as remote:

```bash
git clone s3://my-git-bucket/my-repo my-repo-clone
```

If the repo uses LFS, the transfer agent is auto-configured on clone, so no extra setup is needed; set
`GIT_REMOTE_S3_AUTO_INSTALL_LFS=0` to opt out (details in [How LFS works](#how-lfs-works)).

The bucket's region is detected once and remembered in the repo-local git config as `remote.<name>.s3region`, so
later commands skip the `HeadBucket` round trip. You never need your default region to match the bucket's. Clear it
with `git config --unset remote.origin.s3region`; see [Bucket region cache](#bucket-region-cache) for the details.

### Branches, etc.

Creating branches and pushing them works as normal:

```bash
cd my-repo
git checkout -b new_branch
touch new_file.txt
git add -A
git commit -a -m "new file"
git push origin new_branch
```

All git operations that do not rely on communication with the server should work as usual (eg `git merge`)

### Using S3 remotes for submodules

If you have a repo that uses submodules also hosted on S3, you need to run the following command:

```
git config protocol.s3.allow always
```

Or, to enable globally:

```
git config --global protocol.s3.allow always
```

### DNS bucket aliases

When the bucket component of the remote URI contains at least one dot, it is treated as a DNS hostname aliasing the
real bucket instead of a literal bucket name (bucket names used with this feature must not contain dots). The
hostname is resolved to the real bucket name via a DNS TXT lookup using the system resolver, so split-horizon/VPN
DNS setups work as usual. For example, with the record

```
repos.git.example.com. 300 IN TXT "git-bucket=my-git-bucket-123456789012-us-east-2"
```

the following commands are equivalent:

```bash
git clone s3://repos.git.example.com/my-repo
git clone s3://my-git-bucket-123456789012-us-east-2/my-repo
```

Aliasing is on by default and can be turned off, so a dotted bucket component is treated as a literal bucket name
again:

```bash
# per remote (takes precedence when set):
git config remote.origin.s3-dns-alias false
# for all remotes (used when the per-remote key is unset or no remote name is known):
git config s3.dns-alias false
```

See [DNS alias resolution](#dns-alias-resolution) for the record format, where aliases are accepted, and how the two
config keys interact.

### S3 Access Grants

The [AWS S3 Access Grants boto3 plugin](https://github.com/awslabs/aws-s3-access-grants-plugin-boto3) is bundled and
auto-registered on every S3 client this tool builds — the git remote helper, the `git-lfs-s3` transfer agent, and
the `git-s3` management CLI. There is nothing to configure:

- A caller whose identity holds an S3 Access Grant gets short-lived, prefix-scoped credentials vended by Access
  Grants for each S3 operation.
- A caller using plain IAM credentials (an access-key user or a role with direct S3 policy access and no grant)
  transparently falls back to a direct S3 call — no configuration needed.

To use the Access Grants path, the caller role/identity needs **both** `s3:GetDataAccess` and
`s3:GetAccessGrantsInstanceForPrefix` on the Access Grants instance resource. Grant them together: missing the
second one makes the tool silently fall back to direct credentials, which is exactly the failure mode
[Access Grants and IAM](#access-grants-and-iam) explains.

On the first fallback in a process a one-time notice is printed to stderr. If you *expected* Access Grants to be
used, run [`git-s3 doctor`](#git-s3-doctor): it runs a dedicated entitlement check and names the missing
permission.

## LFS

### Install the transfer agent

To use LFS you need to first install git-lfs. You can refer to the [official documentation](https://git-lfs.com/) on how to do this on your system.

Next, enable the S3 integration in the repo. There are two install modes:

```bash
# Per-remote (recommended; required when other LFS remotes coexist)
git-lfs-s3 install --remote <remote-name>

# Unscoped (back-compat; applies the agent to ALL remotes in the repo)
git-lfs-s3 install
```

`--remote` writes a per-remote scoped configuration so `git-lfs-s3` only fires for that one remote — letting an S3 remote coexist with non-S3 LFS remotes (e.g. GitHub, GitLab) without breaking their LFS push/pull. Use it whenever the repo has more than one remote.

### Creating the repo and pushing

Let's assume we want to store TIFF file in LFS.

```bash
mkdir lfs-repo
cd lfs-repo
git init
git lfs install
git remote add origin s3://my-git-bucket/lfs-repo
git-lfs-s3 install --remote origin
git lfs track "*.tiff"
git add .gitattributes
<put file.tiff in the repo>
git add file.tiff
git commit -a -m "my first tiff file"
git push --set-upstream origin main
```

### Clone the repo

```bash
git clone s3://my-git-bucket/lfs-repo lfs-repo-clone
```

`git-remote-s3` installs the LFS transfer agent in the new repo's local config on first invocation, so `git clone`
and `git submodule add` work without extra setup. Set `GIT_REMOTE_S3_AUTO_INSTALL_LFS=0` to opt out; see
[How LFS works](#how-lfs-works) for exactly what it writes and when it stays out of the way.

### Coexisting with other LFS remotes

The bare `git-lfs-s3 install` form sets `lfs.standalonetransferagent` in the current repo (not globally) and is short for:

```bash
git config --replace-all lfs.customtransfer.git-lfs-s3.path git-lfs-s3
git config --replace-all lfs.standalonetransferagent git-lfs-s3
```

`--replace-all` means re-running it is idempotent: it overwrites the existing value instead of accumulating duplicates.

`git-lfs-s3 install --remote <name>` instead writes:

```bash
git config remote.<name>.lfsurl https://lfs-alias.git-remote-s3.test/<bucket>/<prefix>
git config lfs.<that-url>.standalonetransferagent git-lfs-s3
git config lfs.customtransfer.git-lfs-s3.path git-lfs-s3
```

The `lfs-alias.git-remote-s3.test` host is a synthetic, never-contacted match key (the `.test` TLD is reserved by RFC 6761 for non-resolvable use). It exists only because git-lfs's URL parser does not natively understand `s3://` URLs and would otherwise fall back to SSH-style endpoint discovery; setting `remote.<name>.lfsurl` short-circuits that path and gives the scoped agent lookup a stable URL to match against.

`<bucket>` in that URL is the bucket component of the remote URL verbatim: when the remote uses a DNS bucket alias (e.g. `s3://demos.git.example.com/my-repo`), the alias — not the resolved bucket name — is written into the config, so re-pointing the alias at a different bucket never invalidates existing checkouts. Re-running `git-lfs-s3 install --remote <name>` migrates configs written by older versions that rendered the resolved bucket name.

`lfs.customtransfer.git-lfs-s3.path` is necessarily repo-wide (git-lfs registers transfer adapters globally, not per URL), so `git-lfs-s3` is still listed in the `transfers` array of batch requests sent to other LFS servers. If a server rejects requests naming unknown adapters, set:

```bash
git config lfs.basictransfersonly true
```

which makes git-lfs omit the `transfers` array entirely. This setting is repo-wide and limits other remotes to basic HTTPS transfers; GitHub-style hosts already use these (even over SSH remotes), so only the rare server that speaks exclusively the pure-SSH LFS protocol is affected.

### Fetching through a facade URL (uv and friends)

Some tools invoke git-lfs with a URL instead of a remote name, in a directory that has no remotes configured at all: `uv` does exactly this for a git dependency with `lfs = true`, running `git lfs fetch <url> <sha>` inside its own cache directory. git-lfs passes that URL to the transfer agent verbatim, before any rewriting.

For those callers, map the facade URL to the S3 remote with git's standard URL rewriting:

```bash
git config --global url."s3://<bucket>/<prefix>".insteadOf https://git.example.com/<prefix>
git config --global lfs.customtransfer.git-lfs-s3.path git-lfs-s3
git config --global lfs."https://s3".standalonetransferagent git-lfs-s3
```

The agent re-applies the `insteadOf` mapping itself (longest matching prefix wins, same as git) to recover the `s3://` URI, so the fetch works with no remote configured. Without a matching `insteadOf` entry it fails the transfer with an error naming the URL it could not map.

## Manage the Amazon S3 remote

### Delete branches

To remove remote branches that are not used anymore you can use the `git-s3 delete-branch <remote> <branch_name>` command. This is a refs-only change: a conditional PUT drops the branch from the manifest. The packs it uniquely referenced stay in the bucket until `git-s3 compact` supersedes them.

### Protected branches

To protect/unprotect a branch run `git s3 protect <remote> <branch-name>` respectively `git s3 unprotect <remote> <branch-name>`.

### Compact the remote

`git-s3 compact <remote>` collapses the manifest's whole entry log into a single base pack covering every current
ref, then deletes the packs it superseded. The same pass reclaims orphans — packs no manifest entry has ever named
— once they are older than the grace period (24h by default, `--prune-orphans-older-than <duration>`; values under
1h require `--yes`). `git-s3 compact` is the only command that reclaims packs. See
[How S3 remotes work](#how-s3-remotes-work) for why the log grows and what the grace period protects.

### Migrate a legacy repo

Repos created before 0.6 used a per-ref bundle-file format instead of the `gitwal.json` manifest.
[`git-s3 doctor <remote>`](#git-s3-doctor) reports `gitwal.json: missing (this repo has not been migrated)` on one
of these. Bring it forward in two steps:

```bash
git-s3 migrate <remote>
```

This is phase 1: it reads every `refs/.../*.bundle` key, packs their tips into a single base pack, and writes
`gitwal.json` (carrying the legacy HEAD and protected markers across) alongside the legacy keys. Nothing legacy is
touched or deleted, so the old and new formats are both live and correct at once — check the migrated repo before
committing to it: run `git-s3 doctor <remote>`, and do a real clone and push through the new client. Roll back by
deleting what phase 1 wrote (`gitwal.json` and `packs/`); the legacy keys are untouched.

Once you're satisfied, finalize:

```bash
git-s3 migrate --finalize --yes <remote>
```

This deletes the pre-migration bundle keys. It is irreversible, which is why it needs both `--finalize` and `--yes`.

`git-s3 head <remote>` with no branch argument is also valid: it's a read (no CAS, no write) that just prints the
remote's current default branch and whether it resolves.

### git-s3 doctor

`git-s3 doctor <remote>` audits a repo's manifest and packs: schema validation, missing packs, orphan packs,
whether compaction is due, and an Access Grants entitlement check. It is a read-only auditor — it never writes or
deletes anything, and is safe to run against nested remote prefixes (e.g. `s3://bucket/team/repo`).

The Access Grants section runs the plugin with fallback **disabled** and drives the full vend path against the
repo's prefix, so it surfaces the real error the fallback would otherwise hide. It reports:

- `Access Grants: OK` when credentials were vended for the repo prefix.
- `Access Grants: not available (using direct S3 credentials)` on an `AccessDenied`, naming the exact failing operation and the missing permission — e.g. `caller role is missing s3:GetAccessGrantsInstanceForPrefix` or `caller role is missing s3:GetDataAccess or has no matching grant`.

This is informational: an IAM-key user with no grant legitimately reports "not available" and keeps working via direct credentials — that is expected, not an error.

## Notes about specific behaviors of Amazon S3 remotes

### Arbitrary Amazon S3 URIs

An Amazon S3 URI for a valid bucket and an arbitrary prefix which does not contain the right structure under it, is considered valid.

`git ls-remote` returns an empty list and `git clone` clones an empty repository for which the S3 URI is set as remote origin.

```
% git clone s3://my-git-bucket/this-is-a-new-repo
Cloning into 'this-is-a-new-repo'...
warning: You appear to have cloned an empty repository.
```

**Tip**: This behavior can be used to quickly create a new git repo.

## Under the hood

### How S3 remotes work

A repo is one manifest object, `<prefix>/gitwal.json`, plus the packs it names under `<prefix>/packs/<sha>.pack`. The manifest is the sole authority for what a ref points to: it lists every branch and tag, the HEAD, the protected refs, and a log of entries, each naming a pack and the tips that pack makes reachable.

Listing refs (`git ls-remote`, the start of a clone or fetch) reads the manifest, no bucket listing required.

Pushing packs the new objects with `git pack-objects`, excluding whatever the manifest already has, uploads the pack to its content-addressed key, then commits with a single conditional PUT to `gitwal.json` (see [Concurrency and locking](#concurrency-and-locking)). The PUT is the only step that can fail on a race; the pack upload before it is inert until an entry in the manifest names it.

Because entries accumulate with every push, the log slowly grows one pack per push and the same objects can end up duplicated across several packs. [`git-s3 compact <remote>`](#compact-the-remote) collapses the whole log into a single base pack covering every current ref, then deletes the packs it superseded. The same pass reclaims orphans — packs under `<prefix>/packs/` that no manifest entry has ever named, left by a push whose CAS lost or whose process died between the upload and the commit. Only orphans older than the grace period are collected, because a recently uploaded unreferenced pack may belong to a push that is still in flight. The referenced set is re-read from the manifest after the bucket is listed, and each pack's age is re-checked immediately before its delete, so a pack that becomes referenced mid-run is never collected.

### Concurrency and locking

`git-remote-s3` has no locks. Every ref in the repo — branch, tag, and HEAD — lives in a single object, `<prefix>/gitwal.json`, and every write to that repo is one conditional PUT against it: `If-None-Match: *` to create it, `If-Match: <etag>` to update it. S3 only accepts the PUT if the etag still matches what the client read, so two pushers racing each other cannot both win.

The loser's PUT is rejected with a precondition failure, and `git-remote-s3` reloads the manifest, re-checks the push against the refs it now names (fast-forward, `--force`, `--force-with-lease`, protected-branch), and retries the whole decision from scratch. The retry loop is bounded (8 attempts, backoff capped at 2s); under sustained contention it gives up and fails the push with a clear error per ref. Nothing is left behind either way.

The objects a push uploads — packs under `<prefix>/packs/<sha>.pack`, LFS blobs under `<prefix>/lfs/<oid>` — are content-addressed and written before the manifest CAS, so they are immutable and safe to upload from multiple clients at once; the manifest PUT is the single serialization point that decides which pack(s) actually become part of a ref's history.

A pack uploaded by a push that loses the race is simply never referenced by any entry and becomes an orphan. Orphans are inert, and `git-s3 compact` reclaims them once they are older than its grace period (24h by default, `--prune-orphans-older-than`); the grace period is what keeps the sweep from deleting the pack of a push that has uploaded but not yet committed. No data is lost and no ref is ever left pointing at more than one place.

### Bucket region cache

Every S3 client the remote helper builds is pinned to the bucket's own region, detected via a `HeadBucket` probe (which returns the region even for an unauthorized caller, so it needs no extra permission and is cached per process). Doing that on every single git command would cost a round trip each time, so the first successful detection is written to the repo-local git config as `remote.<name>.s3region`, and every later invocation reads it from there instead:

```bash
git config --get remote.origin.s3region
# eu-west-1
```

The value is written the first time any helper invocation builds an S3 client for a real remote name — not just clone, fetch or push, but also read-only commands like `git ls-remote` and `git remote show`. A push or fetch straight to a URI (`git push s3://bucket/repo ...`) detects the region and does not cache it, since there is no remote name to cache it under. For submodules the key lands in the submodule's own config under `.git/modules/<name>/config`.

If a bucket ever moves to another region, the cached value goes stale. The helper notices the redirect S3 returns, drops the key and retries once, so the operation still succeeds; you can also clear it by hand:

```bash
git config --unset remote.origin.s3region
```

If the region cannot be determined at all, the tool proceeds with your default region and S3's cross-region redirects, exactly as before.

### DNS alias resolution

For a dotted bucket component, the TXT lookup expects:

- A TXT record at the alias hostname itself.
- Among its TXT values, exactly one of the form `git-bucket=<real-bucket-name>`; other TXT values at the same name are ignored.

Aliases work in every place a remote URI is accepted: the git remote helper, the `git-lfs-s3` transfer agent and `git-lfs-s3 install --remote`, and the `git-s3` management CLI. Resolution results are cached for the lifetime of the process. If the alias has no TXT record or no `git-bucket=` value, the command fails with an error describing the expected record instead of falling back to using the hostname as a bucket name.

Both `remote.<name>.s3-dns-alias` and `s3.dns-alias` are booleans; setting the per-remote key to `true` re-enables aliasing for that remote even when `s3.dns-alias` is `false`. The per-remote key applies where a remote name is available (the git remote helper, the LFS transfer agent, `git-lfs-s3 install --remote`). The `git-s3` CLI is invoked with a remote name too, but it resolves the bucket alias without passing that name along, so it only ever consults `s3.dns-alias`.

### Imported-entry high-water mark

`remote.<name>.gitwal-seq` records the highest `gitwal.json` entry seq imported into this clone, so a routine fetch downloads only the packs added since. It is a hint, never state: after importing, the client verifies the fetched tips with `git rev-list --objects` and, when they do not resolve, keeps pulling older entries until they do. A stale or hand-edited value costs a round trip, never correctness, and the key is written only after verification passes.

```bash
git config --get remote.origin.gitwal-seq
# 43
```

As with the region cache, a fetch straight to a URI has no remote section to write to and re-imports from the start of the log.

### Access Grants and IAM

The Access Grants plugin is registered transparently and always runs with fallback enabled, so a single code path serves both credential models. The two IAM actions the vend path needs are:

- `s3:GetDataAccess` — vends the scoped credentials.
- `s3:GetAccessGrantsInstanceForPrefix` — resolves which account owns the Access Grants instance for the requested `s3://bucket/prefix`.

The plugin calls `GetAccessGrantsInstanceForPrefix` **before** it can call `GetDataAccess`, because it must first learn the owner account id to target. This is a separate IAM action that is easy to overlook: if the caller has `s3:GetDataAccess` but not `s3:GetAccessGrantsInstanceForPrefix`, the plugin fails during that preflight and — because fallback is enabled — **silently** drops to direct S3 credentials. The user then sees only a misleading downstream `AccessDenied` from the direct call (or a successful direct call that never used Access Grants at all), with nothing pointing at the real cause. Grant both actions together.

[`git-s3 doctor`](#git-s3-doctor) is the way out: its entitlement check disables fallback and drives the full vend path, including that preflight, so the real error surfaces.

### How LFS works

The LFS integration stores the file in the bucket defined by the remote URI, under a key `<prefix>/lfs/<oid>`, where oid is the unique identifier assigned by git-lfs to the file.

If an object with the same key already exists, git-lfs-s3 does not upload it again.

The auto-install that runs on first remote-helper invocation writes exactly the same per-remote keys as `git-lfs-s3 install --remote <name>` — `lfs.customtransfer.git-lfs-s3.path`, `remote.<name>.lfsurl` and the URL-scoped `lfs.<url>.standalonetransferagent` — and never the repo-wide `lfs.standalonetransferagent`. An existing `lfs.standalonetransferagent` naming another agent, or an existing `remote.<name>.lfsurl`, suppresses the install entirely, and nothing is written for a remote that is not an `s3://` URL — including a bucket-root remote with no prefix (`s3://my-git-bucket`), which has no repo-specific path to build an LFS URL from. `GIT_REMOTE_S3_AUTO_INSTALL_LFS=0` disables it.

### Debugging

Use `--verbose` flag or set `transfer.verbosity=2` to print debug information when performing git operations:

```bash
git -c transfer.verbosity=2 push origin main
```

For early errors (like credential issues), use the environment variable:

```bash
GIT_REMOTE_S3_VERBOSE=1 git push origin main
```

Logs will be put to stderr.

For LFS operations you can enable and disable debug logging via `git-lfs-s3 enable-debug` and `git-lfs-s3 disable-debug` respectively. Logs are put in `.git/lfs/tmp/git-lfs-s3.log` in the repo. For submodules the log lands under the superproject's `.git/modules/<name>/lfs/tmp/git-lfs-s3.log` instead, same as the region cache.

## Credits

The git S3 integration was inspired by the work of Bryan Gahagan on [git-remote-s3](https://github.com/bgahagan/git-remote-s3).

The LFS implementation benefitted from [lfs-s3](https://github.com/nicolas-graves/lfs-s3) by [@nicolas-graves](https://github.com/nicolas-graves). If you do not need to use the git-remote-s3 transport you should use that project.
