Metadata-Version: 2.1
Name: c7n-logexporter
Version: 0.4.1
Summary: Cloud Custodian - Cloud Watch Log S3 exporter
Home-page: https://cloudcustodian.io
Author: Cloud Custodian Project
License: UNKNOWN
Platform: UNKNOWN
Requires-Python: >=3.6,<4.0
Description-Content-Type: text/markdown
Requires-Dist: argcomplete (==1.11.1)
Requires-Dist: attrs (==19.3.0)
Requires-Dist: boto3 (==1.13.14)
Requires-Dist: botocore (==1.16.14)
Requires-Dist: c7n (==0.9.2)
Requires-Dist: click (==7.1.2)
Requires-Dist: docutils (==0.15.2)
Requires-Dist: importlib-metadata (==1.6.0)
Requires-Dist: jmespath (==0.10.0)
Requires-Dist: jsonschema (==3.2.0)
Requires-Dist: pyrsistent (==0.16.0)
Requires-Dist: python-dateutil (==2.8.1)
Requires-Dist: pyyaml (==5.3.1)
Requires-Dist: s3transfer (==0.3.3)
Requires-Dist: six (==1.14.0)
Requires-Dist: tabulate (==0.8.7)
Requires-Dist: urllib3 (==1.25.9)
Requires-Dist: zipp (==3.1.0)

# c7n-log-exporter: Cloud watch log exporter automation

A small serverless app to archive cloud logs across accounts to an archive bucket. It utilizes
cloud log export to s3 feature for historical exports.

It also supports kinesis streams / firehose to move to realtime exports in the same format
as the periodic historical exports.


## Features

 - Log group filtering by regex
 - Incremental support based on previously synced dates
 - Incremental support based on last log group write time
 - Cross account via sts role assume
 - Lambda and CLI support.
 - Day based log segmentation (output keys look
   like $prefix/$account_id/$group/$year/$month/$day/$export_task_uuid/$stream/$log)


## Assumptions

 - The archive bucket has already has appropriate bucket policy permissions.
   For details see:
   https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasks.html#S3Permissions
 - Default periodicity for log group archival into s3 is daily.
 - Exporter is run with account credentials that have access to the archive s3 bucket.
 - Catch up archiving is not run in lambda (do a cli run first)


## Cli usage

```
make install
```

You can run on a single account / log group via the export subcommand
```
c7n-log-exporter export --help
```

## Config format

To ease usage when running across multiple accounts, a config file can be specified, as
an example.

```
destination:
  bucket: custodian-log-archive
  prefix: logs2

accounts:
  - name: custodian-demo
    role: "arn:aws:iam::111111111111:role/CloudCustodianRole"
    groups:
      - "/aws/lambda/*"
      - "vpc-flow-logs"
```

## Multiple accounts via cli

To run on the cli across multiple accounts, edit the config.yml to specify multiple
accounts and log groups.

```
c7n-log-exporter run --config config.yml
```

## Serverless Usage

Edit config.yml to specify the accounts, archive bucket, and log groups you want to
use.

```
make install
make deploy
```



