Metadata-Version: 2.1
Name: airflow-cdk
Version: 0.3.1
Summary: Custom cdk constructs for apache airflow
Home-page: https://github.com/knowsuchagency/airflow-cdk
Author: Stephan Fitzpatrick
Author-email: stephan@knowsuchagency.com
License: UNKNOWN
Keywords: aws,cdk,airflow,k8s
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: JavaScript
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Topic :: Software Development :: Code Generators
Classifier: Topic :: Utilities
Classifier: Typing :: Typed
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: aws-cdk.core (>=1.34.1)
Requires-Dist: aws-cdk.aws-ecs (>=1.34.1)
Requires-Dist: aws-cdk.aws-ecs-patterns (>=1.34.1)
Requires-Dist: aws-cdk.aws-rds (>=1.34.1)
Requires-Dist: aws-cdk.aws-s3 (>=1.34.1)
Requires-Dist: aws-cdk.aws-elasticloadbalancingv2 (>=1.34.1)
Provides-Extra: dev
Requires-Dist: pytest ; extra == 'dev'
Requires-Dist: toml ; extra == 'dev'
Requires-Dist: black ; extra == 'dev'
Requires-Dist: twine ; extra == 'dev'
Requires-Dist: invoke ; extra == 'dev'
Requires-Dist: jinja2 ; extra == 'dev'
Requires-Dist: apache-airflow[aws,celery,postgres] (>=1.10.10) ; extra == 'dev'

# airflow-cdk

This project makes it simple to deploy airflow via ECS fargate using the aws cdk in Python.

## Usage

`pip install airflow-cdk`

```python3
from aws_cdk import core
from airflow_cdk import FargateAirflow


app = core.App()

FargateAirflow(
    app,
    "airflow-cdk",
    postgres_password="replacethiswithasecretpassword")

app.synth()
```

`cdk deploy`

That's it.

## Components

The following aws resources will be deployed as ecs tasks within the same cluster and vpc by default:

* an airflow webserver task
  * and an internet-facing application load-balancer
* an airflow scheduler task
* an airflow worker task
  * (note) it will auto-scale based on cpu and memory usage up to a total of 16 instances at a time by default starting from 1
* a rabbitmq broker
* an rds instance
* an s3 bucket for logs

## TODOs

* create a custom component to deploy airflow to an ec2 cluster as opposed to fargate
* improve documentation
* (possibly) subsume the [airflow stable helm chart](https://hub.helm.sh/charts/stable/airflow) as a cdk8s chart
* (possibly) a flower service

## Contributions Welcome!

