Metadata-Version: 2.1
Name: bbp
Version: 0.1
Summary: A wrapper for boto3 paginators to iterate per resource
Home-page: https://github.com/mlda065/bbp
Author: Matthew Davis
Author-email: bbp@mdavis.xyz
License: GPLv3
Download-URL: https://github.com/mlda065/bbp/archive/v_01.tar.gz
Keywords: boto3,boto,AWS,Amazon,paginator,pagination,page,api,cloud
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Build Tools
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Description-Content-Type: text/markdown
Requires-Dist: boto3

# BBP - Better Boto Paginator

The [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) module has [pagination](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/paginators.html) functionality.
So if you're trying to enumerate a long list of resources, the paginator will provides an easier way to fetch chunk after chunk of the resource list, compared to raw `list_` calls.

The problem with how the module exposes these pages is that you end up with a list of lists.
For example, to get a list of all objects within an S3 bucket, you can do:

```
import boto3
client = boto3.client('s3')
paginator = client.get_paginator('list_objects_v2')
objects = [p['Contents'] for p in paginator.paginate(Bucket='my-bucket')]
```

This returns a list of lists of object information.
Do you remember off the top of your head how to flatten a list of lists into one list?
I sure don't.
Yes I could have a for loop and append to a list each iteration, but that feels like more effort than should be required.

Even if you're not loading the whole resource list into a list in memory, and are instead processing within a for loop, you end up with a messy nested for loop.

```
for page in paginator.paginate(Bucket='my-bucket'):
    if ['Contents'] in page:
        for element in page['Contents']:
             process(element)
```

I find this a bit awkard. 
What I really want is:

```
for element in function(Bucket='my-bucket'):
   process(element)
```

This library provides that function.

## Installation

`pip install bbp`

## Usage

Here's an example of how to use it for the [Lambda `ListFunctions` paginator](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/lambda.html#Lambda.Paginator.ListFunctions).


```
from wrapper import paginator
from pprint import pprint
for lam in paginator('lambda', 'list_functions', 'Functions'):
    pprint(lam) # process just one element at a time
```

* `lambda` is what you would pass to `boto3.client()`
* `list_functions` is what you would pass to `client.get_paginator()`
* `Functions` is the key within the response to `list_objects_v2` which contains the list of resources for each page.
  This varies for each type of pagination call. You have to look up [the documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/lambda.html#Lambda.Paginator.ListFunctions).
  Eventually I'll try to get this tool to lookup/remember that.

Here's another example, using the [S3 `ListObjectsV2` paginator](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Paginator.ListObjectsV2).
In this example we need to pass in the bucket name as an extra argument.
Just specify this as a `name=value` pair at the end of the argument list.

```
for obj in paginator('s3', 'list_objects_v2', 'Contents', Bucket='mybucket'):
    pprint(obj) # process a single resource
``` 

* `s3` is what you would pass to `boto3.client()`
* `list_objects_v2` is what you would pass to `client.get_paginator()`
* `Bucket='mybucket'` and any other `name=value` arguments are what get passed to [the paginator](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Paginator.ListObjectsV2).


## Packaging

This is my first ever package on PyPI.
I used [this guide](https://medium.com/@joel.barmettler/how-to-upload-your-python-package-to-pypi-65edc5fe9c56) to learn how to do this.


