Metadata-Version: 2.1
Name: delphi-utils
Version: 0.3.26
Summary: Shared Utility Functions for Indicators
License: MIT License
Project-URL: Homepage, https://github.com/cmu-delphi/covidcast-indicators
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.8
Classifier: License :: OSI Approved :: MIT License
Requires-Python: ==3.8.*
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: boto3
Requires-Dist: covidcast
Requires-Dist: cvxpy
Requires-Dist: epiweeks
Requires-Dist: gitpython
Requires-Dist: importlib-resources >=1.3
Requires-Dist: numpy
Requires-Dist: pyarrow
Requires-Dist: pandas >=1.1.0
Requires-Dist: requests
Requires-Dist: slackclient
Requires-Dist: scs <3.2.6
Requires-Dist: structlog
Requires-Dist: xlrd
Provides-Extra: dev
Requires-Dist: darker[isort] ~=2.1.1 ; extra == 'dev'
Requires-Dist: pylint ==2.8.3 ; extra == 'dev'
Requires-Dist: pytest ; extra == 'dev'
Requires-Dist: pydocstyle ; extra == 'dev'
Requires-Dist: pytest-cov ; extra == 'dev'
Requires-Dist: mock ; extra == 'dev'
Requires-Dist: moto ~=4.2.14 ; extra == 'dev'
Requires-Dist: requests-mock ; extra == 'dev'
Requires-Dist: freezegun ; extra == 'dev'
Provides-Extra: flash
Requires-Dist: scipy ; extra == 'flash'

# Delphi Python Utilities

This package provides various utilities used by the [Delphi group](https://delphi.cmu.edu/) at [Carnegie Mellon
University](https://www.cmu.edu) for its data pipelines and analyses.

Submodules:
- `archive`: Diffing and archiving CSV files.
- `export`: DataFrame to CSV export.
- `geomap`: Mappings between geographic resolutions.
- `logger`: Structured JSON logger.
- `nancodes`: Enum constants encoding not-a-number cases.
- `runner`: Orchestrator for running an indicator pipeline.
- `signal`: Indicator (signal) naming.
- `slack_notifier`:  Slack notification integration.
- `smooth`: Data smoothing functions.
- `utils`: JSON parameter interactions.
- `validator`: Data sanity checks and anomaly detection.


Source code can be found here:
[https://github.com/cmu-delphi/covidcast-indicators/](https://github.com/cmu-delphi/covidcast-indicators/)

## Logger Usage

To make our structured logging as useful as it can be, particularly within the context of how we use logs in Elastic, the `event` argument (typically the first unnamed arg) should be a static string (to make filtering easier), and each dynamic/varying value should be specified in an individual meaningfully- and consistently-named argument to the logger call (for use in filtering, thresholding, grouping, visualization, etc).

### Commonly used argument names:
- data_source
- geo_type
- signal
- issue_date
- filename

Single-thread usage.

```py
from delphi_utils.logger import get_structured_logger

logger = get_structured_logger('my_logger')
logger.info('Hello', name='World')
```

Multi-thread usage.

```py
from delphi_utils.logger import get_structured_logger, pool_and_threadedlogger

def f(x, threaded_logger):
    threaded_logger.info(f'x={x}')
    return x*x

logger = get_structured_logger('my_logger')
logger.info('Hello, world!')
with pool_and_threadedlogger(logger, n_cpu) as (pool, threaded_logger):
    for i in range(10):
        pool.apply_async(f, args=(i, threaded_logger))
```
