# dbprocessing

# Building a release

- Release notes

  - Release note subsubsections

- PRs and issues in release notes

- Preparing the release commit

- Isolated anaconda environment

- Preparing the distributions

- Docs

- Test PyPI

- Release to PyPI

- Release to github

- Web page update

## Release notes

The release notes should be updated with the date of release and checked for completeness and accuracy. The release series (x.y) is formatted as a section, the actual release (x.y.z) as a subsection, and the possible subsubsections follow here.

Normally release notes should be updated as functionality is added/changed, not only at release time.

### Release note subsubsections

If any of these is not used for a particular release, it may be omitted.

#### New features

Brief description of any features that were added in this release. Link to appropriate documentation of the feature (e.g. user documentation such as the script, or API documentation). Link to either the Github pull request that introduced the feature or the enhancement request issue related to it, whichever has the most information.

This includes added functionality in existing scripts, functions, and classes.

#### Deprecations and removals

Any functionality that has been removed or is planned to be removed (deprecated). This includes API deprecations. For deprecated features, link their documentation; for deprecated or removed, link the appropriate replacement.

This also includes any changes in support for older database versions.

#### Dependency requirements

Any changes to dependencies (e.g. new dependencies, new minimum versions.)

#### Major bugfixes

Fixed bugs that are likely to be of interest to a substantial fraction of dbprocessing users. Omit very obscure bugs that are unlikely to be hit; they will be included in the full list of closed issues.

Normally link to the relevant Github issue, although in some cases the pull request that closed the bug may be more appropriate.

#### Other changes

Any other changes that are likely to be of interest to a substantial fraction of users and do not fit the above categories. In particular, any changes in behavior or expectations that are not new features or bug fixes.

#### Pull requests merged this release

Should be generated and pasted into release notes at release time; see below.

#### Other issues closed this release

Also generated at release time.

## PRs and issues in release notes

The list of pull requests and issues closed in a release is created from the script `git_notes.py` in `developer/scripts`.

This script requires GitPython and PyGithub. On Ubuntu these are packaged as `python-git` and `python-github` (or `python3-git` and `python3-github`.)

You need a Github token to avoid throttling. No scopes are required.

Run the script with your token and the name of the last release tag. The relevant rST will be sent to stdout; redirect to a file, and copy/paste into the release notes. Do not do this on a shared machine, as the token will be visible in the process table.

    python ./git_notes.py -t GITHUB_TOKEN -r release-0.1

Note that the PR including the release should also be in the list; you can open a draft PR to get the PR number, and manually add that to the list. Similarly, the commit against `gh-pages` that updates the website should be in the list.

## Preparing the release commit

The following changes should result in either the final release version, or the appropriate release candidate (i.e. add “rc1” if building rc).

> - Change version in `setup.py`, in setup_kwargs near bottom.

> - Change `__version__` around line 209 of `dbprocessing/__init__.py`.

Commit these changes.

If this is the release version, make a second commit setting all the versions to the next version number and `rc0`. Then submit the PR. The tagging is done in github and will be done from the last commit before the `rc0` commit.

## Isolated anaconda environment

You may wish to perform all the steps after this within an isolated Anaconda environment, separate from your machine’s Python setup. In that case, download Miniconda, install it, and use that environment:

    bash ./Miniconda3-latest-Linux-x86_64.sh -b -p ~/miniconda
    ~/miniconda/bin/conda create -y -n dbp_build python=3
    export PYTHONNOUSERSITE=1
    export PYTHONPATH=
    source ~/miniconda/bin/activate dbp_build
    conda install sqlalchemy python-dateutil sphinx numpydoc twine numpy wheel

Note numpy is only required for the `reports` module (and thus its documentation).

## Preparing the distributions

    python setup.py sdist --formats=gztar,zip
    python setup.py bdist_wheel --universal

Note this will perform a “build” and then rebuild the documentation (in `sphinx/build`) for the souce distribution. Tarball, zip, and wheel are in the dist directory.

## Docs

From the `sphinx` directory:

    make latexpdf
    cp build/latex/dbprocessing.pdf dbprocessing-x.y.z-doc.pdf
    cd build/html
    zip -r ../../dbprocessing-x.y.z-doc.zip *

## Test PyPI

Test PyPI allows the upload and test of a complete build.

A release candidate build should be uploaded first. `rc0` is used as the version number throughout development; actual release candidates should start with `rc1`.

Make a .pypirc file

    [distutils]
    index-servers =
        pypi
        testpypi

    [pypi]
    username: <username>

    [testpypi]
    repository: https://test.pypi.org/legacy/
    username: <username>

Put all the builds (source dists) into one directory and then do the upload:

    twine upload -r testpypi dbprocessing-*.zip dbprocessing-*.whl

PyPI can only take one source distribution (zip or tar.gz), so we use zip.

Test installing with:

    pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ dbprocessing

You can use the `--pre` flag to install the RC version; in that case, probably want to use `--no-deps` so don’t get RC version of dependencies! (Or can specify the rc version, e.g. `dbprocessing==0.1.0rc1`).

## Release to PyPI

See PyPI upload directions

    twine upload dbprocessing-*.zip dbprocessing-*.whl

Do not upload the .tar.gz since can only upload one source package per release.

There’s no longer any capability to edit information on PyPI, it’s straight from the setup.py metadata.

## Release to github

See GitHub directions.

On the code tab, click on “n releases” (on the right column, below “about”). Click “Draft a new release.” Make the tag “release-x.y.z” and select the appropriate commit as the target (the one where the version number was updated).

Use just “x.y.z” as the title. The “describe” should link the release notes on the website and also have a brief version of the release notes included.

Click in the “upload binaries” area and upload all the files: source distributions (zip and tar), wheel, documentation PDF (`dbprocessing-x.y.z-doc.pdf`) and a zip (`dbprocessing-x.y.z-doc.zip`).

## Web page update

Check out the `gh-pages` branch. Right now the root of the branch is basically the root of the `sphinx/build/html` output. Copy all the freshly-built docs there, commit, submit PR against the `gh-pages` branch.

Release: 0.1.0 Doc generation date: Feb 10, 2022