Metadata-Version: 2.4
Name: developers-chamber
Version: 1.0.6
Summary: A small plugin which help with development, deployment, git
Home-page: https://github.com/druids/django-project-info
Author: Druids team
Author-email: matllubos@gmail.com
License: MIT
Keywords: django,skripts,easy live,git,bitbucket,Jira
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1
Requires-Dist: requests>=2.23.0
Requires-Dist: python-dotenv==1.2.3
Requires-Dist: python-hosts==0.4.6
Requires-Dist: coloredlogs==15.0.1
Requires-Dist: click-completion==0.5.2
Requires-Dist: toml>=0.10.2
Requires-Dist: PyYAML>=6.0.3
Provides-Extra: test
Requires-Dist: pytest>=8.0; extra == "test"
Provides-Extra: slack
Requires-Dist: slack-sdk==3.43.0; extra == "slack"
Provides-Extra: aws
Requires-Dist: boto3<2; extra == "aws"
Provides-Extra: qa
Requires-Dist: gitpython==3.1.37; extra == "qa"
Requires-Dist: isort==5.12.0; extra == "qa"
Requires-Dist: flake8>=7.0.0; extra == "qa"
Provides-Extra: bitbucket
Requires-Dist: gitpython==3.1.37; extra == "bitbucket"
Provides-Extra: gitlab
Requires-Dist: gitpython==3.1.37; extra == "gitlab"
Provides-Extra: jira
Requires-Dist: gitpython==3.1.37; extra == "jira"
Requires-Dist: jira==2.0.0; extra == "jira"
Requires-Dist: unidecode==1.4.0; extra == "jira"
Provides-Extra: toggle
Requires-Dist: TogglPy==0.1.2; extra == "toggle"
Provides-Extra: git
Requires-Dist: gitpython==3.1.37; extra == "git"
Dynamic: author
Dynamic: author-email
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: summary

# Developers-chamber

Library of helpers for python development.

# Installation

## Register plugin in requirements.txt and in you Django configuration

```python
pip install developers-chamber
```

## Optional modules

The commands which need a heavier library are shipped as extras and the command group appears only when its extra is installed:

| extra | commands |
| --- | --- |
| `git` | `pydev git` |
| `gitlab` | `pydev gitlab` |
| `bitbucket` | `pydev bitbucket` |
| `jira` | `pydev jira` |
| `toggle` | `pydev toggle` |
| `slack` | `pydev slack` |
| `aws` | `pydev ecs` |
| `qa` | `pydev qa` |

```bash
pip install 'developers-chamber[git,jira]'
```

# Development

Install the library in the editable mode together with the extras you need and the test extra:

```bash
pip install -e '.[git,test]'
```

Run the tests with pytest from the root of the repository:

```bash
pytest
```

The tests of the `pydev git` commands are skipped without the `git` extra, the rest of the suite needs no extra at all.

## Required system libraries

### Docker

Docker and docker-compose is required for most of `pydev project` commands.

### Bindfs

If you want to use pydev command `pydev project bind-library` you have to install `bindfs` for mounting location to the right directory. For example you can mount directory with library to a docker shared volume. 

Installation command for linux/macOS:
```bash
apt-get install bindfs
yum install bindfs
brew install bindfs
```

## Bash completion

You can install a bash completion with pydev command `init-completion`:

```bash
pydev init-completion
```

## Configuration files

You can configure pydev for your project with config files. You can define more config files which is loaded with alphabetical order (file with the highest order will override the same configuration with the lowest order).
There are two places for the pydev configuration files. One place is used for all the projects and the second one is project specific. 
The purpose of the general pydev settings is to have some place where you can store your personal or secret configuration for example key to your personal toggle account. The directory for general settings is `$HOME/.pydev`.
The project configuration directory is stored in path `$PROJECT_DIR/.pydev` and works only if pydev commands are run in the `$PROJECT_DIR` directory. The same values in the general configuration will be overriden by to project configuration. 


Configuration files which are stored in the `.pydev` directory can be written in three formats which are recognized by the file suffix:

* `.conf` - the plain `KEY=VALUE` format
* `.yaml` or `.yml`
* `.json`

The formats can be mixed in one directory and the files are applied in the alphabetical order of their whole name no matter the format:

```bash
./.pydev/
    base.yaml
    dev.conf
    prod.json
```

If you want to disable configuration file you can rename it to start with character `~`. In the example `~prod.yaml` will be ignored:

```bash
./.pydev/
    base.yaml
    dev.conf
    ~prod.yaml
```

The configuration file may contain project settings and aliases.

### Settings

Project settings saves the time to the developers because pydev input parameters may be omitted if the parameter is defined in the configuration file. Example of such file can be:

```bash
JIRA_PROJECT_KEY=PROJ
PROJECT_DOCKER_COMPOSE_FILES=path/to/the/compose.yaml
PROJECT_DOCKER_COMPOSE_DEFAULT_UP_CONTAINERS=container_a,container_b
PROJECT_DOCKER_COMPOSE_CONTAINERS_DIR_TO_COPY=container_a:/usr/local/lib/python3.11/site-packages:var/site-packages
PROJECT_DOCKER_COMPOSE_CONTAINERS_INSTALL_COMMAND=base:./manage.py compilemessages && ./manage.py migrate && ./manage.py collectstatic
PROJECT_DOCKER_COMPOSE_VAR_DIRS=var,var/data/media,var/data/static
PROJECT_LIBRARY_DIR=var/site-packages
```

The same settings written in the YAML format:

```yaml
jira:
  project_key: PROJ

project:
  library_dir: var/site-packages
  docker_compose:
    files: [path/to/the/compose.yaml]
    default_up_containers: [container_a, container_b]
    var_dirs: [var, var/data/media, var/data/static]
    containers_dir_to_copy:
      container_a:
        /usr/local/lib/python3.11/site-packages: var/site-packages
    containers_install_command:
      base: ./manage.py compilemessages && ./manage.py migrate && ./manage.py collectstatic
```

The structured formats are translated into the same settings which the commands read, therefore the rules are:

* A nested object is a section and its name is joined to the name of its keys with an underscore, so `jira.project_key` becomes `JIRA_PROJECT_KEY`. Dashes in the names are read as underscores.
* A list is joined with a comma, so `files: [a.yaml, b.yaml]` becomes `a.yaml,b.yaml`.
* A `null` value is ignored, which is useful for turning a setting of the general configuration off.
* Settings which have their own format - `aliases`, `containers_dir_to_copy`, `containers_install_command` and `containers_env` - accept the structured notation shown above. Each of them accepts the plain string notation too, so you can always fall back to it.

A setting which is defined in a configuration file overrides the environment variable of the same name.

### Alias

Alias is a shortcut for the developer to save his time and are defined with the `aliases` setting:

```yaml
aliases:
  up: project up
  up-all: project up -a
```

In the `.conf` format the same aliases have to be written as a JSON string:

```bash
ALIASES='{
    "up": "project up",
    "up-all": "project up -a"
}'
```

You can now use aliases to run pydev commands:

```bash
pydev up     # instead of pydev project up
pydev up-all # instead of pydev project up -a
```

You can write alias with arguments too:

```yaml
aliases:
  migrate: project run "python manage.py migrate $app"
```

You can now run migration to the concrete app with:

```bash
pydev migrate --app=users # it will run pydev project run "python manage.py migrate users"
```

There can be situations when you need to alias on more commands. You can use a list to specify it:

```yaml
aliases:
  build-js:
    - project run -c static "build-js.sh"
    - project run -c base "./manage.py collectstatic"
```

Now you can run only one pydev command to build js and collect django static files:

```bash
pydev buildjs
```

An alias can describe itself. Instead of the command write an object with the `description` and the `command` key, where the command is again either a single command or a list of them:

```yaml
aliases:
  build-js:
    description: Build js and collect django static files
    command:
      - project run -c static "build-js.sh"
      - project run -c base "./manage.py collectstatic"
```

The description is then shown next to the alias in `pydev --help` and above the list of the aliased commands in `pydev build-js --help`:

```bash
Aliases:
  build-js  Build js and collect django static files
```

Everything what you define after pydev alias will be sent into the command:

```yaml
aliases:
  run-dj: project run "./manage.py"
```

```bash
pydev run-dj migrate --no-input
```

Will run `./manage.py migrate --no-input` on the docker instance.

# Commands

For list of all commands you can run command:

```bash
pydev --help
```

### Bitbucket

Group of commands or helpers which provides manage pull requests in the Bitbucket service.

#### Commands
* `pydev bitbucket create-release-pull-request` - create a new pull request to the bitbucket repository

#### Configuration
* `BITBUCKET_USERNAME` - username fo the Bitbucket service
* `BITBUCKET_PASSWORD` - password to the Bitbucket service
* `BITBUCKET_BRANCH_NAME` - pull request destination branch name
* `BITBUCKET_REPOSITORY` - name of the destination repository

### Docker

Docker utilities helping with managing docker in your project:

#### Commands
* `pydev docker login` - login to the docker registry
* `pydev docker tag` - tag a docker image
* `pydev docker push-image` - push image to the docker repository

### Git

Helpers to run git commands which simplify releasing.

#### Commands
* `pydev git create-release` - bump the version file, create a release branch, tag it and push both to the remote repository if the remote name is specified
* `pydev git create-release-branch` - create a release branch and push it to the remote repository if the remote name is specified
* `pydev git create-deployment-branch` - create a deployment branch and new commit to trigger a deployment event
* `pydev git checkout-to-release-branch` - checkout git repository back to the release branch from deployment branch
* `pydev git bump-version-from-release-branch` - get version defined in the release branch and bump version files
* `pydev git commit-version` - commit version files and add git tag to the commit
* `pydev git merge-release-branch` - merge current branch to the selected branch
* `pydev git init-hooks` - initialize git hooks defined in the directory ./.pydev/git/hooks

The `create-release` and `create-release-branch` commands accept `--release-prefix/-P` (default from `RELEASE_PREFIX`). When set, the prefix is joined to the version with `@`, so a monorepo can release multiple packages without collisions:

| | with `--release-prefix habarico` | without prefix |
|---|---|---|
| branch | `release/habarico@v1.3` | `release/v1.3` |
| tag | `habarico@1.3.0` | `1.3.0` |

#### Configuration
* `GIT_REMOTE_NAME` - remote repository name
* `GIT_BRANCH_NAME` - source branch name for create-release-branch command
* `RELEASE_PREFIX` - package/component prefix for the release branch name and tag (see `--release-prefix`)

### Gitlab

Helpers for GitLab management.

#### Commands
* `pydev gitlab create-release-merge-request` - create a new merge request from the current branch in a GitLab project. It is often used after the project release
* `pydev gitlab create-merge-request` - create a new merge request with the given title, optionally with automerge and removal of the source branch
* `pydev gitlab activate-merge-request-automerge` - merge an existing merge request as soon as its pipeline succeeds
* `pydev gitlab run-job` - run a pipeline of the selected branch, optionally with variables (`--variables "A=1,B=2"`)
* `pydev gitlab get-project-id` - print the numeric ID of the GitLab project
* `pydev gitlab create-release-record` - create a GitLab release object for an already existing release tag

The `create-release-record` command hangs a [GitLab release](https://docs.gitlab.com/user/project/releases/) on a tag created by `pydev git create-release`. The tag stays the source of truth for publishing, the release is metadata on top of it, so that "what is released" is readable per project and across the group:

```bash
pydev gitlab create-release-record --tag-name "$CI_COMMIT_TAG" \
    --description "$(sed -n '/^## \[1.3.0\]/,/^## /p' CHANGELOG.md)" \
    --asset "Wheel=https://pypi.example.com/pkg-1.3.0.whl"
```

* the release name defaults to the tag with the monorepo prefix separated by a space (`lib-a@1.3.0` -> `lib-a 1.3.0`), `--name` overrides it
* `--description` is passed in by the caller, usually as the `CHANGELOG.md` section of the released version
* `--asset "name=URL"` can be repeated for every published artifact (package in the registry, image tag)
* in a CI job neither a token nor a project has to be set up: `CI_JOB_TOKEN` and `CI_COMMIT_TAG` are read from the environment and `--project` accepts the numeric `$CI_PROJECT_ID`. Only `--url` has to be passed as `$CI_SERVER_URL` when `GITLAB_API_URL` is not set
* a request rejected by GitLab (an already existing release among them) ends the command with an error. A release is an addition, not a critical path, so let the release job keep going with `|| true`

#### Configuration
* `GITLAB_API_URL` - URL to gitlab API service
* `GITLAB_PROJECT` - GitLab project name
* `GITLAB_TARGET_BRANCH` - target branch for the merge request
* `GITLAB_TOKEN` - GitLab authentication token, sent as the `PRIVATE-TOKEN` header
* `CI_JOB_TOKEN` - CI job token of the running job, sent as the `JOB-TOKEN` header and preferred over `GITLAB_TOKEN`. Accepted by `create-release-record` only, because the endpoints of the other commands do not take a job token
* `CI_COMMIT_TAG` - tag of the release for the `create-release-record` command

### QA

Helpers for python project quality assurance.

#### Commands
* `pydev qa all` - run all QA checks
* `pydev qa missing-migrations` - run a missing Django migrations QA check. It will try to generate a Django migrations if there is one or more missing check is failed
* `pydev qa migration-filenames` - run migration filenames QA check. Migration name should be in format "[0-9]{4}_migration.py" (ex. 0001_migration.py)
* `pydev qa missing-translations` - run missing translations QA check. It will try to generate a Django `makemessages` if there is one or more missing check is failed
* `pydev qa import-order` - run import order QA check. It will check if all the new python code imports have the right order defined with isort command
* `pydev qa unused-imports` - run unused imports QA check. It will check if the new python code does not contain unused imports
* `pydev qa test-method-names` - runs test method names QA check. It will check if the new test methods has the right name in format defined in `QA_DISALLOWED_TEST_METHOD_REGEXP` setting

#### Configuration
* `QA_DISALLOWED_TEST_METHOD_REGEXP` - your disallowed test method regex. Example: `"def (test_should_[^\(]+)\("` 

### Version

Helpers for automatic update version in the version file.

#### Commands
* `pydev version bump-to-next` - bump version in the JSON file (or files) and print it
* `pydev version print` - return current project version according to version JSON file
* `pydev version print-next` - return next version according to input release type, build hash and version JSON file

#### Configuration
* `VERSION_FILES` - path to the files containing information about project version. There can be more paths which are split with character `,`.
* `VERSION_FILE_TYPE` - explicit version file type (`toml`, `json`, `npm`, `xml`, `text`). When not set, the type is detected from the file extension; a file without an extension (e.g. a Ruby-style plain text `VERSION` file that contains only the version number) defaults to `text`. Can also be passed with the `--file-type` option.

### Sh

You can run shell commands with this utility:

* `pydev sh ls` 

The utility can be used in the `ALIASES`.

### ecs

Helpers for AWS ECS management.

#### Commands
* `pydev ecs deploy-new-task-definition` - deploy new task definition in AWS ECS. This command also updates the service and forces new deployment
* `pydev ecs register-new-task-definition` - register new task definition in AWS ECS
* `pydev ecs update-service-to-latest-task-definition` - update service with the latest available task_definition
* `pydev ecs stop-service` - stop an AWS ECS service by updating its desiredCount to 0
* `pydev ecs start-service` - start an AWS ECS service by updating its desiredCount to 0
* `pydev ecs start-cluster-services` - start an AWS ECS service by updating its desiredCount to 0
* `pydev ecs start-services` - start an AWS ECS service by updating its desiredCount to 0
* `pydev ecs run-task` - run a single task in AWS ECS
* `pydev ecs run-task-and-wait-for-success` - run a single task in AWS ECS and wait for it to stop with success
* `pydev ecs run-service-task` - run a single task based on service's task definition in AWS ECS and wait for it to stop with success
* `pydev ecs run-service-task-fargate` - run a single task based on service's task definition in AWS ECS and wait for it to stop with success
* `pydev ecs get-tasks-for-service` - return list of tasks running under specified service
* `pydev ecs get-task-definition-for-service` - return task definition arn for specified service
* `pydev ecs stop-service-and-wait-for-tasks-to-stop` - stop service and wait for the tasks to stop
* `pydev ecs stop-services-and-wait-for-tasks-to-stop` - stop services and wait for the tasks to stop
* `pydev ecs get-services-names` - get names of the cluster services
* `pydev ecs redeploy-services` - redeploy services by forcing new service deployment
* `pydev ecs redeploy-cluster-services` - redeploy all cluster services by forcing new service deployment
* `pydev ecs wait-for-services-stable` - wait until all non-daemon services in cluster are stable

#### Configuration
* `AWS_REGION` - your AWS region
* `AWS_ECS_CLUSTER` - name of your AWS ECS cluster

### Jira

Helpers for Jira management.

#### Commands
* `pydev git my-issues` - list issue of the current user according to JQL query
* `pydev git get-branch-name` - generates a new branch name according to the Jira issue
* `pydev git show-issue` - open an issue with the key in a web browser
* `pydev git log-issue-time` - log a spend time to the issue in the Jira worklog
* `pydev git print-issue-worklog` - print Jira issue worklog
* `pydev git invoke-issues-transition` - invoke Jira transition defined with transition name which may change Jira issues status.

#### Configuration
* `JIRA_URL` - URL to the Jira application
* `JIRA_USERNAME` - Username for access to the Jira
* `JIRA_API_KEY` - API key to the Jira
* `JIRA_JQL` - JQL for issue listing. Defaut value is:
```JQL
'project = {project_key} and assignee = currentUser() and status not in ("Done", "Canceled", "Closed") and sprint != null order by created DESC'
```
* `JIRA_PROJECT_KEY` - key of the project in the Jira

### Toggle

Helpers for time tracking with toggle service.

#### Commands
* `pydev toggle start` - start the Toggle timer with description in workspace ID and project ID
* `pydev toggle stop` - stops currently running the Toggl timer
* `pydev toggle print-toggl` - print detail of the currently running Toggle timer
* `pydev toggle print-report` - print summary from the Toggle time tracker. Data can be filtered according to workspace, project, description or by date and time
* `pydev toggle print-report-tasks` - print tracked the Toggle tasks one by one in table. Data can be filtered according to workspace, project, description or by date and time

#### Configuration
* `TOGGL_API_KEY` - the Toggle access API key
* `TOGGL_PROJECT_ID` - the Toggle project identifier
* `TOGGL_WORKSPACE_ID` - the Toggle workspace ID

### Project

Group of commands which joins previous commands for simplifier development.

#### Commands
* `pydev project set-domain` - set local hostname translation to localhost. It updates `/etc/hosts` headers according to input domain
* `pydev project build` - build docker container via compose file and name it with the project name
* `pydev project run` - run one time command on docker container defined in compose file. You can specify image environment variables
* `pydev project exec-command` - run command on already executed docker container defined in compose file
* `pydev project up` - builds, (re)creates, starts, and attaches to containers for a service defined in compose file
* `pydev project stop` - stop all the docker containers from the compose file
* `pydev project install` - builds, (re)creates, starts, and attaches to containers for a service defined in compose file
* `pydev project copy-container-dirs` - copy directories from a docker image to the host
* `pydev project bind-library` - mount a directory to an another location on docker host with bindfs
* `pydev project kill-all` - kill all running docker instances
* `pydev project clean` - clean docker images and its volumes. If you want to clean all the images and volumes use parameter -a
* `pydev project task start` - get information from Jira about issue and start Toggle timer
* `pydev project task stop` - stop Toggle timer and logs time to the Jira issue
* `pydev project task create-branch-from-issue` - create a new git branch from the source branch with name generated from the Jira issue
* `pydev project task create-or-update-pull-request` - create a Bitbucket pull request named according to the Jira issue
* `pydev project task sync-timer-log-to-issues` - synchronize logged time in Toggle timer with issues worklog in Jira
* `pydev project task print-last-commit-build` - print the last commit test results in Bitbucket of the selected branch

#### Configuration
* `PROJECT_DOCKER_COMPOSE_PROJECT_NAME` - project name which be used as a prefix aff all builded docker images with compose
* `PROJECT_DOCKER_COMPOSE_FILES` - list of the paths to the compose files separated by a character `,`
* `PROJECT_DOMAINS` - list of the project domain names separated by a character `,`
* `PROJECT_DOCKER_COMPOSE_CONTAINERS` - list of all your docker container names separated by a character `,`
* `PROJECT_DOCKER_COMPOSE_DEFAULT_UP_CONTAINERS` - list of docker container names separated by a character `,` which will be by default started with command `pydev project up`
* `PROJECT_DOCKER_COMPOSE_VAR_DIRS` - list of your variable directories which will be created before project build
* `PROJECT_DOCKER_COMPOSE_CONTAINERS_DIR_TO_COPY` - container dir which will be copied after build in format `DOCKER_CONTAINER_NAME:CONTAINER_DIRECTORY:HOST_DIRECTORY`
* `PROJECT_DOCKER_COMPOSE_CONTAINERS_INSTALL_COMMAND` - install that will be triggered after project build
* `PROJECT_LIBRARY_DIR` - the project library directory path on the docker host
* `PROJECT_SOURCE_BRANCH_NAME` - your project source branch name. By default value is set to `next`

### Slack

Anything what you need to send to Slack.

#### Commands
* `pydev slack upload-new-migrations` - send all new Django migration files between releases etc. to Slack

#### Configuration
* `SLACK_BOT_TOKEN` - Token to connect Slack. The [link](https://github.com/slackapi/python-slack-sdk/blob/main/tutorial/01-creating-the-slack-app.md) how to obtain the token.
