Metadata-Version: 2.1
Name: stage-foundary
Version: 0.1.6
Summary: Foundary of standard stages
Home-page: https://gitlab.opencode.de/oc00006954184/stage-foundary
Author: Reiner Jung
Author-email: reiner.jung@stk.landsh.de
License: EUPL-1.2
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: European Union Public Licence 1.2 (EUPL 1.2)
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jira

# Stage Foundary

Processing data with a pipe and filter (stages) framework is a common approach. This stage foundary contains a collection
of commonly used stages to read CSVs, XLSXs, Jira-tickets and other sources and search filter and combine data
to create reports. They follow the basic layout of stages from the TeeTime Java and C++ framework. This has been
chosen to be able to later port it to the Python implementation of TeeTime.

The following contains general information on the stage foundary.

## Key concepts

### ProducerStage

Each pipeline must contain at least one `ProducerStage`

```
class MyProducerStage(ProducerStage):

    def __init__(self, logger: logging.Logger):
        super().__init__(logger)

    def execute(self):
        self.output_port.send(EVENT TO BE SENT)
        # to mark the end of processing
        self.work_complete()
```

### ConsumerStage

In pipeline single stages have only one input and one output port. To ease the implementation,
the `ConsumerStage` supports this processing style

```
class MyConsumerStage(ConsumerStage):

    def __init__(self, logger: logging.Logger):
        super().__init__(logger)

    # element is the incoming event
    def consume(self, element: any) -> None:
        self.output_port.send(EVENT TO BE SENT)        
```

### SinkStage

The end of a pipeline conains a `SinkStage`. They do not posses an output port.

```
class MySinkStage(SinkStage):

    def __init__(self, logger: logging.Logger):
        super().__init__(logger)

    # element is the incoming event
    def consume(self, element: any) -> None:
```

### Basic program

```
import argparse as arg
import logging
import datetime

import stage_foundary.teetime

#
# Setup logger
#
logger = logging.getLogger(__file__)
logging.basicConfig(level=logging.INFO)

# often one needs the current date
date = datetime.datetime.now()

#
# parameter
#

parser = arg.ArgumentParser(NAME OF THE TOOL)

parser.add_argument("-i", dest="input_path", required=True)

# check parameters here

#
# create stages
#
class Configuration(teetime.Configuration):

    def create_configuration(self):
        producer = MyProducerStage(logger)
        consumer = MyConsumerStage(logger)
        sink = MySinkStage(logger)

        # connect stages
        self.connect(producer, consumer)
        self.connect(consumer, sink)


execution = Execution()
execution.run(FuPoStatsConfiguration())

```

Stages must be connected to each other. Stages with only one input and one output port
can be connected by handing the stages to the `connect` method. Other stages, e.g.,
the `split` stage has multiple output ports, here you must give the correct output port
as parameter to the connect method, e.g., `split.get_port(0)`.

## CI and CD

- Development is conducted in the `main` branch or, for larger new features, in a
  dedicated feature branch.
- The release train should be conducted as follows:
  - make sure all features are merged into the main branch
  - determine new version number
  - create a release branch
  - add infos to the changelog on changes
  - update version number in project.toml and setup.py, use `hatch version x.y.z`
  - push branch
  - the CI will:
    - build and test the package
    - run integration tests
    - deploy to testpypi
    - sign
    - deploy to pypi
  - Tag release
  - Merge release branch back to main and delete branch

## Integrate with your tools

* [Set up project integrations](https://gitlab.opencode.de/oc00006954184/stage-foundary/-/settings/integrations)

## Collaborate with your team

* [Invite team members and collaborators](https://docs.gitlab.com/user/project/members/)
* [Create a new merge request](https://docs.gitlab.com/user/project/merge_requests/creating_merge_requests/)
* [Automatically close issues from merge requests](https://docs.gitlab.com/user/project/issues/managing_issues/#closing-issues-automatically)
* [Enable merge request approvals](https://docs.gitlab.com/user/project/merge_requests/approvals/)
* [Set auto-merge](https://docs.gitlab.com/user/project/merge_requests/auto_merge/)

## Test and Deploy

Use the built-in continuous integration in GitLab.

* [Get started with GitLab CI/CD](https://docs.gitlab.com/ci/quick_start/)
* [Analyze your code for known vulnerabilities with Static Application Security Testing (SAST)](https://docs.gitlab.com/user/application_security/sast/)
* [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/topics/autodevops/requirements/)
* [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/user/clusters/agent/)
* [Set up protected environments](https://docs.gitlab.com/ci/environments/protected_environments/)

***

## Authors and acknowledgment

Reiner Jung

## License

The code is licensed under the EUPL-1.2

## Project status

We will add to this every time a stage is used in multiple analyses to gradually build reusable parts and
reduce the effort necessary to create analyses in project managing and rollout coordination-


