Metadata-Version: 2.4
Name: hyrrokkin
Version: 0.2.0
Summary: A lightweight asynchronous DAG execution engine for Python and other languages
Author-email: Visual Topology <dev@visualtopology.com>
License-Expression: MIT
Project-URL: Homepage, https://hyrrokk.in
Project-URL: Documentation, https://hyrrokk.in
Project-URL: Repository, https://codeberg.org/visual-topology/hyrrokkin
Project-URL: Issues, https://codeberg.org/visual-topology/hyrrokkin/issues
Project-URL: Changelog, https://codeberg.org/visual-topology/hyrrokkin/src/branch/main/CHANGELOG.md
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: JavaScript
Classifier: Operating System :: POSIX :: Linux
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Development Status :: 4 - Beta
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: webapps
Requires-Dist: moonlette>=0.2.0; extra == "webapps"
Requires-Dist: psutil; extra == "webapps"
Provides-Extra: yaml
Requires-Dist: pyyaml; extra == "yaml"
Provides-Extra: validation
Requires-Dist: jsonschema; extra == "validation"
Provides-Extra: all
Requires-Dist: pyyaml; extra == "all"
Requires-Dist: jsonschema; extra == "all"
Requires-Dist: moonlette>=0.2.0; extra == "all"
Requires-Dist: psutil; extra == "all"
Dynamic: license-file

# hyrrokkin

```
 _                               _     _     _
| |__   _   _  _ __  _ __  ___  | | __| | __(_) _ __
| '_ \ | | | || '__|| '__|/ _ \ | |/ /| |/ /| || '_ \
| | | || |_| || |   | |  | (_) ||   < |   < | || | | |
|_| |_| \__, ||_|   |_|   \___/ |_|\_\|_|\_\|_||_| |_|
        |___/
```

Hyrrokkin is a lightweight directed acyclic graph (DAG) framework API/designer for running graphs implemented with code written in Python and Typescript/Javascript 

In Hyrrokkin, a directed acyclic graph is termed a `topology` and is composed of `nodes` and `links`.

## Topology Python API

Use a python API to construct and run topologies (DAG)

```python

from hyrrokkin.api import Topology

# create a topology, stored in my_topology sub-folder 
t = Topology(topology_folder='my_topology',include_package_list=["hyrrokkin.packages.textgraph"])

# define the topology to analyse word frequencies with 3 nodes and 2 links

t.add_node("n0", "textgraph:text_input_node", data={"value":b"This is some simple text"})
t.add_node("n1", "textgraph:word_frequency_node", properties={"threshold":1})
t.add_node("n2", "textgraph:table_display_node")

t.add_link("l0", "n0", "data_out", "n1", "data_in")
t.add_link("l1", "n1", "data_out", "n2", "data_in")

# create and link to an output port at which we can listen for results
t.add_port("output_word_frequencies",direction="output")
t.add_link("l2", "n1", "data_out", "", "output_word_frequencies")

# intercept output from the output_word_frequency port and print it
t.add_output_listener("output_word_frequencies", lambda v:print(v))

# run the topology and wait for all nodes to complete, check no errors were returned
assert(t.run() == {}) 

# close the topology
t.close()
```

## Topology Designer UI

The same topology can be created and run within the web-based designer application

![Topology Designer](docs/src/images/topology.png)

## Hyrrokkin - Main Features

Hyrrokkin provides a framework enabling a user to:

* Define packages of nodes using the python or typescript/javascript Package API
* Create topologies which link nodes from these packages together:
  * using a text editor to create a YAML file
  * using the python Topology API
  * using the Command Line Interface
  * using a graphical design tool
* Run topologies using the design tool, python Topology API or CLI
* Create and deploy applications - web based interfaces for end-users based on a topology (but hiding the topology structure)

Hyrrokkin is licensed under the (MIT open source license)[LICENSE.txt].

> NOTE: This repo provides a very simple [textgraph example package](src/hyrrokkin/static/packages/textgraph) with implementations in python and javascript.  Hyrrokkin is designed to support packages created for specific domains and does not try to provide general purpose package for working with any kind of data out of the box.  Instead, packages should be developed separately and can be `pip` installed into the environment where hyrrokkin is installed.

## Installation Options

### (1) Installation into a python environment

It is recommended to create a fresh python environment (hyrrokkin requires python 3.11 or later).

One option is to use `conda` from conda-forge:

Conda can be installed from https://github.com/conda-forge/miniforge

Once conda is installed, create an environment and activate it

```
conda create -n hyrrokkin_env python=3.11
conda activate hyrrokkin_env
```

Once an environment is set up, install hyrrokkin into it using pip or from source code, choose either option 1.1 or 1.2 below:

### (1.1) Installing Hyrrokkin into a python environment via pip

Hyrrokkin is tested on linux and requires python >= 3.11

Installation without any dependencies (no support for YAML import/export, JSON schema checking, or web-based UI):

```
pip install hyrrokkin
```

Installation with optional dependencies for YAML import/export:

```
pip install hyrrokkin[YAML]
```

Installation with optional dependencies for JSON schema validation:

```
pip install hyrrokkin[VALIDATION]
```

Installation with optional dependencies for UI webapps (designer, directory, applications):

```
pip install hyrrokkin[WEBAPPS]
```

Installation with all optional dependencies

```
pip install hyrrokkin[ALL]
```

### (1.2) Installing Hyrrokkin from source code

Clone this repo and install it into the environment

```
git clone https://codeberg.org/visual-topology/hyrrokkin.git
cd hyrrokkin
pip install -e ".[ALL]"
```

If installing into a development environment, you may wish to install dependencies only:

```
git clone https://codeberg.org/visual-topology/hyrrokkin.git
cd hyrrokkin
pip install .[ALL] --only-deps
```

### (2) Quick Installer **Experimental**

Use this option to install hyrrokkin, python and the pixi package manager (https://pixi.prefix.dev/) into an isolated sub-directory.

```
mkdir hyrrokkin_install
cd hyrrokkin_install
curl -fsSL https://codeberg.org/visual-topology/hyrrokkin/raw/branch/main/installer/quick_install_linux.sh | sh
```

This will install the following:

```
hyrrokkin_install\
   hyrrokkin                ... launch hyrrokkin user interface 
   python                   ... the python executable that was installed
   dist\                    ... a pixi default environment is where python, hyrrokkin and dependent packages are installed        
```


## Rebuilding hyrrokkin

Run the following command to rebuild hyrrokkin.  This step is needed if any of the typescript files are modified, and compiles the updated typescript files to javascript.

To rebuild you'll need to install the typescript compiler (tsc) into your environment, perhaps using `conda install typescript`

```
python -m hyrrokkin.tools.dev.rebuild
```

## Running tests

These cover the hyrrokkin CLI and API, the python engine API, and the execution of simple textgraph-based topologies in various python, typescript or javascript engine configurations

Deno needs to be installed (see https://deno.com/) for testing with the typescript and javascript engines.  Deno can be installed into a conda environment using `conda install deno`.

Run the tests using:

```
python -m hyrrokkin.tools.dev.test
```

### Building Documentation

Documentation is built using zensical which can be installed using:

```
pip install zensical mkdocstrings[python]
```

To build documentation:

```
cd docs
./build.sh
```

Documentation is built to `docs/built/index.html`







