Metadata-Version: 2.1
Name: aina
Version: 0.1.0
Summary: Let the logs flow
Home-page: https://github.com/ilovetux/aina
Author: iLoveTux
Author-email: cliffbressette@gmail.com
License: GNU General Public License v3
Keywords: aina
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Requires-Dist: Click (>=6.0)

=====
aina
=====


.. image:: https://img.shields.io/travis/iLoveTux/aina.svg
        :target: https://travis-ci.org/ilovetux/aina

.. image:: https://readthedocs.org/projects/aina/badge/?version=latest
        :target: https://aina.readthedocs.io/en/latest/?badge=latest
        :alt: Documentation Status


Let the logs flow

aina is a general-purpose stream processing framework. It includes
a simple but powerful templating system and all the utilities
to shape your data streams.

NOTE: This is new code. Master is in flux and docs are lacking,
but it is in a point where it could be useful to someone. If
it is useful to you, help us get to 1.0.0. You can start by reading
the contributing guide at https://github.com/ilovetux/aina/CONTRIBUTING.rst.

* Free software: GNU General Public License v3
* Documentation: https://aina.readthedocs.io.


Features
--------

* Simple, Powerful templating system
* Extensible input and output system
* Command line utilities
* TODO: Web GUI
* TODO: Many default use cases covered
* TODO: --no-overwrite option
* TODO: Improve test coverage

Installing
----------

Currently the only way to install this package is to clone it which
should look like the following::

  $ git clone https://github.com/ilovetux/aina
  $ cd aina

Then you can run the tests::
  $ python setup.py test

And if they pass (fingers crossed!), go ahead with::

  $ pip install .

Concepts
--------

The built-in templating engine is very simple, it consists
of a namespace and a template. The template is rendered within
the context of the namespace.

Rendering involves two stages:

  1. scanning the template for strings matching the pattern `{%<Expression>%}`
     where `<Expression>` is Python source code which is executed (`exec`)
     within the context of the namespace and removed from the output.
  2. scanning the remaining output for strings matching the pattern
     `{{<Statement>}}` where `<Statement>` is a Python statement which
     is replaced (along with `{{` and `}}`) with the value to which
     it evaluates (`eval`)

This concept is applied to a variety of use cases and embodied in the form of
command line utilities which cover a number of common use cases.

Usage
=====

The command line utility, aina, can be run in two modes:

  1. Streaming mode: Data is streamed through and used to populate templates
  2. Document mode: Render files src and write the results to dst

Streaming mode
--------------

Streaming mode runs in the following manner:

  1. reads data from `filenames`, which defaults to stdin
  2. At this point any expressions passed to `--begins` are executed
  3. The files specified are processed as follows in order
    1. Any expressions passed to `--begin-files` are executed
    2. The data from the current file is read line-by-line
      1. Any statements passed to `--tests` are evaluated
      2. Iff all tests pass, the following process is performed.
        1. Any expressions passed to `--begin-lines` are executed
        2. Any templates are rendered through the python logging system
      3. Any expressions passed to `--end-lines` are executed
    3. Any expressions passed to `--end-files` are executed
  4. Any expressions passed to `--ends` are executed

Below are a few examples. See the documentation for more details::

  $ # Like grep
  $ aina stream --test "'error' in line.lower()" --template "{{line}}" *.log
  $ # Like wc -l
  $ aina stream --end-files "print(fnr, filename)" *.log
  $ # Like wc -wl
  $ aina stream --begins "words=0" --begin-lines "words += nf" --end-files "print(words, fnr, filename)"
  $ # Find the count of numbers "\d+" for each line
  $ aina stream --begins "import re" --begin-lines "print(re.findall(r'\d+', line))" *.log

Please see the documentation for more as well as trying::

  $ aina stream --help

Important Note:

If anything passed to any of the hooks is determined to exist by `os.path.exists`
then it will be read and executed as if that text was passed in on the CLI. This
is useful for quickly solving character escaping issues.

Document mode
-------------

Document mode runs tries to render a group of files from one location
to another. It is used like this::

  $ aina doc <src> <dst>

There are options to control behavior, but the gist of it is:

  1. if src is a file
    1. if dst is a filename, src is rendered and written to dst
    2. if dst is a directory, src is rendered and written to a file in dst with the same basename as src
  2. if src is a directory
    1. dst must be a directory and every file in src is rendered into a file in dst with the same basename as the file from src
    2. If `--recursive` is specified, the subdirectories will be reproduced in dst

Some important notes:

* File and directory names can be templated
* If `--interval` is passed an integer value, the program will sleep for that many seconds and check for changes to your templates in which case they will be re-rendered

Use Cases
---------

Streaming mode is great for processing incoming log files with `tail --follow=name`
or for ad-hoc analysis of text files.

Document mode is incredibly useful for a powerful configuration templating
system. The `--interval` option is incredibly useful as it will only re-render
on a file change, so is great for developing your templates as you can view
the results in real-time.

Document mode is also useful for near-real-time rendering of static
web resources such as charts, tables, dashboards and more.



Credits
-------

Author: iLoveTux
This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.

.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage


=======
History
=======

0.1.0 (2018-06-20)
------------------

* First release on PyPI.


