Metadata-Version: 2.1
Name: ampeg
Version: 0.1.1
Summary: A simple and lightweight package for parallel computing
Home-page: https://github.com/sjpet/ampeg
Author: Stefan Peterson
Author-email: stefan.peterson@rubico.com
License: GPL-3.0
Download-URL: https://github.com/sjpet/ampeg/tarball/0.1.1
Description: Ampeg
        =====
        
        Ampeg is a simple and lightweight package for parallel computation. It provides
        simple functions for scheduling and execution of a set of dependent or
        independent computational tasks over multiple processes using the 
        ``multiprocessing`` package.
        
        Requirements
        ------------
        
        Python 2.7 or later
        
        Python 3.4 or later
        
        Installation
        ------------
        
        pip install ampeg
        
        Usage
        -----
        
        Ampeg exposes a scheduling function ``earliest_finish_time``, an execution
        function ``execute_task_lists`` and a ``Dependency`` class. The former takes a
        directed acyclic graph (DAG) and a number of processes and produces a set of
        task lists for each process and a corresponding set of task IDs for translating
        the execution result. These two form the input to ``execute_task_lists``, which
        returns a dict with the result of each task in the original graph.
        
        The DAG is represented by a python dict of vertices where each key is the ID of
        a task and each value is a triple of (function, args or kwargs, computation
        cost). Edges are implicitly defined by instances of the ``Dependency`` class in
        the args or kwargs.
        
        A simple usage example computing (3^2 + 4^2) - (3^2 * 10/2):
        
        ```
        >>> import ampeg as ag
        >>> n_processes = 3
        >>> my_graph = {0: (lambda x: x**2, 3, 10.8),
                        1: (lambda x: x**2, 4, 10.8),
                        2: (lambda x: x/2, 10, 11),
                        3: (lambda x, y: x + y, (ag.Dependency(0, None, 1),
                                                 ag.Dependency(1, None, 1), 10.7),
                        4: (lambda x, y: x*y, (ag.Dependency(0, None, 1),
                                               ag.Dependency(2, None, 1)), 10.8),
                        5: (lambda x, y: x - y, (ag.Dependency(3, None, 1),
                                                 ag.Dependency(4, None, 1)), 10.9)}
        >>> task_lists, task_ids = ag.earliest_finish_time(my_graph, n_processes)
        >>> ag.execute_task_lists(task_lists, task_ids)
        {0: 9, 1: 16, 2: 5, 3: 25, 4: 45, 5: -20}
        ```
        
        The Dependency class
        --------------------
        
        A dependency is a triple of (task ID or index, key (if any) and communication
        cost). The key may be a single key, index or slice, or it may be an iterable of
        such values to be applied in sequence. For example, the key ``('values', 2)``
        extracts the value 5 from the dict ``{'values': [1, 3, 5]}``. Dependency
        instances are created by ``ampeg.Dependency(task, key, cost)`` where cost is
        optional and defaults to 0.
        
        Exceptions
        ----------
        
        Ampeg catches exceptions raised by individual tasks, returning them as results
        encapsulated in the ``Err`` class. When an ``Err`` instance is found among the
        dependencies for a task, the result for this task will be an ``Err`` instance
        encapsulating a ``DependencyError``.
        
        Windows
        -------
        Note that under Windows, the functions and their arguments must all be 
        picklable.
        
Keywords: machine learning data mining out-of-memory
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License (GPL)
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Topic :: Utilities
Provides-Extra: dev
