Metadata-Version: 2.0
Name: argcmdr
Version: 0.3.0
Summary: Thin argparse wrapper for quick, clear and easy declaration of hierarchical console command interfaces
Home-page: https://github.com/dssg/argcmdr
Author: Center for Data Science and Public Policy
Author-email: datascifellows@gmail.com
License: MIT
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.6
Requires-Python: >=3.6
Requires-Dist: Dickens (==1.0.1)
Requires-Dist: plumbum (==1.6.4)

=======
argcmdr
=======

The thin ``argparse`` wrapper for quick, clear and easy declaration of (hierarchical) console command interfaces via Python.

``argcmdr``:

* handles the boilerplate of CLI

  * while maintaining the clarity and extensibility of your code
  * without requiring you to learn Yet Another argument-definition syntax
  * without reinventing the wheel or sacrificing the flexibility of ``argparse``

* enables invocation via

  * executable script (``__name__ == '__main__'``)
  * ``setuptools`` entrypoint
  * command-defining module (like the ``Makefile`` of ``make``)

* determines command hierarchy flexibly and cleanly

  * command class declarations are nested to indicate CLI hierarchy *or*
  * commands are decorated to indicate their hierarchy

* includes support for elegant interaction with the operating system, via ``plumbum``

Setup
=====

``argcmdr`` is developed for Python version 3.6.3 and above, and may be built via ``setuptools``.

Python
------

If Python 3.6.3 or greater is not installed on your system, it is available from python.org_.

However, depending on your system, you might prefer to install Python via a package manager, such as Homebrew_ on Mac OS X or APT on Debian-based Linux systems.

Alternatively, pyenv_ is highly recommended to manage arbitrary installations of Python, and may be most easily installed via the `pyenv installer`_.

argcmdr
-------

To install from PyPI::

    pip install argcmdr

To install from Github::

    pip install git+https://github.com/dssg/argcmdr.git

To install from source::

    python setup.py install

Tutorial
========

The Command
-----------

``argcmdr`` is built around the base class ``Command``. Your console command extends ``Command``, and optionally defines:

* ``__init__(parser)``, which adds to the parser the arguments that your command requires, as supported by ``argparse`` (see argparse_)
* ``__call__([args, parser, ...])``, which is invoked when your console command is invoked, and which is expected to implement your command's functionality

For example, let's define the executable file ``listdir``, a trivial script which prints the current directory's contents::

    #!/usr/bin/env python

    import os

    from argcmdr import Command, main

    class Main(Command):
        """print the current directory's contents"""

        def __call__(self):
            print(*os.listdir())

    if __name__ == '__main__':
        main(Main)

Should we execute this script, it will perform much the same as ``ls -A``.

Let's say, however, that we would like to optionally print each item of the directory's contents on a separate line::

    class Main(Command):
        """print the current directory's contents"""

        def __init__(self, parser):
            parser.add_argument(
                '-1',
                action='store_const',
                const='\n',
                default=' ',
                dest='sep',
                help='list one file per line',
            )

        def __call__(self, args):
            print(*os.listdir(), sep=args.sep)

We now optionally support execution similar to ``ls -A1``, via ``listdir -1``.

Fittingly, this is reflected in the script's autogenerated usage text – ``listdir -h`` prints::

    usage: listdir [-h] [--tb] [-1]

    print the current directory's contents

    optional arguments:
      -h, --help         show this help message and exit
      --tb, --traceback  print error tracebacks
      -1                 list one file per line

Local execution
---------------

As much as we gain from Python and its standard library, it's quite typical to need to spawn non-Python subprocesses, and for that matter for your script's purpose to be entirely to orchestrate workflows built from operating system commands. Python's – and argcmdr's – benefit is to make this work easier, debuggable, testable and scalable.

In fact, our above, trivial example could be accomplished easily with direct execution of ``ls``::

    import argparse

    from argcmdr import Local, main

    class Main(Local):
        """list directory contents"""

        def __init__(self, parser):
            parser.add_argument(
                'remainder',
                metavar='arguments for ls',
                nargs=argparse.REMAINDER,
            )

        def __call__(self, args):
            print(self.local['ls'](args.remainder))

``local``, bound to the ``Local`` base class, is a dictionary which caches path look-ups for system executables.

This could, however, still be cleaner. For this reason, the ``Local`` command features a parallel invocation interface, ``prepare([args, parser, ...])``::

    class Main(Local):
        """list directory contents"""

        def __init__(self, parser):
            parser.add_argument(
                'remainder',
                metavar='arguments for ls',
                nargs=argparse.REMAINDER,
            )

        def prepare(self, args):
            return self.local['ls'][args.remainder]

Via the ``prepare`` interface, standard output is printed by default, and your command logic may be tested in a "dry run," as reflected in the usage output of the above::

    usage: listdir [-h] [--tb] [-q] [-d] [-s] [--no-show] ...

    list directory contents

    positional arguments:
      arguments for ls

    optional arguments:
      -h, --help         show this help message and exit
      --tb, --traceback  print error tracebacks
      -q, --quiet        do not print command output
      -d, --dry-run      do not execute commands, but print what they are (unless
                         --no-show is provided)
      -s, --show         print command expressions (by default not printed unless
                         dry-run)
      --no-show          do not print command expressions (by default not printed
                         unless dry-run)

To execute multiple local subprocesses, ``prepare`` may either return an iterable (*e.g.* ``list``) of the above ``plumbum`` bound commands, or ``prepare`` may be defined as a generator function, (*i.e.* make repeated use of ``yield`` – see below).

Inspecting execution
~~~~~~~~~~~~~~~~~~~~

Subprocess commands emitted by ``Local.prepare`` are executed in order and, by default, failed execution is interrupted by a raised exception::

    class Release(Local):
        """release the package to pypi"""

        def __init__(self, parser):
            parser.add_argument(
                'part',
                choices=('major', 'minor', 'patch'),
                help="part of the version to be bumped",
            )

        def prepare(self, args):
            yield self.local['bumpversion'][args.part]
            yield self.local['python']['setup.py', 'sdist', 'bdist_wheel']
            yield self.local['twine']['upload', 'dist/*']

Should the ``bumpversion`` command fail, the ``deploy`` command will not proceed.

In some cases, however, we might like to disable this functionality, and proceed regardless of a subprocess's exit code. We may pass arguments such as ``retcode`` to ``plumbum`` by setting this attribute on the ``prepare`` method::

    def prepare(self, args):
        yield self.local['bumpversion'][args.part]
        yield self.local['python']['setup.py', 'sdist', 'bdist_wheel']
        yield self.local['twine']['upload', 'dist/*']

    prepare.retcode = None

Subprocess commands emitted by the above method will not raise execution exceptions, regardless of their exit code. (To allow only certain exit code(s), set ``retcode`` as appropriate – see plumbum_.)

Having disabled execution exceptions – and regardless – we might need to inspect a subprocess command's exit code, standard output or standard error. As such, (whether we manipulate ``retcode`` or not), ``argcmdr`` communicates these command results with ``prepare`` generator methods::

    def prepare(self, args):
        (code, out, err) = yield self.local['bumpversion']['--list', args.part]

        yield self.local['python']['setup.py', 'sdist', 'bdist_wheel']

        if out is None:
            version = 'DRY-RUN'
        else:
            (version_match,) = re.finditer(
                r'^new_version=([\d.]+)$',
                out,
                re.M,
            )
            version = version_match.group(1)

        yield self.local['twine']['upload', f'dist/*{version}*']

In the above, ``prepare`` stores the results of ``bumpversion`` execution, in order to determine from its standard output the version to be released.

Moreover, we might like to define special handling for execution errors; and, perhaps rather than manipulate ``retcode`` for all commands emitted by our method, we might like to handle them separately. As such, execution exceptions are also communicated back to ``prepare`` generators::

    def prepare(self, args):
        try:
            (_code, out, _err) = yield self.local['bumpversion']['--list', args.part]
        except self.local.ProcessExecutionError:
            print("execution failed but here's a joke ...")
            ...

Command invocation signature
----------------------------

Note that in our last trivial examples of listing directory contents, we made our script dependent upon the ``ls`` command in the operating environment. ``argcmdr`` will not, by default, print tracebacks, and it will colorize unhandled exceptions; however, we might prefer to print a far friendlier error message.

One easy way of printing friendly error messages is to make use of ``argparse.ArgumentParser.error()``. As we've seen, ``Command`` invocation, via either ``__call__`` or ``prepare``, may accept zero arguments, or it may require the parsed arguments ``argparse.Namespace``. Moreover, it may require a second argument, and receive the argument parser::

    class Main(Local):
        """list directory contents"""

        def __init__(self, parser):
            parser.add_argument(
                'remainder',
                metavar='arguments for ls',
                nargs=argparse.REMAINDER,
            )

        def prepare(self, args, parser):
            try:
                local_exec = self.local['ls']
            except self.local.CommandNotFound:
                parser.error('command not available')

            yield local_exec[args.remainder]

If ``ls`` is not available, the user is presented the following message upon executing the above::

    usage: listdir [-h] [--tb] [-q] [-d] [-s] [--no-show] ...
    listdir: error: command not available

Access to the parsed argument namespace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The command invocation's parsed arguments are most straight-forwardly accessible as the first argument of the ``Command`` invocation signature, either ``__call__`` or ``prepare``. However, in less-than-trivial implementations, wherein command methods are factored for reusability, passing the argument namespace from method to method may become tedious. To support such scenarios, this object is made additionally available via the ``Command`` *property*, ``args``.

Consider a class of commands which require a database password. We don't want to store this password anywhere in plain text; rather, we expect it to be input, either via (piped) standard input or the TTY::

    class DbSync(Command):
        """sync databases"""

        def __init__(self, parser):
            parser.add_argument(
                '-p', '--password',
                action='store_true',
                dest='stdin_password',
                default=False,
                help="read database password from standard input",
            )

        def __call__(self, args):
            engine = self.dbengine(args)
            ...

        def dbcreds(self, args):
            dbcreds = {
                'username': os.getenv('PGUSER'),
                'host': os.getenv('PGHOST'),
                'port': os.getenv('PGPORT'),
                'database': os.getenv('PGDATABASE'),
            }

            missing = [key for (key, value) in dbcreds.items() if not value]
            if missing:
                raise RuntimeError(
                    "database connection information missing from "
                    "environmental configuration: " + ', '.join(missing)
                )

            if args.stdin_password:
                dbcreds['password'] = sys.stdin.read().rstrip('\n\r')

                # we're done with the (pipe) stdin, so force it back to TTY for
                # any subsequent input()
                sys.stdin = open('/dev/tty')
            else:
                dbcreds['password'] = os.getenv('PGPASSWORD')
                if not dbcreds['password']:
                    dbcreds['password'] = getpass.getpass(
                        'enter password for '
                        + ('{username}@{host}:{port}'.format_map(dbcreds) | colors.bold)
                        + ': '
                        | colors.yellow
                    )

            return dbcreds

        def dburi(self, args):
            return sqlalchemy.engine.url.URL('postgres', **self.dbcreds(args))

        def dbengine(self, args):
            return sqlalchemy.create_engine(self.dburi(args))

Not only were we forced to verbosely daisy-chain the arguments namespace, ``args``, from method to method; moreover, we were prevented from (trivially) caching the result of ``dbcreds``, to ensure that the password isn't ever requested more than once.

Now, let's reimplement the above, making use of the property ``args``::

    class DbSync(Command):
        """sync databases"""

        def __init__(self, parser):
            parser.add_argument(
                '-p', '--password',
                action='store_true',
                dest='stdin_password',
                default=False,
                help="read database password from standard input",
            )

        def __call__(self):
            engine = self.dbengine
            ...

        @cachedproperty
        def dbcreds(self):
            dbcreds = {
                'username': os.getenv('PGUSER'),
                'host': os.getenv('PGHOST'),
                'port': os.getenv('PGPORT'),
                'database': os.getenv('PGDATABASE'),
            }

            missing = [key for (key, value) in dbcreds.items() if not value]
            if missing:
                raise RuntimeError(
                    "database connection information missing from "
                    "environmental configuration: " + ', '.join(missing)
                )

            if self.args.stdin_password:
                dbcreds['password'] = sys.stdin.read().rstrip('\n\r')

                # we're done with the (pipe) stdin, so force it back to TTY for
                # any subsequent input()
                sys.stdin = open('/dev/tty')
            else:
                dbcreds['password'] = os.getenv('PGPASSWORD')
                if not dbcreds['password']:
                    dbcreds['password'] = getpass.getpass(
                        'enter password for '
                        + ('{username}@{host}:{port}'.format_map(dbcreds) | colors.bold)
                        + ': '
                        | colors.yellow
                    )

            return dbcreds

        @property
        def dburi(self):
            return sqlalchemy.engine.url.URL('postgres', **self.dbcreds)

        @property
        def dbengine(self):
            return sqlalchemy.create_engine(self.dburi)

In this form, ``args`` needn't be passed from method to method; in fact, methods of the ``DbSync`` command needn't worry about arguments which don't directly interest them at all. And, using ``cachedproperty`` from Dickens_, the database credentials are trivially cached, ensuring they aren't needlessly re-requested.

Note that attempting to access the ``args`` property before invocation arguments have been parsed – *e.g.* within ``__init__`` – is not allowed, and will raise ``RuntimeError``.

Command hierarchy
-----------------

Our tools should be modular and composable, favoring atomicity over monolithism. Nevertheless, well-designed, -structured and -annotated code and application interfaces pay their users and developers tremendous dividends over time – no less in the case of more extensive interfaces, and particularly so for project management libraries (consider the ``Makefile``).

``argcmdr`` intends to facilitate the definition of ``argparse``-based interfaces regardless of their structure. But it's in multi-level, or hierarchical, command argumentation that ``argcmdr`` shines.

Nested commands
~~~~~~~~~~~~~~~

Rather than procedurally defining subparsers, ``Command`` class declarations may simply be nested.

Let's define an executable file ``manage`` for managing a codebase::

    #!/usr/bin/env python

    import os

    from argcmdr import Local, main

    class Management(Local):
        """manage deployment"""

        def __init__(self, parser):
            parser.add_argument(
                '-e', '--env',
                choices=('development', 'production'),
                default='development',
                help="target environment",
            )

        class Build(Local):
            """build app"""

            def prepare(self, args):
                req_path = os.path.join('requirements', f'{args.env}.txt')
                yield self.local['pip']['-r', req_path]

        class Deploy(Local):
            """deploy app"""

            def prepare(self, args):
                yield self.local['eb']['deploy', args.env]

    if __name__ == '__main__':
        main(Management)

``Local`` command ``Management``, above, defines no functionality of its own. As such, executing ``manage`` without arguments prints its autogenerated usage::

    usage: manage [-h] [--tb] [-q] [-d] [-s] [--no-show]
                  [-e {development,production}]
                  {build,deploy} ...

Because ``Management`` extends ``Local``, it inherits argumentation controlling whether standard output is printed and offering to run commands in "dry" mode. (Note, however, that it could have omitted these options by extending ``Command``. Moreover, it may override class method ``base_parser()``.)

``Management`` adds to the basic interface the optional argument ``--env``. Most important, however, are the related, nested commands ``Build`` and ``Deploy``, which define functionality via ``prepare``. Neither nested command extends its subparser – though they could; but rather, they depend upon the common argumentation defined by ``Management``.

Exploring the interface via ``--help`` tells us a great deal, for example ``manage -h``::

    usage: manage [-h] [--tb] [-q] [-d] [-s] [--no-show]
                  [-e {development,production}]
                  {build,deploy} ...

    manage deployment

    optional arguments:
      -h, --help            show this help message and exit
      --tb, --traceback     print error tracebacks
      -q, --quiet           do not print command output
      -d, --dry-run         do not execute commands, but print what they are
                            (unless --no-show is provided)
      -s, --show            print command expressions (by default not printed
                            unless dry-run)
      --no-show             do not print command expressions (by default not
                            printed unless dry-run)
      -e {development,production}, --env {development,production}
                            target environment

    management commands:
      {build,deploy}        available commands
        build               build app
        deploy              deploy app

And ``manage deploy -h``::

    usage: manage deploy [-h]

    deploy app

    optional arguments:
      -h, --help  show this help message and exit

As such, a "dry run"::

    manage -de production deploy

prints the following::

    > /home/user/.local/bin/eb deploy production

and without the dry-run flag the above operating system command is executed.

Decorated commands
~~~~~~~~~~~~~~~~~~

There is no artificial limit to the number of levels you may add to your command hierarchy. However, application interfaces are commonly "wider" than they are "deep". For these reasons, as an alternative to class-nesting, the hierarchical relationship may be defined by class decorator.

Let's define the executable file ``git`` with no particular purpose whatsoever::

    #!/usr/bin/env python

    from argcmdr import Command, RootCommand, main

    class Git(RootCommand):
        """another stupid content tracker"""

        def __init__(self, parser):
            parser.add_argument(
                '-C',
                default='.',
                dest='path',
                help="run as if git was started in <path> instead of the current "
                     "working directory.",
            )

    @Git.register
    class Stash(Command):
        """stash the changes in a dirty working directory away"""

        def __call__(self, args):
            self['save'](args)

        class Save(Command):
            """save your local modifications to a new stash"""

            def __init__(self, parser):
                parser.add_argument(
                    '-p', '--patch',
                    dest='interactive',
                    action='store_true',
                    default=False,
                    help="interactively select hunks from the diff between HEAD "
                         "and the working tree to be stashed",
                )

            def __call__(self, args):
                interactive = getattr(args, 'interactive', False)
                print("stash save", f"(interactive: {interactive})")

        class List(Command):
            """list the stashes that you currently have"""

            def __call__(self):
                print("stash list")

    if __name__ == '__main__':
        main(Git)

We anticipate adding many subcommands to ``git`` beyond ``stash``; and so, rather than nest all of these command classes under ``Git``:

* we've defined ``Git`` as a ``RootCommand``
* we've defined ``Stash`` at the module root
* we've decorated ``Stash`` with ``Git.register``

The ``RootCommand`` functions identically to the ``Command``; it only adds this ability to extend the listing of its subcommands by those registered via its decorator. (Notably, ``LocalRoot`` composes the functionaliy of ``Local`` and ``RootCommand`` via multiple inheritance.)

The ``stash`` command, on the other hand, contains the entirety of its hierarchical functionality, nesting its own subcommands ``list`` and ``save``.

Walking the hierachy
~~~~~~~~~~~~~~~~~~~~

Unlike the base command ``git`` in the example above, the command ``git stash`` – despite defining its own subcommands – also defines its own functionality, via ``__call__``. This functionality, however, is merely a shortcut to the ``stash`` command ``save``. Rather than repeat the definition of this functionality, ``Stash`` "walks" its hierarchy to access the instantiation of ``Save``, and invokes this command by reference.

Much of ``argcmdr`` is defined at the class level, and as such many ``Command`` methods are ``classmethod``. In the static or class context, we might walk the command hierarchy by reference, *e.g.* to ``Stash.Save``; or, from a class method of ``Stash``, as ``cls.Save``. Moreover, ``Command`` defines the class-level "property" ``subcommands``, which returns a list of ``Command`` classes immediately "under" it in the hierarchy.

The hierarchy of executable command objects, however, is instantiated at runtime and cached within the ``Command`` instance. To facilitate navigation of this hierarchy, the ``Command`` object is itself subscriptable. Look-up keys may be:

* strings – descend the hierarchy to the named command
* negative integers – ascend the hierarchy this many levels
* a sequence combining the above – to combine "steps" into a single action

In the above example, ``Stash`` may have (redundantly) accessed ``Save`` with the look-up key::

    (-1, 'stash', 'save')

that is with the full expression::

    self[-1, 'stash', 'save']

(The single key ``'save'``, however, was far more to the point.)

Because command look-ups are relative to the current command, ``Command`` also offers the ``property`` ``root``, which returns the base command. As such, our redundant expression could be rewritten::

    self.root['stash', 'save']

The management file
-------------------

In addition to the interface of custom executables, ``argcmdr`` endeavors to improve the generation and maintainability of non-executable but standardized files, intended for management of code development projects and operations.

Similar to a project's ``Makefile``, we might define our previous codebase-management file as the following Python module, ``manage.py``::

    import os

    from argcmdr import Local, main

    class Management(Local):
        """manage deployment"""

        def __init__(self, parser):
            parser.add_argument(
                '-e', '--env',
                choices=('development', 'production'),
                default='development',
                help="target environment",
            )

        class Build(Local):
            """build app"""

            def prepare(self, args):
                req_path = os.path.join('requirements', f'{args.env}.txt')
                yield self.local['pip']['-r', req_path]

        class Deploy(Local):
            """deploy app"""

            def prepare(self, args):
                yield self.local['eb']['deploy', args.env]

Unlike our original script, ``manage``, ``manage.py`` is not executable, and need define neither an initial shebang line nor a final ``__name__ == '__main__'`` block.

Rather, ``argcmdr`` supplies its own, general-purpose ``manage`` executable command, which loads Commands from any ``manage.py`` in the current directory, or as specified by option ``--manage-file PATH``. As such, the usage and functionality of our ``manage.py``, as invoked via argcmdr's installed ``manage`` command, is identical to our original ``manage``. We need only ensure that ``argcmdr`` is installed, in order to make use of it to manage any or all project tasks, in a standard way, with even less boilerplate.

Bootstrapping
~~~~~~~~~~~~~

To ensure that such a friendly – and *relatively* high-level – project requirement as ``argcmdr`` is satisfied, consider the expressly low-level utility install-cli_, with which to guide contributors through the process of provisioning your project's most basic requirements.

.. _argparse: https://docs.python.org/3/library/argparse.html
.. _python.org: https://www.python.org/downloads/
.. _Homebrew: https://brew.sh/
.. _pyenv: https://github.com/pyenv/pyenv
.. _pyenv installer: https://github.com/pyenv/pyenv-installer#installation--update--uninstallation
.. _plumbum: https://plumbum.readthedocs.io/en/latest/local_commands.html#exit-codes
.. _Dickens: https://github.com/dssg/dickens
.. _install-cli: https://github.com/dssg/install-cli


