..
  nd_node:
    nde_name: Documentation
  parent: Home

-----------------------------------------------------------------------
Using Lokai
-----------------------------------------------------------------------

Out of the box, Lokai comes with enough abilities to get started on
business processes that rely on activities. These abilities can be
extended by your own or other people's extension packages but, by
their nature, such packages cannot be covered here. This documentation
covers the basic built-in abilities of Lokai.

Building something useful in Lokai consists in building a system by
creating nodes in a hierarchy, and interlinking hierarchies to provide
alternative systems relating to the same data.

The simplest such system might be a to-do list. More complex systems
might include customer support ticketing, client management for a
sales team, agile project management, or hierarchical project
planning. Any of these systems can have associated help text or
documentation.

The basic elements for this are three node types.

  :Generic: A non-specific node that acts as a placeholder.

  :Text: Similar to a generic node except that its formatted display
      is limited to text only, without any of the management
      detail. This is the basis for documentation.

  :Activity: A node with additional details that allow an activity to
      be managed.

There is no constraint on how these nodes can be placed in a
hierarchy. A node of any type can form the head of a sub-tree, and
siblings of a node of some type can be of the same or any other
type. In this way you can mix activities and documentation, and you
can build hierarchies of activities.

Any node can also have more than one parent. This allows you to build
cross-linked hierarchies so that sub-systems can be shared. You can
use this to provide different ways of searching for or referencing
information, or you can use it to represent something in a business
process. For example, customer tickets may be linked to the support
desk tree, the customer who raised the ticket, and the corrective
action ticket that is raised to fix the problem.

Lokai is intended to support many such systems and sub-systems in the
same database. As a result, access control is important. The basic
principle of access control comes from the ability to allocate a user
as a resource to a node. Once a user is allocated to a node then the
privileges that the user has as a result of their role on that node
are automatically available on all sub-trees of that node. Thus,
managers may be allocated to nodes that cover their management remit,
while workers would be allocated to nodes that they need to see for
their appointed tasks.

The remaining sections give more detail on the key elements of the Lokai
package.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: User Interface
  parent: Documentation

User Interface
-----------------------------------------------------------------------


.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Page Layout
  parent: User Interface

Page Layout
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

The page on the screen is made up of four key elements. The exact
layout will depend on the templates and CSS that you might choose to
use, but these essential elements should appear somewhere.

:Main Menu: Normally on the left of the screen. This displays the top
    two or three layers of the sub-trees that you are allowed to
    see. Before you log in you will see only items that the anonymous
    user can see. After you log in you will also see the sub-trees
    that you have been allocated to.

:Application view: The main pane in the centre of the screen. This
    contains the data for the current node. In the basic package there
    are four possible views here.

    :display view: Formatted, read-only view suitable for reading
        documentation or reviewing content.

    :detail view: Data entry form that allows modification of the
        content.

    :list: List of child nodes. This is presented as a paged listing
        of the immediate children of the current node, with options
        for ordering the entries.

    :search: A search page that supports searching the sub-tree below
        the current node.

:Navigation links: A set of links that can take you up, left or right
    in the hierarchy. Only the relevant links are shown. If you do not
    have permission to see a node, then it does not appear.

:Function tabs: A set of links that allow you to choose the view in
    the application pane. Access to these is controlled by your role,
    and if you do not have permission for a particular view then the
    link does not appear.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Searching
  parent: User Interface

Searching
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

The search function brings up a page that allows for search using a
combination of fields. You may leave a field empty, in which case it
takes no part in the filtering selection. A non-empty field is used as
a filter on the associated node data. Multiple filters, if set, must
all be satisfied for a node to pass the filters.

The basic search fields are:

    :Search from here downwards: Gives the ID of the node at the top
        of the sub-tree to be searched. This is pre-set to the ID of
        the current node and cannot be changed.

    :External Reference: A unique external reference value. By itself,
        this will return only the single node that has the given
        external reference, if it exists. Other filters may be set and
        may fail, resulting in an empty result.

    :Node Name: Either a single partial name or a partial path to a
        node, using node names. The partial name and the elements of
        the path may contain wild cards.

        Since node names are not unique this may result in more than
        one match, even if wild cards are not used.

        A node name is recognised as any text not starting with
        '='. The text is wrapped in wild cards so that the text will
        match any name that *contains* the text.

        A path specification begins with '='.

        The path specification is a pseudo path made up of one or more
        of three element types.
    
        A node is returned if the rightmost element matches the node
        name and the route to the node matches the preceding left part
        of the specification.
    
        Path elements are:
    
            :'*': matches all the immediate children of each current candidate
                node.
    
            :'**': matches zero or more levels of descendants. If this
                 appears at the end of the path it matches the
                 immediate children only.
    
            :{name}: matches the name of a node.

                A name may contain '%' as a wild card.

        If the path begins with '/' the match starts with the given set of
        candidates.

        If the path does not begin with '/' the search is done as though
        the path starts with /`**`/

        A trailing '/' is ignored.
    
    :Tags: One or more tag values, separated by spaces. A node is
        returned if it has **any one** of the tags.

If you have full search permission you will see these additional
fields, largely based on activities:

    :Select assigned user: A drop down list of all users who have been
        allocated to this node or its parents. Your user name is set
        as default. Select a single user to filter on nodes that have
        that user *assigned* to the node.

    :Status range: A drop down list of: All, Open, Open&Active,
        Active, Closed. Your selection is used to filter on the node
        status. The status values have been chosen so that each
        selection corresponds to a range of possible status
        values. Thus, Closed will select status values of 'closed,
        'resolved - ok' and 'resolved - nok'. The default is
        Open&Active.

    :Bring Forward Date (From - To): A pair of dates that filter on
        the Reminder Date. Either date can be empty, in which case is
        ignored. Entering both dates allows for a fine grained
        selection.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Data Entry
  parent: User Interface

Data Entry
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Select the 'Detail' tab. Some of the fields will be hidden behind
closed field sets. These will be marked by '+' at the left of the
heading. Click to open.

You can generally do anything you like here, except that some fields
may be set read-only because you only have read permission.

If you change the type of the node you should update the screen to see
any new fields that might appear.

Adding attachments and resources has to be done one at a time. You
will need to update the screen after each one.

Text in the node description and the activity message is formatted
using `reStructuredText
<http://docutils.sourceforge.net/rst.html>`_. This is covered below.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Text Formatting
  parent: Data Entry

Text Formatting
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

All text is formatted using `reStructuredText
<http://docutils.sourceforge.net/rst.html>`_. This is an easy-to-read,
what-you-see-is-what-you-get plaintext markup syntax and parser
system. In other words, the plain text layout is both readable and
reflects the final document.

Lokai provides some extended reST features:

    :Link to another page: Support for a directive that inserts a link
        into a new paragraph on the page.

        Usage::

            .. page:: target [text]

        where

        :target: is either a node identifier by itself, or a node
            identifier with a file name appended. (MyPage or
            MyPage/file/attachment_file).

            The special target '#' refers to the current page.

        :text: is an optional piece of text that will appear in the
            output document as the link.

        The link is made to the relevant page in the lk_worker
        hierarchy.

    :In-line link to another page: Define a role for a page link. A
        link is inserted in line in the output text.

        This looks like::

            :page:`target`

        where target is as for the directive, above.

    :image: Support a ``.. page_img::`` directive to make an image from a
        reference to an attachment.

        The allowed URIs are the same as for the ``.. page::``
        directive.

    :figure: Support a ``.. page_fig::`` directive to make a figure from a
        reference to an attachment.

        The allowed URIs are the same as for the ``.. page::``
        directive.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Add New Node
  parent: User Interface

Adding a new Node
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Select the 'List' tab. There is a link, 'Add new node', at the bottom
of the page. You will be presented with a data entry screen based on
the 'generic' node.

The new node will be added at the same level as other nodes in the
list, as a child of the node you were visiting when you clicked the
link.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Node Types
  parent: User Interface

Node Types
-----------------------------------------------------------------------

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Generic Node
  parent: Node Types

Generic
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

The generic node holds the most basic information of all.

    :Node ID: An automatically allocated ID number for the node. This
        is guaranteed to be unique in the database and is generally
        used internally to build URL links to nodes (and to cross link
        data tables in the database).

    :Node Name: A free text item that appears as the title of the node
        in most displays, and can be used in a URL to fetch the
        node. The node name is not necessarily unique.

    :Node Description: A free text item that contains optional
        descriptive text.

    :Node Type: One of: Generic, Text Only or Activity. Or, indeed, any
        other type that has been defined by a extension package. The
        type defines primarily what data is displayed and how it is
        displayed.

    :External Reference: An optional unique reference text that can be
        used as an alternative unique ID. This is primarily aimed at
        client software applications that may wish to generate their
        own ID for backward reference.

    :Date Created: The date the node was created.

    :Date Modified: The date the node was last modified.

    :Attachments: Zero, one or more files attached to this node.

By default a generic node presents its list view when displayed.

A generic node also provides basic node manipulation data.

    :Resources: 

        :Allocated Users: Zero, one or more users allocated to this
            node, with a specific role for each user. Allocating a
            user here gives that user access to this node and the
            complete sub-tree, with whatever read/write privileges
            are available to the role.

        :Assigned User: A single user who is assumed to be the person
            currently working on or responsible for the node. It is
            only possible to assign users who are allocated to this
            node or a parent node.

    :Parents: Zero, one or more parent nodes. Parents may be added or
        removed. There is a check to avoid creating circular linkages.

    :Display Sequence: A number that defines the order in which the
        node will appear in the left/right sequence of its
        siblings. The sequence is relevant to the current parent
        node. The ordering may be different if the node has been
        reached through a different node. Normally the sequence
        represents the order in which the nodes are added to the
        parent but you are free to change it if you need to.

    :Subscribers: An optional comma separated list of email
        addresses. If changes are made to any node in the sub-tree
        below this node then an email notification will be sent to
        these email addresses. You can enter usernames if you
        like. These will be converted to email addresses if possible.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Text Only Node
  parent: Node Types

Text Only
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

This is an alternative view for a generic node. No new data is
involved.

The display view shows the formatted node description text, together
with a simple list of child nodes. No other data is displayed. If you
want to display attachments you must put a reference into the
description text. See Text Formatting.

By default a text node presents its display view.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Activity Node
  parent: Node Types

Activity
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

An activity builds on a generic node by adding:

    :Start Date: Optional date when the activity might start.

    :End Date: Optional date when the activity might (or actually did)
        finish.

    :Reminder Date: Optional date when next something needs to be done
        in relation to this activity.

    :Status: One of: open, active, closed, resolved-ok, resolved-nok.

    :Priority: One of: immediate, high, medium, low, parked.

    :Message: A one-off message that can be entered whenever the node
        is modified. Messages are accumulated in the activity history.

        Changes to the data are also recorded in the activity history.

The choices of dates, status codes and priority codes are not meant to
imply a specific work practice. They can be used, or not, as required.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Job Manager Logger Node
  parent: Node Types

Job Manager Logger
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

If you are using the job manager then you might also see nodes of this
type. These are, in effect, activity nodes with the addition of:

    :Severity: One of: critical, error, warning, notification, debug.

Nodes of this type are generated as part of the logging process for
background batch jobs. Users can use these entries to manage whatever
issues have arisen.


.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Resource Group Node
  parent: Node Types

Resource Group
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

A Resource Group is a way of allocating users to groups so that
members of the group can have access to sub-trees that belong to that
group. Allocating a resource to the group automatically confers access
to the connected sub-groups.

In this case the group is modelled as a node and users are allocated
to the node. Sub-trees that are accessible by this group are linked as
children to this node.

A user must be allocated a role on the group, and this role is then
applied to the connected sub-trees.

This mechanism models, and replaces, the equivalent mechanism where a
group is modelled as a named resource that is allocated to each
sub-tree. Users are aliased to that resource by being members of that
group. 

There are two advantages to using a node to represent groups rather
than mapping to a psuedo-resource. It allows users to be given
different roles in the group, and it allows groups to be defined in
different parts of the hierarchy, thus allowing the same group name to
be used in different scopes.

There is, in principle, no reason why any node could not be used as a
group. However, there is a potential security issue in that any user
allocated to the group node also has access to the details of that
group node. Creating a specific resource_group type allows access to
the details to be controlled.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Administration
  parent: Documentation

Administration
-----------------------------------------------------------------------

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Main Menu Build
  parent: Administration

Building a Main Menu
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

The main menu displays only those nodes that you have been allocated
to. The default order of display is arbitrary, so Lokai provides a
mechanism for building a menu using a node system.

    Create a node under the root node with a node name of 'Menu
    Groups'.

    Allocate the system administrator to this node, and remove the
    parent. The Menu Groups node is then floating without a parent,
    but the system administrator has access to it.

    Create one or more menu groups under this node by inserting new
    generic nodes where:

    :Node Name: is a number. This is defines a menu group and all
        allowed nodes under this node will belong to this menu
        group. Menu groups are displayed in group order.

    :Node Description: is a short text. This is the text that is
        displayed on the menu.

    Link relevant sub-trees to the appropriate menu group. These
    sub-trees can be quite high level. The algorithm searches down
    through each sub-tree to find the first relevant node for the
    user. If there is no relevant node then the group remains empty
    and is not displayed.

Any nodes that are not captured by this mechanism are displayed in the
default order.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: User Permissions
  parent: Administration

User Permissions
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

User permissions are role-based. That is, a user may have one or more
roles and each role brings with it one or more functions. A function
may have one or more of read/write/modify/delete permissions.

Because the permission structure is node based a user may have
different permissions on different nodes. A user gains permissions in
four ways:

1. From the 'anonymous' user. Any role given to the anonymous user
   applies to any access to the system, whether a user is logged in or
   not.

2. By being given a base role as a user. The user also inherits the
   role or roles associated with the anonymous user. This base role,
   being part of the user, is independent of node. A sufficiently
   powerful role can bypass all node-based permission checking.

3. By being given an explicit role on a node.

4. By inheriting one or more roles from parent nodes. Inheritance is
   cumulative. A node will generally have more than one node above it
   in a particular sub-tree, and a node may be linked to more than one
   parent. All roles found in these upper levels are merged.

   Using :pages:`Resource Group nodes <Resource Group Node> can be a
   convenient way of allocating resources.


Lokai comes with three roles:

    :lkw_anonymous: This is the default role. Anyone not logged in has
        this role. It is used to make parts of the site public.

    :lkw_manager: A management role that gives full access to a
        node. This includes the ``nde_resource`` function.

        Lokai comes with a lokai.administrator user who has the
        ``lkw_manager`` role as a base role. This user has full access
        to all nodes.

    :lkw_task: A task user role that gives full access to a node but
        without the ``nde_resource`` function.

A user needs some of these functions to make use of Lokai:
 
    :nde_tasks: Access to lk_worker nodes

      This is the main function. A user must have at least 'read'
      access on this function to be able to see anything at all.

    :nde_resource: Managing resources attached to lk_worker nodes

        Provides access to resource lists on nodes. This can be used
        to prevent other users seeing who else has a role on a node
        and to prevent them changing roles.

    :nde_search_full: Search all searchable aspects of lk_worker nodes

    :nde_search_text: Limited search facility aimed at text nodes

        These give access to the search screen. The results of the
        search are limited to nodes where the user role includes
        ``nde_tasks``.

	``nde_search_full`` includes all fields on the search
	form. ``nde_search_text`` is limited.


.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Page Layout Control
  parent: Administration

Page Layout
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

TO DO


.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Useful Scripts
  parent: Administration

Useful Scripts
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

There are some useful scripts that can be run from the command line on
the server. All scripts must be run in the working directory for the
implementation. All scripts proved a ``--help`` option for further
detail.

Start a server:

:lk_fcgi_server.py:

    Run a fast cgi server.  See the configuration file for port settings.

:lk_fcgi_server_down.py:

    Kill the fast cgi server.

Import and export (including initial data):

    :lk_export_node.py:

        Output the content of a node in yaml format. Output does not
        contain all the data and this script needs further work.

    :lk_import_document.py:

        Reads a reStructuredText file into one or more nodes. The
        input file is an rst file that can be edited and run through
        rst formatters, such as rst2pdf, in the normal way. This
        allows for off-line editing and validation of texts that are
        intended for storage in text nodes in the lk_worker database.

        The input file contains control information, a control block,
        that identifies the node to be updated. This control
        information is hidden in a specifically structured comment
        block, so that it is invisible to the rst processing.

        A single file may contain text for more than one node.

        Each node's worth of text consists of a control block,
        followed by the text itself. The text is terminated by the end
        of the file or by the next control block.

        See the source code for a full description of the format.

    :lk_import_node.py:

        Import a node using the data output by ``lk_export_node.py``.

    :lk_initial_data.py:

        Set up node and login data. The information handled is minimal
        and is suitable for creating a basic position that can be
        worked up to provide a useful system. Some initial data is
        required before anything can be done through the user
        interface and this tool is intended to create this data from
        an input file. See the files in the ``config`` directory of
        the Lokai package.

    :tb_migrate.py:

        Database creation and migration tool. This tool must be run at
        least once, to set up the database, before anything else can
        be done.

Node management:

    :lk_find_node.py:

        Find one or more nodes, identified by a node ID or a node name
        partial path. See the search description for how to specify a
        partial path. The output can be piped into the other tools in
        this set.
        
    :lk_link_node.py:

        Link one or more nodes, identified by a node ID or a node name
        partial path, to a parent node, identified by a unique ID or
        node name. See the search description for how to specify a
        partial path.

    :lk_make_node.py:

        Create a generic node given a node name. The new node is not
        linked to any parent.

    :lk_remove_node.py:

        Delete one or more nodes, identified by a node ID or a node
        name partial path. See the search description for how to
        specify a partial path. The deletion removes the full sub-tree
        for the node.

Job management:

    :tb_print_job_tree.py:

        Print the tree of interlinked jobs inferred from the input and
        output directories of all the jobs listed in the job parameter
        directory.

.. raw:: pdf

   PageBreak

..
  nd_node:
    nde_name: Installation
  parent: Documentation

Installation
-----------------------------------------------------------------------

Download the package tarball and unpack it. Go into the distribution
directory and run::

  python setup.py install

or, more simply, don't bother with the download and just type::

  pip install lokai

To build a workable instance:

  Create a working directory for your new instance. And change into it.

  Copy ``lk_config.ini`` from the ``config`` folder of
  the installation.

  Review the content of the ``.ini`` file. It should work out of the
  box. However, you might want to change the default database name to
  something other than ``lokai_soft``.

  Create the database::

    tb_migrate.py login.login_db=max

    tb_migrate.py lk_worker.nodes_db=max

  Copy ``initial_data.yml`` from the ``doc/initial_settings`` folder
  of the installation. This contains a basic administrator login and a
  starting node for the database.

  Use the initial data::

    lk_initial_data -f initial_data.yml

  For a quick hit, start the publisher stand alone::
  
    python -mlokai.lk_ui.publisher

  Now browse to localhost:8080.

Fast CGI installation
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

The script ``lk_fcgi_server.py`` provides a pre-forked fcgi server
that can be used as a back end for any httpd server.

If you are running with whatever the distribution gives you then this
script can be run automatically by the httpd server using whatever
configuration that server allows.

Conversely, if you are running with virtualenv then using the global
python executable will not work (it will see the wrong environment),
so you need to run with the python from whichever virtualenv you want
to pick up.

Generally, it is probably best to run this fgci_server as a daemon in
your working environment. One way to achieve this in a production
environment is to run the server automatically at boot using the
crontab 'reboot' time setting.

e.g.::
    
    @reboot /home/site_manager/site_wrapper path/to/server.py

Where ``site_wrapper`` sets up the virtualenv and does other essentials
such as setting $PYTHONPATH and entering the working directory before
running python.

An example site_wrapper for bash might look like::

    cd `dirname $0`
    export PYTHONPATH=`pwd`/application
    prog=$1
    shift
    python $prog "$@"

When running fcgi_server as a daemon you need to tell it what to
listen for. This can either be a unix socket file, or an address/port
pair. Whatever you do must be matched with the settings for the httpd
server.

Listen settings are defined in the configuration file, in group
``server``. You provide either::

    socket = {{some file name}}

or::
    listen = {{host address}}
    port = {{port number}}

If you give both ``socket`` and ``listen`` then ``socket`` is ignored.

To run this with lighttpd you need something like the following in
your server configuration (where port 4001 is actually whatever you
set up in the lokai config file)::

  fastcgi.server = (
                   "/" =>
                     ("localhost" =>
                       (
                         "host" => "127.0.0.1",
                         "port" => 4001,
                         "check-local" => "disable",
                         "fix-root-scriptname" => "enable"
                       )
                     )
                   )


