Metadata-Version: 2.0
Name: arachnado
Version: 0.2
Summary: Scrapy-based Web Crawler with an UI
Home-page: https://github.com/TeamHG-Memex/arachnado
Author: Mikhail Korobov
Author-email: kmike84@gmail.com
License: MIT
Platform: UNKNOWN
Classifier: Framework :: Scrapy
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: scrapy (>=1.0.0)
Requires-Dist: Twisted (>=12)
Requires-Dist: psutil (>=2.2)
Requires-Dist: tornado (>=4.2)
Requires-Dist: docopt (>=0.6)
Requires-Dist: service-identity

Arachnado
=========

Arachnado is a tool to crawl a specific website.
It provides a Tornado_-based HTTP API and a web UI for a Scrapy_-based
crawler.

License is MIT.

.. _Tornado: http://www.tornadoweb.org
.. _Scrapy: http://scrapy.org/

Install
-------

Arachnado requires Python 2.7.
To install Arachnado use pip::

    pip install arachnado

To install Arachnado with MongoDB support use this command::

    pip install arachnado[mongo]

Run
---

To start Arachnado execute ``arachnado`` command::

    arachnado

and then visit http://0.0.0.0:8888 (or whatever URL is configured).

To see available command-line options use

    arachnado --help

Arachnado can be configured using a config file. Put it to one of the common
locations ('/etc/arachnado.conf', '~/.config/arachnado.conf'
or '~/.arachnado.conf') or pass the file name as an argument when starting
the server::

    arachnado --config ./my-config.conf

For available options check
https://github.com/TeamHG-Memex/arachnado/blob/master/arachnado/settings/defaults.conf.

Development
-----------

* Source code: https://github.com/TeamHG-Memex/arachnado
* Issue tracker: https://github.com/TeamHG-Memex/arachnado/issues

To build Arachnado static assets node.js + npm are required.
Install all JavaScript requirements using npm - run the following command
from the repo root::

    npm install

then rebuild static files (we use Webpack_)::

    npm run build

or auto-build static files on each change during development::

    npm run watch

.. _Webpack: https://github.com/webpack/webpack


Changes
=======

0.2 (2015-08-07)
----------------

Initial release.


