Metadata-Version: 2.4
Name: python-darc
Version: 1.0.5
Summary: Darkweb crawler & search engine.
Home-page: https://github.com/JarryShaw/darc
Download-URL: https://github.com/JarryShaw/darc/archive/v1.0.5.tar.gz
Author: Jarry Shaw
Author-email: jarryshaw@icloud.com
Maintainer: Jarry Shaw
Maintainer-email: jarryshaw@icloud.com
License: BSD 3-Clause License
Keywords: darkweb,crawler
Platform: any
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Software Development
Classifier: Topic :: Utilities
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/x-rst
License-File: LICENSE
Requires-Dist: beautifulsoup4[html5lib]
Requires-Dist: file-magic
Requires-Dist: peewee
Requires-Dist: pottery
Requires-Dist: psutil
Requires-Dist: python-datauri
Requires-Dist: redis[hiredis]
Requires-Dist: requests-futures
Requires-Dist: requests[socks]
Requires-Dist: selenium<4
Requires-Dist: stem
Requires-Dist: typing_extensions
Requires-Dist: dataclasses; python_version < "3.7"
Provides-Extra: sqlite
Requires-Dist: pysqlite3; extra == "sqlite"
Provides-Extra: mysql
Requires-Dist: PyMySQL[rsa]; extra == "mysql"
Provides-Extra: postgresql
Requires-Dist: psycopg2; extra == "postgresql"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: download-url
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: license-file
Dynamic: maintainer
Dynamic: maintainer-email
Dynamic: platform
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

``darc`` - Darkweb Crawler Project
========================================

``darc`` is designed as a swiss army knife for darkweb crawling.
It integrates ``requests`` to collect HTTP request and response
information, such as cookies, header fields, etc. It also bundles
``selenium`` to provide a fully rendered web page and screenshot
of such view.

The general process of ``darc`` can be described as following:

There are two types of *workers*:

* ``crawler`` -- runs the ``darc.crawl.crawler`` to provide a
  fresh view of a link and test its connectability

* ``loader`` -- run the ``darc.crawl.loader`` to provide an
  in-depth view of a link and provide more visual information

