Metadata-Version: 2.1
Name: AirSpider
Version: 2.0.1
Summary: A light and fast python Web crawler framework based on asyncio.
Home-page: https://pypi.org/project/airspider/
Author: Liu Xunzhuo
Author-email: mixdeers@gmail.com
License: MIT
Project-URL: Documentation, https://github.com/xunzhuo/AirSpider
Project-URL: Source, https://github.com/xunzhuo/AirSpider
Platform: UNKNOWN
Classifier: Framework :: AsyncIO
Classifier: Intended Audience :: Developers
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX :: BSD
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: aiohttp (>=3.5.4)
Requires-Dist: cssselect
Requires-Dist: lxml
Provides-Extra: uvloop
Requires-Dist: uvloop ; extra == 'uvloop'

<p align="center">
    <img src="https://picreso.oss-cn-beijing.aliyuncs.com/airs.png" width="150px">
    <h1 align="center"><img src="http://picreso.oss-cn-beijing.aliyuncs.com/font.png" /></h1>
    <p align="center" style="font-weight:bold">
        Distributed🌍  -  Asynchronous🏃  -  Light☁️  -  Fast⚡️ - Easy😄
      <br>
       <br>
     		AirSpider🕷️, a Light and Fast Python Web Crawler Framework Based on Redis🕷️
      <img src="http://picreso.oss-cn-beijing.aliyuncs.com/airdemo.png">
</p>    




---

## Overview👀

> + **AirSpider** is a high-performance asynchronous crawler framework for developers 🚀
> + Based on Redis: task distribution, task deduplication, and distributed ☁️
>

## Requirements☁️

- Python 3.6➕
- Works on Linux, Windows, macOS🍎

## Features🌲

+ **Quick to Start ☑️**
+ **Low Coupling ☑️**
+ **High Cohesion ☑️**
+ **Easy Expansion ☑️**
+ **Orderly Workflow ☑️**

## Installation🔨

``` shell
---------------------------

# For Linux && MacOS🔥
pip3 install airspider

---------------------------

# For Windows🔥
pip3 install airspider

---------------------------
```

+ ## Documents🔥

  + [Introduction](https://github.com/Xunzhuo/AirSpider/tree/master/Docs/cn/introduction.md)：介绍airspider
  + [Tutorials](https://github.com/Xunzhuo/AirSpider/tree/master/Docs/cn/tutorials.md)：使用airspider快速编写一个稳健的爬虫
  + [Plugins](https://github.com/Xunzhuo/AirSpider/tree/master/Docs/cn/plugins.md)：编写airspider扩展

  ## Topics

  - [Item](https://github.com/Xunzhuo/AirSpider/tree/master/Docs/cn/topics/item.md)：定义爬虫的目标字段
  - [Selector](https://github.com/Xunzhuo/AirSpider/tree/master/Docs/cn/topics/selector.md)：从HTML中提取出目标字段
  - [Request](https://github.com/Xunzhuo/AirSpider/tree/master/Docs/cn/topics/request.md)：请求并抓取目标网站资源
  - [Response](https://github.com/Xunzhuo/AirSpider/tree/master/Docs/cn/topics/response.md)：进一步封装响应内容
  - [Middleware](https://github.com/Xunzhuo/AirSpider/tree/master/Docs/cn/topics/middleware.md)：使爬虫支持第三方扩展
  - [Spider](https://github.com/Xunzhuo/AirSpider/tree/master/Docs/cn/topics/spider.md)：爬虫程序的入口

## TODO✈️

+ Complete Plugins of Redis🔥
+ Complete Distributed Architecture☁️

## Contributing👬

**AirSpider**🕷️ is still under **Developing**🔨

> Feel free to open issues💬 and pull requests💗

- Report or Fix bugs🌈
- Build Powerful plugins🔥
- Make documentation Better📖
- Add Examples of Crawling 🕷️

![](http://picreso.oss-cn-beijing.aliyuncs.com/monkey.jpg)


