文件名称:colar
介绍说明--下载内容均来自于网络,请自行研究使用
一个分布式网路爬虫框架,用户只要编写爬虫,提交到框架上,即可多台电脑联合采集数据-A distributed network reptiles fr a mework, users can write reptiles, submitted to the fr a me, you can collect data combined with multiple computers
(系统自动生成,下载前可以参看下载内容)
下载文件列表
colar
.....\.gitignore
.....\AUTHORS
.....\LICENSE
.....\README.md
.....\bin
.....\...\coca.py
.....\...\start_master.py
.....\...\start_worker.py
.....\cola
.....\....\__init__.py
.....\....\conf
.....\....\....\main.yaml
.....\....\core
.....\....\....\__init__.py
.....\....\....\bloomfilter
.....\....\....\...........\__init__.py
.....\....\....\...........\hashtype.py
.....\....\....\config.py
.....\....\....\dedup.py
.....\....\....\errors.py
.....\....\....\extractor
.....\....\....\.........\__init__.py
.....\....\....\.........\preprocess.py
.....\....\....\.........\readability.py
.....\....\....\.........\utils.py
.....\....\....\logs.py
.....\....\....\mq
.....\....\....\..\SDFD.txt
.....\....\....\..\__init__.py
.....\....\....\..\client.py
.....\....\....\..\hash_ring.py
.....\....\....\..\node.py
.....\....\....\opener.py
.....\....\....\parsers.py
.....\....\....\rpc.py
.....\....\....\unit.py
.....\....\....\urls.py
.....\....\....\utils.py
.....\....\....\zip.py
.....\....\job
.....\....\...\__init__.py
.....\....\...\context.py
.....\....\...\loader.py
.....\....\master
.....\....\......\__init__.py
.....\....\......\loader.py
.....\....\......\watcher.py
.....\....\worker
.....\....\......\__init__.py
.....\....\......\loader.py
.....\....\......\recover.py
.....\....\......\watcher.py
.....\contrib
.....\.......\__init__.py
.....\.......\generic
.....\.......\.......\__init__.py
.....\.......\.......\generic.yaml
.....\.......\.......\stop.py
.....\.......\.....\__init__.py
.....\.......\.....\bundle.py
.....\.......\.....\conf.py
.....\.......\.....\login.py
.....\.......\.....\parsers.py
.....\.......\.....\stop.py
.....\.......\.....\storage.py
.....\.......\.....\utils.py
.....\.......\.....\weibo.yaml
.....\.......\weibosearch
.....\.......\...........\__init__.py
.....\.......\...........\bundle.py
.....\.......\...........\conf.py
.....\.......\...........\keywords.txt
.....\.......\...........\login.py
.....\.......\...........\parsers.py
.....\.......\...........\starts.py
.....\.......\...........\stop.py
.....\.......\...........\storage.py
.....\.......\...........\weibosearch.yaml
.....\.......\wiki
.....\.......\....\__init__.py
.....\.......\....\stop.py
.....\.......\....\wiki.yaml
.....\tests
.....\.....\__init__.py
.....\.....\contrib
.....\.....\.......\__init__.py
.....\.....\.......\test_weibo.py
.....\.....\.......\test_wiki.py
.....\.....\test_bloom_filter_mq.py
.....\.....\test_bloomfilter.py
.....\.....\test_config.py
.....\.....\test_context.py
.....\.....\test_extractor.py
.....\.....\test_loader_log.py
.....\.....\test_log.py
.....\.....\test_master_watcher.py
.....\.....\test_mq.py
.....\.....\test_mq_node.py