文件名称:larbin-2.6.3
介绍说明--下载内容均来自于网络,请自行研究使用
larbin是一种开源的网络爬虫/网络蜘蛛,由法国的年轻人Sébastien Ailleret独立开发。larbin目的是能够跟踪页面的url进行扩展的抓取,最后为搜索引擎提供广泛的数据来源。 Larbin只是一个爬虫,也就是说larbin只抓取网页,至于如何parse的事情则由用户自己完成。另外,如何存储到数据库以及建立索引的事情 larbin也不提供。
latbin最初的设计也是依据设计简单但是高度可配置性的原则,因此我们可以看到,一个简单的larbin的爬虫可以每天获取500万的网页,实在是非常高效。
利用larbin,我们可以轻易的获取/确定单个网站的所有联结,甚至可以镜像一个网站;也可以用它建立url 列表群,例如针对所有的网页进行 url retrive后,进行xml的联结的获取。或者是 mp3,或者定制larbin,可以作为搜索引擎的信息的来源。-no
latbin最初的设计也是依据设计简单但是高度可配置性的原则,因此我们可以看到,一个简单的larbin的爬虫可以每天获取500万的网页,实在是非常高效。
利用larbin,我们可以轻易的获取/确定单个网站的所有联结,甚至可以镜像一个网站;也可以用它建立url 列表群,例如针对所有的网页进行 url retrive后,进行xml的联结的获取。或者是 mp3,或者定制larbin,可以作为搜索引擎的信息的来源。-no
(系统自动生成,下载前可以参看下载内容)
下载文件列表
larbin-2.6.3\adns\adns.h
............\....\check.c
............\....\config.h.in
............\....\configure
............\....\dlist.h
............\....\event.c
............\....\general.c
............\....\install-sh
............\....\internal.h
............\....\Makefile
............\....\parse.c
............\....\poll.c
............\....\query.c
............\....\reply.c
............\....\setup.c
............\....\transmit.c
............\....\tvarith.h
............\....\types.c
............\configure
............\COPYING
............\CREDITS
............\doc\custom-eng.html
............\...\download.html
............\...\index-eng.html
............\...\index.html
............\...\l-en.jpg
............\...\l-fr.jpg
............\...\use-eng.html
............\larbin.conf
............\Makefile
............\options.h
............\README
............\src\fetch\checker.cc
............\...\.....\checker.h
............\...\.....\defaultspecbuf.cc
............\...\.....\dynamicspecbuf.cc
............\...\.....\dynamicspecbuf.h
............\...\.....\fetchOpen.cc
............\...\.....\fetchOpen.h
............\...\.....\fetchPipe.cc
............\...\.....\fetchPipe.h
............\...\.....\file.cc
............\...\.....\file.h
............\...\.....\hashTable.cc
............\...\.....\hashTable.h
............\...\.....\Makefile
............\...\.....\savespecbuf.cc
............\...\.....\savespecbuf.h
............\...\.....\sequencer.cc
............\...\.....\sequencer.h
............\...\.....\site.cc
............\...\.....\site.h
............\...\.....\specbuf.cc
............\...\.....\specbuf.h
............\...\global.cc
............\...\global.h
............\...\interf\defaultuseroutput.cc
............\...\......\input.cc
............\...\......\input.h
............\...\......\Makefile
............\...\......\mirrorsaveuseroutput.cc
............\...\......\output.cc
............\...\......\output.h
............\...\......\saveuseroutput.cc
............\...\......\statsuseroutput.cc
............\...\......\useroutput.cc
............\...\......\useroutput.h
............\...\larbin.make
............\...\main.cc
............\...\Makefile
............\...\types.h
............\...\utils\connexion.cc
............\...\.....\connexion.h
............\...\.....\ConstantSizedFifo.h
............\...\.....\debug.cc
............\...\.....\debug.h
............\...\.....\Fifo.h
............\...\.....\hashDup.cc
............\...\.....\hashDup.h
............\...\.....\histogram.cc
............\...\.....\histogram.h
............\...\.....\Makefile
............\...\.....\mypthread.cc
............\...\.....\mypthread.h
............\...\.....\PersistentFifo.cc
............\...\.....\PersistentFifo.h
............\...\.....\string.cc
............\...\.....\string.h
............\...\.....\SyncFifo.h
............\...\.....\text.cc
............\...\.....\text.h
............\...\.....\url.cc
............\...\.....\url.h
............\...\.....\Vector.h
............\...\.....\webserver.cc
............\...\.....\webserver.h
............\...\fetch
............\...\interf
............\...\utils
............\adns
............\....\check.c
............\....\config.h.in
............\....\configure
............\....\dlist.h
............\....\event.c
............\....\general.c
............\....\install-sh
............\....\internal.h
............\....\Makefile
............\....\parse.c
............\....\poll.c
............\....\query.c
............\....\reply.c
............\....\setup.c
............\....\transmit.c
............\....\tvarith.h
............\....\types.c
............\configure
............\COPYING
............\CREDITS
............\doc\custom-eng.html
............\...\download.html
............\...\index-eng.html
............\...\index.html
............\...\l-en.jpg
............\...\l-fr.jpg
............\...\use-eng.html
............\larbin.conf
............\Makefile
............\options.h
............\README
............\src\fetch\checker.cc
............\...\.....\checker.h
............\...\.....\defaultspecbuf.cc
............\...\.....\dynamicspecbuf.cc
............\...\.....\dynamicspecbuf.h
............\...\.....\fetchOpen.cc
............\...\.....\fetchOpen.h
............\...\.....\fetchPipe.cc
............\...\.....\fetchPipe.h
............\...\.....\file.cc
............\...\.....\file.h
............\...\.....\hashTable.cc
............\...\.....\hashTable.h
............\...\.....\Makefile
............\...\.....\savespecbuf.cc
............\...\.....\savespecbuf.h
............\...\.....\sequencer.cc
............\...\.....\sequencer.h
............\...\.....\site.cc
............\...\.....\site.h
............\...\.....\specbuf.cc
............\...\.....\specbuf.h
............\...\global.cc
............\...\global.h
............\...\interf\defaultuseroutput.cc
............\...\......\input.cc
............\...\......\input.h
............\...\......\Makefile
............\...\......\mirrorsaveuseroutput.cc
............\...\......\output.cc
............\...\......\output.h
............\...\......\saveuseroutput.cc
............\...\......\statsuseroutput.cc
............\...\......\useroutput.cc
............\...\......\useroutput.h
............\...\larbin.make
............\...\main.cc
............\...\Makefile
............\...\types.h
............\...\utils\connexion.cc
............\...\.....\connexion.h
............\...\.....\ConstantSizedFifo.h
............\...\.....\debug.cc
............\...\.....\debug.h
............\...\.....\Fifo.h
............\...\.....\hashDup.cc
............\...\.....\hashDup.h
............\...\.....\histogram.cc
............\...\.....\histogram.h
............\...\.....\Makefile
............\...\.....\mypthread.cc
............\...\.....\mypthread.h
............\...\.....\PersistentFifo.cc
............\...\.....\PersistentFifo.h
............\...\.....\string.cc
............\...\.....\string.h
............\...\.....\SyncFifo.h
............\...\.....\text.cc
............\...\.....\text.h
............\...\.....\url.cc
............\...\.....\url.h
............\...\.....\Vector.h
............\...\.....\webserver.cc
............\...\.....\webserver.h
............\...\fetch
............\...\interf
............\...\utils
............\adns