文件名称:crawler
- 所属分类：
- JSP源码/Java
- 资源属性：
- [Java] [源码]
- 上传时间：
- 2012-11-26
- 文件大小：
- 21kb
- 下载次数：
- 0次
- 提 供 者：
- 王**
- 相关连接：
- 无
- 下载说明：
- 别用迅雷下载，失败请重下，重下不扣分！
介绍说明－－下载内容均来自于网络，请自行研究使用
Spider又叫WebCrawler或者Robot，是一个沿着链接漫游Web 文档集合的程序。它一般驻留在服务器上，通过给定的一些URL，利用HTTP等标准协议读取相应文档,然后以文档中包括的所有未访问过的URL作为新的起点，继续进行漫游，直到没有满足条件的新URL为止。WebCrawler的主要功能是自动从Internet上的各Web 站点抓取Web文档并从该Web文档中提取一些信息来描述该Web文档,为搜索引擎站点的数据库服务器追加和更新数据提供原始数据,这些数据包括标题、长度、文件建立时间、HTML文件中的各种链接数目等-Spider called WebCrawler or Robot, a collection of documents along the Web link roaming procedures. It generally resides on the server, by giving some of the URL, using HTTP and other standard protocols to read the documentation, then all included in the document URL is not visited as a new starting point, continue to roam until the conditions are not met until the new URL. WebCrawler' s main function is to automatically from the Web site on the Internet crawled Web documents and Web documents from the extraction of some information to describe the Web document, the site for the search engine' s database server and update the data provided additional raw data, including title, length, file creation time, HTML file, the number of various links, etc.
(系统自动生成,下载前可以参看下载内容)
下载文件列表
crawler\src\edu\uci\ics\crawler4j\crawler\Configurations.java
.......\...\...\...\...\.........\.......\CrawlController.java
.......\...\...\...\...\.........\.......\HTMLParser.java
.......\...\...\...\...\.........\.......\IdleConnectionMonitorThread.java
.......\...\...\...\...\.........\.......\LinkExtractor.java
.......\...\...\...\...\.........\.......\Page.java
.......\...\...\...\...\.........\.......\PageFetcher.java
.......\...\...\...\...\.........\.......\PageFetchStatus.java
.......\...\...\...\...\.........\.......\WebCrawler.java
.......\...\...\...\...\example\advanced\Controller.java
.......\...\...\...\...\.......\........\CrawlStat.java
.......\...\...\...\...\.......\........\Downloader.java
.......\...\...\...\...\.......\........\MyCrawler.java
.......\...\...\...\...\frontier\DocIDServer.java
.......\...\...\...\...\........\Frontier.java
.......\...\...\...\...\........\WebURLTupleBinding.java
.......\...\...\...\...\........\WorkQueues.java
.......\...\...\...\...\url\URLCanonicalizer.java
.......\...\...\...\...\...\WebURL.java
.......\...\...\...\...\.til\IO.java
.......\...\...\...\...\....\Util.java
.......\...\...\...\...\crawler4j\crawler
.......\...\...\...\...\example\advanced
.......\...\...\...\...\crawler4j
.......\...\...\...\...\example
.......\...\...\...\...\frontier
.......\...\...\...\...\url
.......\...\...\...\...\util
.......\...\...\...\ics
.......\...\...\uci
.......\...\edu
.......\src
crawler
.......\...\...\...\...\.........\.......\CrawlController.java
.......\...\...\...\...\.........\.......\HTMLParser.java
.......\...\...\...\...\.........\.......\IdleConnectionMonitorThread.java
.......\...\...\...\...\.........\.......\LinkExtractor.java
.......\...\...\...\...\.........\.......\Page.java
.......\...\...\...\...\.........\.......\PageFetcher.java
.......\...\...\...\...\.........\.......\PageFetchStatus.java
.......\...\...\...\...\.........\.......\WebCrawler.java
.......\...\...\...\...\example\advanced\Controller.java
.......\...\...\...\...\.......\........\CrawlStat.java
.......\...\...\...\...\.......\........\Downloader.java
.......\...\...\...\...\.......\........\MyCrawler.java
.......\...\...\...\...\frontier\DocIDServer.java
.......\...\...\...\...\........\Frontier.java
.......\...\...\...\...\........\WebURLTupleBinding.java
.......\...\...\...\...\........\WorkQueues.java
.......\...\...\...\...\url\URLCanonicalizer.java
.......\...\...\...\...\...\WebURL.java
.......\...\...\...\...\.til\IO.java
.......\...\...\...\...\....\Util.java
.......\...\...\...\...\crawler4j\crawler
.......\...\...\...\...\example\advanced
.......\...\...\...\...\crawler4j
.......\...\...\...\...\example
.......\...\...\...\...\frontier
.......\...\...\...\...\url
.......\...\...\...\...\util
.......\...\...\...\ics
.......\...\...\uci
.......\...\edu
.......\src
crawler