文件名称:spider2006
- 所属分类:
- 搜索引擎
- 资源属性:
- [Windows] [Visual.Net] [源码]
- 上传时间:
- 2012-11-26
- 文件大小:
- 31kb
- 下载次数:
- 0次
- 提 供 者:
- ros****
- 相关连接:
- 无
- 下载说明:
- 别用迅雷下载,失败请重下,重下不扣分!
介绍说明--下载内容均来自于网络,请自行研究使用
可配置:线程数、线程等待时间,连接超时时间,可爬取文件类型和优先级、下载目录等。
状态栏显示统计信息:排入队列URL数,已下载文件数,已下载总字节数,CPU使用率和可用内存等。
有偏好的爬虫:可针对爬取的资源类型设置不同的优先级。
健壮性:十几项URL正规化策略以排除冗余下载、爬虫陷阱避免策略的使用等、多种策略以解析相对路径等。
较好的性能:基于正则表达式的页面解析、适度加锁、维持HTTP连接等。
-C# spider.
状态栏显示统计信息:排入队列URL数,已下载文件数,已下载总字节数,CPU使用率和可用内存等。
有偏好的爬虫:可针对爬取的资源类型设置不同的优先级。
健壮性:十几项URL正规化策略以排除冗余下载、爬虫陷阱避免策略的使用等、多种策略以解析相对路径等。
较好的性能:基于正则表达式的页面解析、适度加锁、维持HTTP连接等。
-C# spider.
(系统自动生成,下载前可以参看下载内容)
下载文件列表
spider_demo
...........\spider_demo.sln
...........\spider_demo.suo
...........\spider_demo
...........\...........\base64.cs
...........\...........\workThread.cs
...........\...........\spider_demo.cs
...........\...........\spider.cs
...........\...........\spider_demo.csproj
...........\...........\spider_demo.Designer.cs
...........\...........\Listener.cs
...........\...........\spider_demo.resx
...........\...........\bin
...........\...........\...\Release
...........\...........\...\.......\WebRegex.dll
...........\...........\...\.......\html
...........\...........\...\.......\yy.txt
...........\...........\...\Debug
...........\...........\...\.....\WebRegex.dll
...........\...........\...\.....\yy.txt
...........\...........\...\.....\html
...........\...........\obj
...........\...........\...\spider_demo.csproj.FileList.txt
...........\...........\...\Release
...........\...........\...\.......\TempPE
...........\...........\...\.......\......\Properties.Resources.Designer.cs.dll
...........\...........\...\.......\Refactor
...........\...........\...\Debug
...........\...........\Properties
...........\...........\..........\AssemblyInfo.cs
...........\...........\..........\Resources.resx
...........\...........\..........\Resources.Designer.cs
...........\...........\..........\Settings.settings
...........\...........\..........\Settings.Designer.cs
...........\...........\HtmlAnalyzer.cs
...........\...........\GetHtml.cs
...........\...........\Program.cs
...........\readme.txt
...........\spider_demo\CRC.cs
...........\spider_demo.sln
...........\spider_demo.suo
...........\spider_demo
...........\...........\base64.cs
...........\...........\workThread.cs
...........\...........\spider_demo.cs
...........\...........\spider.cs
...........\...........\spider_demo.csproj
...........\...........\spider_demo.Designer.cs
...........\...........\Listener.cs
...........\...........\spider_demo.resx
...........\...........\bin
...........\...........\...\Release
...........\...........\...\.......\WebRegex.dll
...........\...........\...\.......\html
...........\...........\...\.......\yy.txt
...........\...........\...\Debug
...........\...........\...\.....\WebRegex.dll
...........\...........\...\.....\yy.txt
...........\...........\...\.....\html
...........\...........\obj
...........\...........\...\spider_demo.csproj.FileList.txt
...........\...........\...\Release
...........\...........\...\.......\TempPE
...........\...........\...\.......\......\Properties.Resources.Designer.cs.dll
...........\...........\...\.......\Refactor
...........\...........\...\Debug
...........\...........\Properties
...........\...........\..........\AssemblyInfo.cs
...........\...........\..........\Resources.resx
...........\...........\..........\Resources.Designer.cs
...........\...........\..........\Settings.settings
...........\...........\..........\Settings.Designer.cs
...........\...........\HtmlAnalyzer.cs
...........\...........\GetHtml.cs
...........\...........\Program.cs
...........\readme.txt
...........\spider_demo\CRC.cs