文件名称:nwebcrawlercSharp
- 所属分类：
- C#编程
- 资源属性：
- [Windows] [Visual.Net] [源码]
- 上传时间：
- 2012-11-26
- 文件大小：
- 2.02mb
- 下载次数：
- 0次
- 提 供 者：
- c*
- 相关连接：
- 无
- 下载说明：
- 别用迅雷下载，失败请重下，重下不扣分！
下载
别用迅雷、360浏览器下载。
如迅雷强制弹出，可右键点击选“另存为”。
失败请重下，重下不扣分。
如迅雷强制弹出，可右键点击选“另存为”。
失败请重下，重下不扣分。
介绍说明－－下载内容均来自于网络，请自行研究使用
可配置：线程数、线程等待时间，连接超时时间，可爬取文件类型和优先级、下载目录等。
状态栏显示统计信息：排入队列URL数，已下载文件数，已下载总字节数，CPU使用率和可用内存等。
有偏好的爬虫：可针对爬取的资源类型设置不同的优先级。
健壮性：十几项URL正规化策略以排除冗余下载、爬虫陷阱避免策略的使用-c# crawler Threads
状态栏显示统计信息：排入队列URL数，已下载文件数，已下载总字节数，CPU使用率和可用内存等。
有偏好的爬虫：可针对爬取的资源类型设置不同的优先级。
健壮性：十几项URL正规化策略以排除冗余下载、爬虫陷阱避免策略的使用-c# crawler Threads
(系统自动生成,下载前可以参看下载内容)
下载文件列表
BuildProcessTemplates
.....................\DefaultTemplate.xaml
.....................\UpgradeTemplate.xaml
data
....\crawlerdb.s3db
....\pdc_09.txt
....\sina_12_28.txt
....\spider.ico
lib
...\x64
...\...\System.Data.SQLite.dll
...\...\System.Data.SQLite.lib
...\x86
...\...\System.Data.SQLite.dll
...\...\System.Data.SQLite.lib
setup
.....\build.cmd
.....\Files.wxs
.....\license.rtf
.....\Setup.wxs
.....\wix3
.....\....\candle.exe
.....\....\darice.cub
.....\....\light.exe
.....\....\wconsole.dll
.....\....\winterop.dll
.....\....\wix.dll
.....\....\WixUIExtension.dll
src
...\NWebCrawler.sln
...\NWebCrawlerLib
...\..............\Common
...\..............\......\FileSystemUtility.cs.cs
...\..............\......\Logger.cs
...\..............\......\PriorityQueue.cs
...\..............\......\SQLiteUtility.cs
...\..............\CrawlerThread.cs
...\..............\CrawlHistroyEntry.cs
...\..............\Downloader.cs
...\..............\EventStream.cs
...\..............\MemCache.cs
...\..............\NWebCrawlerLib.csproj
...\..............\PageRank.cs
...\..............\Parser.cs
...\..............\Program.cs
...\..............\Properties
...\..............\..........\AssemblyInfo.cs
...\..............\..........\Resource.en-US.resx
...\..............\..........\Resource.zh-CN.resx
...\..............\ResourceCulture.cs
...\..............\Settings.cs
...\..............\UrlQueueManager.cs
...\..............\Utility.cs
...\NWebCrawler
...\...........\app.config
...\...........\config.ini
...\...........\MainForm.cs
...\...........\MainForm.Designer.cs
...\...........\MainForm.resx
...\...........\NWebCrawler.csproj
...\...........\Program.cs
...\...........\Properties
...\...........\..........\AssemblyInfo.cs
...\...........\..........\Resources.Designer.cs
...\...........\..........\Resources.resx
...\...........\..........\Settings.Designer.cs
...\...........\..........\Settings.settings
...\...........\SettingsForm.cs
...\...........\SettingsForm.Designer.cs
...\...........\SettingsForm.resx
test
....\NWebCrawler.Test
....\................\NWebCrawler.Test.sln
....\................\NWebCrawler.Test
....\................\................\NormalizeTest.cs
....\................\................\NWebCrawler.Test.csproj
....\................\................\ParserTest.cs
....\................\................\Properties
....\................\................\..........\AssemblyInfo.cs
.....................\DefaultTemplate.xaml
.....................\UpgradeTemplate.xaml
data
....\crawlerdb.s3db
....\pdc_09.txt
....\sina_12_28.txt
....\spider.ico
lib
...\x64
...\...\System.Data.SQLite.dll
...\...\System.Data.SQLite.lib
...\x86
...\...\System.Data.SQLite.dll
...\...\System.Data.SQLite.lib
setup
.....\build.cmd
.....\Files.wxs
.....\license.rtf
.....\Setup.wxs
.....\wix3
.....\....\candle.exe
.....\....\darice.cub
.....\....\light.exe
.....\....\wconsole.dll
.....\....\winterop.dll
.....\....\wix.dll
.....\....\WixUIExtension.dll
src
...\NWebCrawler.sln
...\NWebCrawlerLib
...\..............\Common
...\..............\......\FileSystemUtility.cs.cs
...\..............\......\Logger.cs
...\..............\......\PriorityQueue.cs
...\..............\......\SQLiteUtility.cs
...\..............\CrawlerThread.cs
...\..............\CrawlHistroyEntry.cs
...\..............\Downloader.cs
...\..............\EventStream.cs
...\..............\MemCache.cs
...\..............\NWebCrawlerLib.csproj
...\..............\PageRank.cs
...\..............\Parser.cs
...\..............\Program.cs
...\..............\Properties
...\..............\..........\AssemblyInfo.cs
...\..............\..........\Resource.en-US.resx
...\..............\..........\Resource.zh-CN.resx
...\..............\ResourceCulture.cs
...\..............\Settings.cs
...\..............\UrlQueueManager.cs
...\..............\Utility.cs
...\NWebCrawler
...\...........\app.config
...\...........\config.ini
...\...........\MainForm.cs
...\...........\MainForm.Designer.cs
...\...........\MainForm.resx
...\...........\NWebCrawler.csproj
...\...........\Program.cs
...\...........\Properties
...\...........\..........\AssemblyInfo.cs
...\...........\..........\Resources.Designer.cs
...\...........\..........\Resources.resx
...\...........\..........\Settings.Designer.cs
...\...........\..........\Settings.settings
...\...........\SettingsForm.cs
...\...........\SettingsForm.Designer.cs
...\...........\SettingsForm.resx
test
....\NWebCrawler.Test
....\................\NWebCrawler.Test.sln
....\................\NWebCrawler.Test
....\................\................\NormalizeTest.cs
....\................\................\NWebCrawler.Test.csproj
....\................\................\ParserTest.cs
....\................\................\Properties
....\................\................\..........\AssemblyInfo.cs