简易采集爬虫

# 1.我只做了一个收集url的功能，如果需要将数据入库，可在 parseData 函数里面写处理代码

# 2.需要sqlite3或者pysqlite支持

# 3.可以在DreamHost.com空间上面运行

# 4.可以修改User-Agent冒充搜索引擎蜘蛛

# 5.可以设置暂停的时间，控制采集速度-

Simple collection of reptiles



# 1 I just made a collection of URL function, if the data need to be put in storage, can be written in parseData function inside the processing code



# 2 requires SQLite3 or pysqlite support



# 3 can be run in DreamHost.com space



# 4 can modify User-Agent posing as a search engine spider



# 5 can set pause time, control the acquisition speed