蜘蛛"（Spider）是Internet上一种很有用的程序，搜索引擎利用蜘蛛程序将Web页面收集到数据库，企业利用蜘蛛程序监视竞争对手的网站并跟踪变动，个人用户用蜘蛛程序下载Web页面以便脱机使用，开发者利用蜘蛛程序扫描自己的Web检查无效的链接……对于不同的用户，蜘蛛程序有不同的用途。那么，蜘蛛程序到底是怎样工作的呢？

蜘蛛是一种半自动的程序，就象现实当中的蜘蛛在它的Web（蜘蛛网）上旅行一样，蜘蛛程序也按照类似的方式在Web链接织成的网上旅行。蜘蛛程序之所以是半自动的，是因为它总是需要一个初始链接（出发点），但此后的运行情况就要由它自己决定了，蜘蛛程序会扫描起始页面包含的链接，然后访问这些链接指向的页面，再分析和追踪那些页面包含的链接。从理论上看，最终蜘蛛程序会访问到Internet上的每一个页面，因为Internet上几乎每一个页面总是被其他或多或少的页面引用。

-Spider "(Spider) is the Internet, a very useful procedure, the search engine spider programs will use to collect Web pages to the database, business process using spider to monitor a competitor s site and track changes in individual users to download Web pages using spider programs to off machine use, developers use Web spiders scan your check invalid links ... ... for different users, different use spider programs. So in the end is how the spider program work?

Spider is a semi-automatic procedure, as the reality of which the spider in its Web (web) on the same trip, spider also follow a similar approach in the Web link woven into the online travel. Spider is a semi-automatic procedure is, because it always requires an initial link (starting point), but then the operation will start from its own decision, and the spider program will scan the start page contains the link, and then visit these links point to the page, and then analyze and track those pages contain links. In theory, the