Definition of Spider

Spiders are known by many names including crawlers, web spiders, web robots, bots, ants, automatic indexers, and robots. Spiders are used by the search engines to crawl though the Web in search of information for the search indexes. They are programs that browse through the World Wide Web in a methodical style to access data, both new and old. Spiders crawl from one site to another following the links that are part of the website.

The information that is collected is presented in search engine page results whenever a web user conducts a search on the Internet. The results the search engine returns are relevant to the keywords used for the search. Some search engines are quite large and therefore, they use more than one spider or web crawler to look for information. Spiders are one of the best ways to guarantee up-to-date data, images, videos, or information.

External links:

• Webopedia explanation of Spider
• Wiki definition of Web Crawler
• About.com Spider explanation