Spiders are automatically sent out by search engines to follow hyperlinks from one document to the next over the web. When a spider finds a new site, it sends information back to the main site to be indexed. Spiders also search previously spidered sites because information is ever changing and it will update cataloged sites.