Learn about website crawlers and similar tools. Plus, how to crawl a website effectively and easily.