Elegant Scraper and Crawler Framework for Golang
Pythonic HTML Parsing for Humans™
Scrapy, a fast high-level web crawling & scraping framework for Python.
A scalable web crawler framework for Java.