Open source repositories tagged with #crawling, ranked by health score.
Apache Nutch is an extensible and scalable web crawler