Scrapetron

projects

Web scraping framework

PyPISource

Install

pip
$

Scraper Demo

CLI Reference

scrapetron init <name>Initialize a new scraping project
scrapetron runRun the configured scraper
scrapetron export --format jsonExport results as JSON

About

A Python web scraping framework built on BeautifulSoup and Scrapy with a simplified configuration interface. I kept rewriting the same scraper boilerplate for different projects, so I packaged the pattern into something reusable.

Highlights

  • Available on PyPI
  • Simplified configuration over raw Scrapy
  • 20 users