- Zac Clancy. What is Web Scraping? kite, 4/8/19.
- Danshu Jain. How to Scrape Websites Using Python. freecodecamp, 4/8/19.
- Colin OKeefe. Practical Introduction to Web Scraping in Python. Real Python.
- Kevin Sahn. Web Scraping 101 with Python. ScrapingBee, 8/2019.
- Pierre de Wulf. Web Scraping 101 with Python. Daolf, 8/2019.
Tools
Scrapy
- Scrapy – Stars: 39.4k – Updated: 1/2021 – Checked: 1/2021 – “An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.”
- Daniel Ni. The Ultimate Guide to Building Scalable Web Scrapes with Scrapy. smashing magazine, 7/2019.
- Zac Clancy. Web Scraping with Scrapy: Advanced Examples. kite, 4/2019.
- Portia – Visual scraping for Scrapy. 7/2019.
- Web Scraping with Scrapy and MongoDB. Real Python.
- Web Scraping and Crawling with Scrapy and MongoDB. Real Python.
Beautiful Soup
- Beautiful Soup – Launchpad – Updated: 10/2020 – Checked: 1/2021 – “Beautiful Soup is a Python library designed for quick turnaround projects like screen-scraping.”
- Philip Kiely. DIY Data: Web Scraping with Python and Beautiful Soup. floydhub, 2019.
- Martin Breus. Beautiful Soup: Build a Web Scraper with Python. real python, 12/2019.
- This article covers using BS with requests and pprint on Monster’s job site. Good tutorial.
- Robley Gori. Web Scraping with Python: A Tutorial on the BeautifulSoup HTML Parser. kite, 10/2018.
- Julia Kho. How to Web Scrape with Python in 4 Minutes. towardsdatascience, 9/2018.
Other Tools
- Newspaper3k – Stars: 10.6k – Updated: 9/2020 – Checked: 1/2021 – Article scraping and curation.
- PySpider – Stars: 14.8k – Updated: 8/2020 – Checked: 1/2021 – “A Powerful Spider(Web Crawler) System in Python.”