Web scraping is the process of scraping websites and extracting desired data from the same, and in this course, you will learn and master web scraping using python and scrapy with a step by step and in depth guide.

A Step By Step Guide
Assuming that you don t know anything about web scraping, scrapy python web scraping, or even web scraping meaning we will start from the complete basics. In the first section, you will learn about the web scraping process step by step (with infographics no code), how to scrape data from websites and how to use scrapy for the same (i.e. scrapy meaning).

After getting the basics clear and having an idea of how web scraping works, we will start web scraping using python and scrapy framework! Again, we will move step by step and perform each step learned in the basics with bite sized lessons. We will take it slow so that its easier for you to understand each and every step involved in scraping and extracting data from websites.

Web Scraping and Scrapy Essentials
Having built an actual web scraper, you will get an idea of how web scraping works firsthand. Now it is crucial to cover the essential concepts of web scraping and scrapy, which we will do next.
CSS Selectors to select web elements
XPath to select web elements
Scrapy Shell to test and verify selectors
Items to organize extracted data
Load Items with ItemLoaders with input and output Processors
Export data to JSON, CSV, XLSX (Excel) and XML file formats
Save extracted data to online databases like MongoDB using ItemPipelines

Master Web Scraping In Depth
Learning how to scrape websites and the essentials already makes you a complete web scraper but, we will take this even further and learn the advanced web scraping techniques to become an expert!
Follow links in a webpage to another page
Crawl multiple pages and extract data i.e. Pagination
Scrape data using Regular Expressions (RegEx)
Extract Data From HTML Tables
Login Into Websites Using Scrapy FormRequest
Bypass CSRF protected Login forms
Scrape Dynamic or JavaScript Rendered Websites using Scrapy Playwright
Interact with web elements like fill forms, click buttons, etc.
Handle Infinite Scroll websites
Wait For Elements when contents/data takes time to load
Take Screenshot of websites
Save websites as PDF
Identify API calls from websites and scrape data from APIs
Use middleware in a scrapy project
Configure settings in a scrapy project
Use and Rotate User Agents and Proxies
Web scraping Best Practices

Real World Projects
After master web scraping, we need projects to get started! That is why you will perform three projects as well:
Champions League Table [ ESPN ]
Product Tracker [ Amazon ]
Scraper Application [ GUI ]

Join us in this in depth course where you will learn about web scraping from scratch and master the process of extracting data from websites step by step. Check out the preons to get started and learn how web scraping works! See you there~
–
SKILLS YOU WILL GAIN
Web Scraping
Scrapy Library
WHAT YOU WILL LEARN
Web Crawling and Scraping with Scrapy
Select Data Using CSS Selectors and XPath
Deal Websites With Pagination and Infinite Scroll
Log In Into Websites to Scrape Data