Practical Web Scraping for Data Science: Best Practices and Examples with Python 9781484235812, 9781484235829, 1484235819, 1484235827

This book provides a complete and modern guide to web scraping, using Python as the programming language, without glossi

429 82 5MB

English Pages XVI, 306 Seiten 35 Illustrationen) [313] Year 2018

Table of contents :
Table of Contents......Page 5
About the Authors......Page 8
About the Technical Reviewer......Page 9
Introduction......Page 10
Part I: Web Scraping Basics......Page 14
1.1 What Is Web Scraping?......Page 15
1.1.1 Why Web Scraping for Data Science?......Page 16
1.1.2 Who Is Using Web Scraping?......Page 17
1.2.1 Setting Up......Page 20
1.2.2 A Quick Python Primer......Page 21
2.1 The Magic of Networking......Page 36
2.2 The HyperText Transfer Protocol: HTTP......Page 39
2.3 HTTP in Python: The Requests Library......Page 45
2.4 Query Strings: URLs with Parameters......Page 50
3.1 Hypertext Markup Language: HTML......Page 60
3.2 Using Your Browser as a Development Tool......Page 62
3.3 Cascading Style Sheets: CSS......Page 67
3.4 The Beautiful Soup Library......Page 72
3.5 More on Beautiful Soup......Page 83
Part II: Advanced Web Scraping......Page 89
4.1 Working with Forms and POST Requests......Page 90
4.2 Other HTTP Request Methods......Page 106
4.3 More on Headers......Page 109
4.4 Dealing with Cookies......Page 117
4.5 Using Sessions with Requests......Page 128
4.6 Binary, JSON, and Other Forms of Content......Page 130
5.1 What Is JavaScript?......Page 136
5.2 Scraping JavaScript......Page 137
5.3 Scraping with Selenium......Page 143
5.4 More on Selenium......Page 157
6.1 What Is Web Crawling?......Page 164
6.2 Web Crawling in Python......Page 167
6.3 Storing Results in a Database......Page 170
Part III: Managerial Concerns and Best Practices......Page 182
7.1 The Data Science Process......Page 183
7.2 Where Does Web Scraping Fit In?......Page 187
7.3 Legal Concerns......Page 189
8.1.1 Alternative Python Libraries......Page 195
8.1.3 Caching......Page 196
8.1.4 Proxy Servers......Page 197
8.1.5 Scraping in Other Programming Languages......Page 198
8.1.7 Graphical Scraping Tools......Page 199
8.2 Best Practices and Tips......Page 201
Chapter 9: Examples......Page 204
9.1 Scraping Hacker News......Page 206
9.2 Using the Hacker News API......Page 208
9.3 Quotes to Scrape......Page 209
9.4 Books to Scrape......Page 213
9.5 Scraping GitHub Stars......Page 216
9.6 Scraping Mortgage Rates......Page 221
9.7 Scraping and Visualizing IMDB Ratings......Page 227
9.8 Scraping IATA Airline Information......Page 229
9.9 Scraping and Analyzing Web Forum Interactions......Page 235
9.10 Collecting and Clustering a Fashion Data Set......Page 244
9.11 Sentiment Analysis of Scraped Amazon Reviews......Page 248
9.12 Scraping and Analyzing News Articles......Page 259
9.13 Scraping and Analyzing a Wikipedia Graph......Page 278
9.14 Scraping and Visualizing a Board Members Graph......Page 285
9.15 Breaking CAPTCHA’s Using Deep Learning......Page 288
Index......Page 306

Practical Web Scraping for Data Science: Best Practices and Examples with Python
9781484235812, 9781484235829, 1484235819, 1484235827

Author / Uploaded
Broucke
Seppe vanden;Baesens
Bart

0 0 0
Like this paper and download? You can publish your own PDF file online for free in a few minutes! Sign Up

File loading please wait...

Recommend Papers

A Python Guide for Web Scraping 9789390684991

489 43 2MB Read more

Web Scraping with Python: Data Extraction from the Modern Web, 3rd Edition [3 ed.] 9781098145354

If programming is magic, then web scraping is surely a form of wizardry. By writing a simple automated program, you can

116 16 Read more

Web Scraping with Python: Data Extraction from the Modern Web, 3rd Edition [3 ed.] 9781098145354

If programming is magic, then web scraping is surely a form of wizardry. By writing a simple automated program, you can

99 42 10MB Read more

Web scraping with Python: collecting more data from the modern web [2nd edition] 9781491985571, 1491985577

If programming is magic then web scraping is surely a form of wizardry. By writing a simple automated program, you can q

521 23 5MB Read more

Python Web Scraping: Hands-on data scraping and crawling using PyQT, Selnium, HTML and Python, 2nd Edition 9781786462589, 1786462583

Successfully scrape data from any website with the power of Python 3.x Key FeaturesA hands-on guide to web scraping usin

105 64 15MB Read more

A Python Guide for Web Scraping: Explore Python Tools, Web Scraping Techniques, and How to Automata Data for Industrial Applications (English Edition) 9390684994, 9789390684991

Get hands-on training on any web crawling/scraping tool and uses of web scraping in the real-time industry Key Features

461 117 2MB Read more

Flask Web Development with Python: Build Web Applications Python with Exercises and Examples

Python is one of the most versatile high-level programming languages ever developed. Rather than having to jump into str

119 69 2MB Read more

Python Programming: The Ultimate Crash Course for Learning Python Quickly, with Practical Examples and Coding Language Tricks for Beginners. Computer Programming for Data Science and Machine Learning

Do you want to learn Python programming? Or you just want to improve your skills? In either way this book is right for y

680 65 2MB Read more

Python for Data Science : Clear and Complete Guide to Data Science and Analysis with Python

Are you interested in learning data science with Python? Do you want to know what you need to get started? Then you have

1,024 162 2MB Read more

Python Data Science: Deep Learning Guide for Beginners with Data Science. Python Programming and Crush Course.

Everything You Need to Know About Python Data ScienceDo you want to get started on Python Data Science?Wondering what yo

274 130 4MB Read more