[Data on the Mind 2017] Collecting data from the web

ดูวิดีโอตอนนี้ [Data on the Mind 2017] Collecting data from the web

Abstract: This workshop will explore different ways to collect data from the web with Python. Have you ever needed to copy and paste hundreds (or thousands!) of tables on different web pages? Or click through combinations of dropdown menu selections and download files? Are you interested in collecting social media or news data? After this workshop, you’ll be well on your way to automating these processes. We will first consider getting data from RESTful APIs. We will walk through using the documentation to build a query for a GET request. We’ll then write our response to a CSV spreadsheet. We’ll then discuss web scraping, keeping in mind the Terms of Service for websites and ensuring we are not in violation. We will look at two ways of scraping: 1) parsing the HTML response of a GET request using BeautifulSoup, and 2) utilizing the Selenium web driver to interact directly with dynamic web content.

Instructor: Christopher Hench (University of California, Berkeley)

—

Part of the Data on the Mind 2017 summer workshop:

Funded by the Estes Fund:

Co-produced with the Berkeley D-Lab:

Organized in collaboration with Data on the Mind:

Videography by DeNoise Studios:

Workshop hashtag: #dataonthemind

“[Data on the Mind 2017] Collecting data from the web“, จากแหล่งที่มา: https://www.youtube.com/watch?v=do5Bom0OdjQ

แฮชแท็กสำหรับ [Data on the Mind 2017] Collecting data from the web: #Data #Mind #Collecting #data #web

บทความ [Data on the Mind 2017] Collecting data from the web มีเนื้อหาดังต่อไปนี้: Abstract: This workshop will explore different ways to collect data from the web with Python. Have you ever needed to copy and paste hundreds (or thousands!) of tables on different web pages? Or click through combinations of dropdown menu selections and download files? Are you interested in collecting social media or news data? After this workshop, you’ll be well on your way to automating these processes. We will first consider getting data from RESTful APIs. We will walk through using the documentation to build a query for a GET request. We’ll then write our response to a CSV spreadsheet. We’ll then discuss web scraping, keeping in mind the Terms of Service for websites and ensuring we are not in violation. We will look at two ways of scraping: 1) parsing the HTML response of a GET request using BeautifulSoup, and 2) utilizing the Selenium web driver to interact directly with dynamic web content.