VehiclesFashionRecipesBlogsHuntTravelsSportFunHandmadeITEducation
Mini-Games
x

x
zakruti.com » IT - Software » freeCodeCamp.org
Web Scraping with Node.js

Web Scraping with Node.js

FBTwitterReddit

video description

Rating: 4.0; Vote: 1
Learn how to do basic web scraping using Node.js in this tutorial. The request-promise and cheerio libraries are used. - Github: https://github.com/beaucarnes/fcc-project-tutorials/blob/master/node-web-scraping/index.js - Article on client-side web scraping : https://medium.freecodecamp.org/client-side-web-scraping-with-javascript-using-jquery-and-regex-5b57a271cb86 - Beau Carnes on
Date: 2022-03-14

Comments and reviews: 10


Say if I wanted to make a website displaying this data that updated every time the page was refreshed. Generally how would that be set up? Would this code be running on the server? How would it send the data to the HTML? What would be the role of client side javascript vs node?
reply

Thanks a bunch for the tutorial! You explained it clearly & helped me in understanding this tutorial -
UPDATE:
When I tried to run this code, I got an error that says:
-TypeError: data.directory_items is not iterable-
How do I fix this?

reply

Awesome video Beau! I followed along but was trying to get data from my profile...I managed to get the HTML from the page but couldn't make sense of it so I went ahead with the jQuery way in the article you show. Thanks for the great resource!
reply

I want to create a paid service for car dealers I want to scrape car offers from different websites and the client can search all this sites with 1 click on my Site....Is something like this legal without API ?
reply

you are probably parsing an easy web site but the real worls is diffrent what about tokens, cookies .. not to mentioned that most website load their contect using javasciprt and simple request might not work..
reply

Someone know why he uses a network request link to get all users but go directly on the profile link to get challenges data? Isn't the challenges data on the profile page also dynamic?
reply

If you use this in an app, you would get a CROSS-origin read blocking error. How can you resolve this error and use this nice tutorial in a real app? Thanks -
reply

It's a very good basic tutorial to start off, the only thing that could be improved is the use of async await which I am also doing on my videos on NodeJs Scraping
reply

Nice video. Thanks!
If somebody wants to see alternative video on this topic, i have one. Unfortunatelly with eng subtitles for now.
Cheers

reply

Scrap data from websites where you have to perform several clicks and then scrap data. This is child play what you are showing : )
reply
Add a review, comment






Other channel videos