Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Or you could’ve used xpath and bs4 and have been done in an hour or two and have more understandable code.


it is not that easy, there is lazy loading in the page that is triggered by scroll of specific sections. You need to find clever way, no way to scrape with bs4, so tough with even selenium.




Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: