I’ll preface this by saying that I haven’t built a web scraper before. Please feel free to warn me that it takes more effort than I think. I am flying blind here.
I would like to state my reasons though.
1.) I need a project. Something to allow me to hone my skills, use what I’ve learned, and display them.
2.) I despise job sites (especially the larger more well-known ones) that readily post jobs from third parties that either are farming data, are looking to scam you, or allow reposts from larger corporations every week (they’re not looking to hire and are either also farming data, are required by legislation to post the job, or have some ephemeral third reason to waste applicants’ time).
3.) IF, and this is a very big IF, I finish this project, or I get it to a completed state, then I’ve made a useful tool for myself to be gainfully employed, and maybe for others to be as well. Doing some good in the tech world would feel… rewarding.
I’m posting here to ask with my limited knowledge, whether I’m wasting my time or if there’s an unseen oversight in my decision with my limited knowledge relating to web scraping.
As for my technical knowledge, I’ve worked professionally with Python, Typescript, JavaScrip, React, MySQL, PostgreSQL, for about a year and I’ve gotten familiar with GCP for about a year and a half. Prior to that, I’ve used Ruby on Rails, C#, a smidge of C++, Visual Basic, and I’ve started dipping my toe into Rust.
If anyone feels free to, please let me know if I’m in over my head, if there are key areas I’m not taking into account, or if I would be better spending my time elsewhere. Thank you to anyone who replies.