3 part scraping workflow help
Hey all, I’ll try to keep this brief. Long story short I’m trying to learn how to use python without relying on vibecoding out the wazoo.
One of the ideas I had is a three part workflow that would compare mathematics requirements for different electrical engineering majors at different universities.
Scour the internet from a base browser or landing pages of preselected universities (if moving internally is possible), to find electrical engineering major information, and, output those links to a csv.
From the link CSV, gather the relevant information about math courses.
Output the data into another CSV AND json file (I want to be able to customize the csv output columns from within the script). From the csv, I want hyperlinks for the specific math courses (or, more links to be scraped if the info comes from a PDF curriculum url).
4th (optional): it’d be cool if a local llm could compare and reason the similarities/differences between the math courses.
I work in helpdesk but am otherwise a beginner. What is the best place to learn how to do these functions, and what are my options with making this?