I want to automatically scrape my news, podcasts and youtube in the morning. What kind of tools do you use for similar uses?
So, I am currently using ChannelTube to do a daily scrape of Youtube to download the latest vids from channels I like. These get downloaded directly into a special Jellyfin library and Jellyfin gets ping to sync when the downloads are done.
What I would like to achieve is a globally automated set of scrapers to get different news, videos, podcasts and maybe even social media feeds, all so I can avoid using websites that are assaulting me with content and navigating my attention into things I'm not actively interested in.
What kind of tools and software do you use to control your media consumption? Any ideas on best practices to avoid these services getting flagged as bots? I frequently have to update the cookies on ChannelTube to avoid the client getting rejected. I could probably script something to ensure this gets regularly updated.
Anyone use RSS feeds? Are they still relevant in current year?
Any recommendations for dashboards? I use Glance atm which likely has widgets I could use to filter my news and new videos.
I'm open to scripting my own things but I'm curious what exists out there to scrape websites and download media on a schedule. Anything useful to organize playlists or cues? Would love to sync it to my smart speaker or smart screens, have a little good morning news feed.
Not particularly interested in agents. I don't trust them to be exhaustive or make the right choices, not yet anyways. I'd trust them even less to not accidentally DDOS smaller websites.
Brainstorm with me. 😄
TL;DR: I want to download my news, podcasts and youtube so I don't go on websites that assault my senses and make me forget what I was looking for.