u/BlueLagoon226

AI agents can help get the ball rolling, but I don’t think they work as the final approach.

I’ve seen people treat them like they can just hand over a finished scraper on the first go. The first draft might look decent, but once you test it you still have to clean up the logic and figure out what it misunderstood.

Sometimes the back and forth takes just as long as writing it yourself. At the end of the day its still just a tool to help with some gaps but it shouldn't be blindly trusted.

I’ve been seeing a lot more Playwright lately, but still plenty of people sticking with Requests/BS4 or Scrapy when the site doesn’t need a browser.

I’m mostly using Python with Requests and BS4 for simple stuff, then Playwright when a site forces it.

Always interesting to see what people actually use once the scraper has to run more than once.