I'm running on a money goal, so i'm up to do many of the tech roles at really feasible rates.
- Sadly no new user in last 24 hour.
- Made a instagram page and hoping that reels go viral.
- Full rollercoaster ride.
- Found NO new bugs in last 48 hours.
- Looking for people to brutally roast and give reality check
tasknode.io - best research platform
I own research platform (tasknode). I'm heavily dependent on APIs, one API for websearch and multiple LLM calls for processing web content, judging and contradiction.
I saw on hf and kaggle that multiple datasets related to news, opinions and other bunch of categories are available.
For a long run, should I get as much as datasets possible, process of them with LLM, classify important one. after months, we might have perfect dataset to finetune on base model.
Pros:
- reduction of cost alot
- faster response
Cons:
- processing that much data will cost lot of inference (eventually more $$)
- there are many cons tbh.
What should be right approach?