Hello everybody, this is my first post here. I wanted to ask you guys if you have some resources for exercising my english because i will take the CAE Cambridge exam in almost 3 weeks. I know that you must get asked this kind of thing everyday, but i couldn't find a good answer for this particular question.
If it helps, i have a problem with the second exercise in the use of english part, which involves finding a word that fits that field, without any help.
Most AI coding tools (Cursor, Aider, Claude Code) assume you have a 200k-token model. If you're running local LLMs through Ollama or LM Studio, or hitting free-tier cloud APIs like Groq or OpenRouter, you've got around 8k tokens to work with. That doesn't fit a whole project, barely fits a single large file.
I spent the last few weeks building a CLI coding agent that's designed around the 8k constraint instead of fighting it. Wanted to share what I learned, because some of it surprised me.
The core insight: the LLM never needs to see your whole project.
Most agents try to stuff as much context as possible into a single call. With 8k tokens that's a non-starter. The approach that worked for me is splitting the work into roles:
This split means the LLM only ever reasons about a small, bounded amount of code at any one time. The planner doesn't need to see code at all (just file summaries), and the executor only sees one file. Multi-file refactors stop being a context-window problem and become a scheduling problem.
Token budgeting has to be enforced in code, not promised in a prompt.
Every LLM call goes through a canFit() check that measures: system prompt + reserved output tokens + memory + actual code. If the code doesn't fit, the agent automatically falls back to a per-file line index (generated once for files over ~150 lines) and pulls only the relevant section.
Concrete budget math for 8192 tokens:
Parallel execution is the speed multiplier that makes 8k usable.
Because each executor sees only one file, independent edits across files can run simultaneously. A 5-file refactor that would be slow if run sequentially completes in roughly the time of the longest single edit. The dependency graph (built in pure code from the planner's task list) decides which tasks have to wait for which.
A few things that tripped me up along the way:
action_type: "query" field to the planner's output that routes through a separate code path that never touches disk.What I'm still figuring out:
Whether the planner/executor split scales cleanly to codebases over 50 files. The dependency graph stays manageable, but the project map starts costing real tokens once you have enough folders. Currently dropping folder context first when budget is tight, but that means deeper edits get less context. Curious if anyone else has run into this and how they handle it.
Open-sourced the implementation if anyone wants to dig in: https://github.com/razvanneculai/litecode
Most AI coding tools (Cursor, Aider, Claude Code) assume you have a 200k-token model. If you're running local LLMs through Ollama or LM Studio, or hitting free-tier cloud APIs like Groq or OpenRouter, you've got around 8k tokens to work with. That doesn't fit a whole project, barely fits a single large file.
I spent the last few weeks building a CLI coding agent that's designed around the 8k constraint instead of fighting it. Wanted to share what I learned, because some of it surprised me.
The core insight: the LLM never needs to see your whole project.
Most agents try to stuff as much context as possible into a single call. With 8k tokens that's a non-starter. The approach that worked for me is splitting the work into roles:
This split means the LLM only ever reasons about a small, bounded amount of code at any one time. The planner doesn't need to see code at all (just file summaries), and the executor only sees one file. Multi-file refactors stop being a context-window problem and become a scheduling problem.
Token budgeting has to be enforced in code, not promised in a prompt.
Every LLM call goes through a canFit() check that measures: system prompt + reserved output tokens + memory + actual code. If the code doesn't fit, the agent automatically falls back to a per-file line index (generated once for files over ~150 lines) and pulls only the relevant section.
Concrete budget math for 8192 tokens:
Parallel execution is the speed multiplier that makes 8k usable.
Because each executor sees only one file, independent edits across files can run simultaneously. A 5-file refactor that would be slow if run sequentially completes in roughly the time of the longest single edit. The dependency graph (built in pure code from the planner's task list) decides which tasks have to wait for which.
A few things that tripped me up along the way:
action_type: "query" field to the planner's output that routes through a separate code path that never touches disk.What I'm still figuring out:
Whether the planner/executor split scales cleanly to codebases over 50 files. The dependency graph stays manageable, but the project map starts costing real tokens once you have enough folders. Currently dropping folder context first when budget is tight, but that means deeper edits get less context. Curious if anyone else has run into this and how they handle it.
Open-sourced the implementation if anyone wants to dig in: https://github.com/razvanneculai/litecode