u/Low-Alarm272

I'm exceptional at these things and these things only:
- LLM integrations in systems
- AI related Cost cutting with your old laptop and ram sticks
- Technical writing (see my medium page)
- building in public as a full-stack developer on X.

My AI gitbub project has 30+ stars: https://github.com/workspace-dex/open-agent
Was working for a startup for 2 years as their tech lead.
Also, I do fiction writing as a side hobby and have worked with an indie game studio recently as freelance work.

If you have work related to these thing and ONLY these things, I'd love to hear about it!

Last night I updated llama.cpp after like 2 or 3 weeks. The results were really exciting for someone running a 35B model on 6GB RTX 3050.

Today I was able to get stable token speeds and they didn't fall down to 9 t/s while coding 1000+ lines of code.

Now I can increase my context window to 64k range and I'm still getting 19 t/s minimum. Before it would do down drastically to 4 t/s.

But now it gives a solid 26 t/s. In high context window worflows it falls by 5-7 t/s only. This means I can do 1000$ worth of coding work on my laptop for free.

Yes. The AI bubble will pop for sure if people realizes they can locally get near same quality of the their cloud subscriptions.

Here's how:

I've been running Qwen3.6-35B on an RTX 3050 6GB locally, really fast inside a self made cli tool and it's really good at keeping a stable compression system so the context isn't the issue.

Getting really decently good results on Q3 quant

Thank god llama.cpp exists.

And what's more fun is that I can test out ik_llama to get a few more tokens. This is more than enough for me.

My llama.cpp flags:

-c 45000

--n-gpu-layers 81

-- n-cpu-moe 25

--override-tensor "blk\\.(2\[0-9\]|3\[0-9\]|4\[0-6\])\\.ffn\_(gate\_up|down)\_exps\\.weight=CPU"

\-b 1024 -ub 512 \\

\--cache-type-k q4\_0 \\

\--cache-type-v q4\_0 \\

\--flash-attn on \\

\--cont-batching \\

\--threads 6 --threads-batch 6 \\

\--jinja \\

\--reasoning auto \\

\--ctx-checkpoints 10 \\

\--top-k 64 --top-p 0.75 \\

\--temp 0.7 \\

\--repeat-penalty 1.0 \\

\--cache-prompt

Ask away if you have any questions.

Thank god llama.cpp exists.

And what's more fun is that I can test out ik\_llama to get a few more tokens. This is more than enough for me.

I've been running this really fast inside a linux cli tool (I created it) and it's really good at keeping a stable compression system so the context isn't the issue.

Getting really decently good results on Q3 quant

My llama.cpp flags:

-c 18000

--n-gpu-layers 81

-- n-cpu-moe 25

--override-tensor "blk\\.(2\[0-9\]|3\[0-9\]|4\[0-6\])\\.ffn\_(gate\_up|down)\_exps\\.weight=CPU"

\-b 512 -ub 128 \\

\--cache-type-k q4\_0 \\

\--cache-type-v q4\_0 \\

\--flash-attn on \\

\--cont-batching \\

\--threads 6 --threads-batch 6 \\

\--jinja \\

\--reasoning auto \\

\--ctx-checkpoints 10 \\

\--top-k 64 --top-p 0.75 \\

\--temp 0.7 \\

\--repeat-penalty 1.0 \\

\--cache-prompt

Ask away if you have any questions.

Thank god llama.cpp exists.

And what's more fun is that I can test out ik_llama to get a few more tokens. This is more than enough for me.

I've been running this really fast inside a linux cli tool (I created it) and it's really good at keeping a stable compression system so the context isn't the issue.

Getting really decently good results on Q3 quant

My llama.cpp flags:

-c 18000 \

--n-gpu-layers 81 \

-- n-cpu-moe 25

--override-tensor "blk\.(2[0-9]|3[0-9]|4[0-6])\.ffn_(gate_up|down)_exps\.weight=CPU" \

-b 512 -ub 128 \

--cache-type-k q4_0 \

--cache-type-v q4_0 \

--flash-attn on \

--cont-batching \

--threads 6 --threads-batch 6 \

--jinja \

--reasoning auto \

--ctx-checkpoints 10 \

--top-k 64 --top-p 0.75 \

--temp 0.7 \

--repeat-penalty 1.0 \

--cache-prompt

Ask away if you have any questions.

I'll keep this short because I know self-promo posts are annoying.

I'm a full-stack developer with 2 years of production experience. My niche right now is LLM integrations — specifically the kind that don't require massive cloud spend. Self-hosted models, agent frameworks, AI workflows built into real products.

I also write. Technical articles, documentation, product explainers. If you need someone who can build the thing and explain it clearly, that's a rare combo and I have both.

If you're building an AI product and need a developer who can own it end-to-end — DM me. Open to discussing scope and pricing.

github.com/workspace-dex | medium.com/@strangelyevil

If anyone here needs a dev who actually understands the stack from model to frontend, I'm available for freelance work. DM.