Login

u/Feeling-Ad972

▲ 1 r/AIDiscussion

I have been scaffolding a project lately and while the dev speed was great the performance was garbage I was seeing 250ms plus response times for simple authenticated routes I realized the initial code was full of middleware bloat

To fix it I used Blackbox AI to refactor the logic and find the bottlenecks Most LLMs suggest the easy way like BaseHTTPMiddleware but Blackbox helped me implement more performant patterns like Pure ASGI middleware It is a bit more code but it talks directly to the server and shaved 20ms off every single request

I also moved my Auth logic from global middleware into a FastAPI Dependency This was huge it stopped the app from hitting the database on every single public request or docs page load Lastly I tweaked GZip to only trigger for payloads over 1kb so it does not waste CPU on tiny JSON responses

The result My P95 response times dropped from 280ms to 85ms on my EC2 instance Lesson learned AI is great for the what but you still need to be the how when it comes to speed If your routes feel sluggish look at your middleware stack first it is usually the silent killer

Anyone else notice AI generated code leaning too hard on BaseHTTPMiddleware Curious if there are other hidden overheads I should look out for

TLDR Swapped BaseHTTPMiddleware for pure ASGI using Blackbox AI moved auth to Depends and stopped GZipping tiny responses Cut latency by 70 percent

reddit.com
u/Feeling-Ad972 — 16 days ago
▲ 0 r/codereview

I have been scaffolding a project lately and while the dev speed was great the performance was garbage I was seeing 250ms plus response times for simple authenticated routes I realized the initial code was full of middleware bloat

To fix it I used Blackbox AI to refactor the logic and find the bottlenecks Most LLMs suggest the easy way like BaseHTTPMiddleware but Blackbox helped me implement more performant patterns like Pure ASGI middleware It is a bit more code but it talks directly to the server and shaved 20ms off every single request

I also moved my Auth logic from global middleware into a FastAPI Dependency This was huge it stopped the app from hitting the database on every single public request or docs page load Lastly I tweaked GZip to only trigger for payloads over 1kb so it does not waste CPU on tiny JSON responses

The result My P95 response times dropped from 280ms to 85ms on my EC2 instance Lesson learned AI is great for the what but you still need to be the how when it comes to speed If your routes feel sluggish look at your middleware stack first it is usually the silent killer

Anyone else notice AI generated code leaning too hard on BaseHTTPMiddleware Curious if there are other hidden overheads I should look out for

TLDR Swapped BaseHTTPMiddleware for pure ASGI using Blackbox AI moved auth to Depends and stopped GZipping tiny responses Cut latency by 70 percent

reddit.com
u/Feeling-Ad972 — 16 days ago
▲ 1 r/BlackboxAI_

I have been using LLMs to scaffold a FastAPI project lately. Dev speed was 10/10, but the performance was garbage

I was seeing 250ms+ response times for simple authenticated routes.

Turns out, the AI-generated code was full of middleware bloat. If you're letting an LLM write your stack, here’s what’s probably killing your latency:

  • BaseHTTPMiddleware is a trap: Every AI prompt seems to suggest this. It’s easy to write, but it has a massive performance tax because of how it handles tasks. I swapped my custom logging and header checks for pure ASGI middleware. It’s less "readable" in the code, but it instantly shaved off 20ms per request.

  • Global Middleware vs. Depends: The AI had me running a "VerifyAuth" middleware on every request. This meant the app was hitting the DB even for public health checks and docs. I moved that logic into a FastAPI Dependency (Depends) so it only runs when actually needed.

  • GZip is overkill for small payloads: My generated config was GZipping everything. Compressing a tiny 100-byte JSON object actually adds more latency than it saves in bandwidth. Set a minimum size (like 1kb) before compressing.

The result: My P95 response times dropped from 280ms to 85ms on my EC2 instance.

Lesson learned: AI is great for boilerplate, but it loves "safe," heavy patterns that don't scale. If your routes feel sluggish, look at your middleware stack first.

Anyone else notice AI leaning too hard on BaseHTTPMiddleware?

Curious if I'm the only one who fell for it.

TL;DR: Swapped BaseHTTPMiddleware for pure ASGI, moved auth to Depends, and stopped GZipping tiny responses. Cut latency by 70%

u/Feeling-Ad972 — 16 days ago