u/Choice_Cabinet9091

Each time I want to start a project, I can mentally picture how its architecture would be but when it's time to start building, or should I say, writing the code, it's all chaos in my head.

I thought it'd be fun to build flash attention 2 from scratch, I read the paper and a few blogs. Now, I figured out what happens on the device side since I'd be executing with CUDA. When it was time to build, I simply couldn't remember a thing. And now I feel guilty each time I think of using LLMs to help me with a walkthrough.