https://claude.ai/share/3fe768db-aab3-4d50-a02f-65de609f712b - unfortunately files are not available in shared chats
So I found out that Claude.ai (the web app) has an Ubuntu Linux VM that it uses to run bash scripts, so I asked it to install blender and render a blend file that I uploaded.
These are the results. (Last image is the file that I uploaded, open on my computer)
First of all, I did not expect it to be able to read the whole file in its 32k context window. And okay, I may have lied to Gemini about the total number of lines in the log.
But less than 250? I know for a fact that Gemini could read far more lines. I have uploaded files with over 1000 lines of code, and it was able to read all of it.
What is stranger is that, unless I specify the number of lines in the log, Gemini does not seem to know at all that the file was truncated.
Pasting the 250 lines it claims to see into a few online tokenizers, I get back a count of just under 10k tokens consistently from all of them. That is far under the 32k context window, and I do not believe that the system prompt takes 12k tokens...as such I'm very curious what is happening.
Claude (on the website at least) has the ability to mount the file in a sort of internal shell session and run bash commands (grep) to search through it. This seems like it would be a pretty useful feature for Gemini to have.