Opus 5.5 animates how LLMs "see" images and video
The author had Opus 5.5 read Hugging Face's glm-5.3-flash code and turn it into an animation explaining how images and video become model inputs, which also ended up scoring music with numpy.
so what do LLMs actually "see" when we send them images & video? a great way to develop intuition around vision in models is to rabbit hole on how our inputs get mapped to model inputs i pointed Opus-5.5 at @huggingface transformers repo for glm-5.3-flash to animate the process. a lot of "harness engineering" is needed to go from image -> vectors for models to natively process side note: i did not ask Claude to make music using....numpy, but it kinda goes hard some fun notes w/ glm-flash-5.3: - Video frames get plain-text timestamps like "4.0 seconds" dropped in between them. - Every image gets copied into a 2-frame video before the model sees it. - 4k & 8k photos get preprocessed into the same token count it's pretty amazing that anyone can visually rabbit hole down any interesting concept today with agents, infinitely patient, infinitely willing teachers to go into any and every last detail