Opus 5.5 在 Devin 里逐帧审片并改写视频提示词
把 Seedance 2.5 生成的 12 秒短片交给 Opus 5.5，它自建 ffmpeg 抽帧与音频检查流程，定位提示词矛盾并重写，输出带标注的 PDF 报告。
A self-improvement loop for multimodal generation is here. We gave Opus 5.5, running inside the Devin harness, a 12-second clip generated by Seedance 2.5 along with the prompt that produced it. It essentially debugged the prompt like code. Opus audited the clip frame by frame, found details we had missed, traced several failures back to contradictions or underspecified constraints in the prompt, rewrote it, generated another attempt, and evaluated the result again. Opus did not receive a continuous video stream. It built its own inspection harness in a shell: - Adaptive sampling. ffmpeg at 2 fps across the clip, then 4 fps around critical windows like the camera reveal and dialogue. It assembled contact sheets so frames seconds apart could be compared directly. - Hypothesize, then zoom. It formed a claim and verified it using magnified crops from specific timestamps. This is how it caught a delivery rider wearing one helmet while carrying another, in a blurry region roughly 100 px tall that we had missed. - Rule reconstruction. It explicitly reconstructed the scene's constraints: which direction each signal should face, which traffic phase permits each movement, who should be speaking, and so on. It then evaluated the video against those constraints rather than just asking whether the scene looked realistic. - Audio as a separate channel. It ran faster-whisper with word timestamps, measured loudness in 0.5 s windows, and compared those events against mouth crops to infer who was actually speaking. It then produced a PDF with annotated figures and severity labels, while separating direct observations, model inferences, and vendor claims. Some apparent "video model errors" turned out to be prompt errors. Finding those required reasoning about the internal logic of the scene, not just inspecting pixels. Opus also built its own evaluation harness with minimal instruction. On a fixed 11-criterion scorecard, the generation went from satisfying 2/11 criteria to 7/11 after the prompt revision. More interestingly, the eval made the remaining failure obvious. Explicit spatial constraints helped a lot. Temporal state changes were much harder. Once the scene layout was specified precisely, Seedance could usually render it. Binding a state transition at the right moment to the right object remained unreliable (I believe we need more context engineering and experiments here) We also ran the same task with GPT-6 Astra, and Opus 5.5 surfaced substantially more of the subtle inconsistencies and missing constraints.
这条是引用着另一条写的：x.com/atakoylanai/status/2102977463751938537