Authors respond on DeepSeek V4.1 Flash CoT controllability: a capability limit, not refusal
The authors say the model tries to follow CoT constraints but fails to execute, so results are a lower bound on controllability, and they plan fine-tuning stress tests on the open weights.
Thanks Daniel, these are great questions, and thanks @tomekkorbak and @usmananwar391 for the discussion! 1. We agree with Tomek's assessment here. This looks more like a capability limitation than refusal. In our examples, the model attempts to follow the CoT constraints, but fails to execute. 2. Usman raises a good point regarding elicitation. Our results should be read as a lower bound on controllability, and we plan to test the UK AISI prompt as well. 3. Fine-tuning would be a useful stress test. Since V4.1 Flash is open-weight, this is feasible and something we plan to explore. 4. We can’t separate trained vs. emergent effects from our setup, since we only tested final DeepSeek models. But Tomek’s paper above finds controllability drops with more RL training, which suggests it may be a side effect of outcome-based RL. Still, it's worth tracking as models scale. We’ll keep digging and share what we find!