OpenAI's Boris Power: 80-90% of research targets GPT-7 and 8
OpenAI's Head of Applied Research says 80-90% of the company's research is aimed at GPT-7/8 and beyond, and that a large model like Astra will eventually be distilled into a small GPT-Luna.
OpenAI's Boris Power reveals 80 to 90% of OpenAI research aims at GPT-7, 8, and beyond, then distills a giant Astra teacher into a small Luna model "So a lot of those investments, the way that they're seen within the company, are extremely short-sighted. We do them because they help us iterate and learn faster today, not because that's the right long-term strategy." "So still 80, 90 of the research at OpenAI is focused at GPT-7, GPT-8, and beyond, because we believe that's where fundamentally most of the value comes from." "And then what happens is once you develop a GPT-6 Astra, you can distill that model into a very small GPT-Luna model." "So the best way to train a small specialized model is actually to train a big, super-capable model, and then find a way of just using that model as a teacher for a smaller student model." _______ For more comments from the Head of Applied Research at OpenAI, Boris Power: https://firesidealpha.substack.com/p/fireside-am-brief-cbrs-openai-nbis
This post quotes another post: x.com/firesidealpha/status/2103579227065045482