Developer argues buying OpenAI Pro beats running a small local model
The author says a small local model's code quality resembles a frontier model from 1.5 years ago and needs constant babysitting, so buying Pro for 10 devs to use Astra is the better deal.
I would instead buy the $200/month OpenAI Pro license for all 10 devs ($2k per month, 9 months dev time for same price). Astra is so much better than a tiny open source model that fits to 128GB VRAM. Code quality is on a whole new level. You have to babysit that small LLM all the time. Spend lot of human time fixing issues, debugging, etc. The code quality is similar to 1.5 year old frontier model. You don’t want to be there. Trust me. I have gone through the pain.