Laya-MLX runs a Snake demo at about 11 decisions per second on Apple silicon
Laya-MLX, an open-source MLX port of Laya for Apple silicon, ran its Snake demo on a Mac Studio M4 Max at around 11 decisions per second with all inference done locally and zero output tokens.
Following Jev? Laya-MLX is worth a look. It's an open-source MLX port of Laya for Apple silicon. You give it text and possible actions; it returns probabilities for each, with zero output tokens. I ran its Snake demo on my Mac Studio M4 Max and recorded the clip below. Around 11 decisions/sec, all inference on my Mac. The demo calls the model for each move, with a planner and safety layer assisting. Credit to @mizorewww for the MLX port. Repo in reply 👇