Native macOS inference server built on MLX. Paged SSD KV caching, continuous batching, and drop-in API for Claude Code, OpenClaw, and Cursor.
An M1 MacBook Pro running Omarchy Linux now sends real Qwen3.8 weights through the Apple Neural Engine. The full 24-layer model path works with persistent DeltaNet and attention state. A macOS control shows where the Linux runtime needs to go next.
Aithy is a private local AI runtime for state-of-the-art agent research, local inference, sandboxed tools, durable memory, and LAN Mesh resource sharing.
Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer.
Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
An occasional email when notable AI dev tools and models land in the directory. No spam, unsubscribe anytime.