Software2 hOctober 13, 2026, 10:33

Superfluid: open-source local LLM server for multi-agent inference

Superfluid is a new Apache-2.0 server for running LLMs locally with interactive requests preempting agent workloads. It supports the OpenAI, Anthropic and Ollama APIs plus llama.cpp, MLX and BaseRT runtimes; on an Apple M5 Pro a chat answer behind 8 agents arrives in 1.5s versus 291s for llama-server.

DEV Community

Open source

Apache-2.0 license, one-command install on macOS (Apple silicon) and Linux
Supports OpenAI, Anthropic and Ollama APIs with llama.cpp, MLX and BaseRT runtimes
Chat TTFT behind 8 agents: 1.5s vs 291s for llama-server on M5 Pro
Sessions resume in 1.8s after a worker crash; shared prompt cache hits reach 97%

#Superfluid#Llama.cpp#MLX#Ollama

Run local LLM in ChatGPT Desktop app with opencodex alone

Hardware

Forlinx launches 20-TOPS M.2 AI accelerator for local LLM inference

Software

Superfluid: open-source local LLM server for multi-agent inference

Run local LLM in ChatGPT Desktop app with opencodex alone

Forlinx launches 20-TOPS M.2 AI accelerator for local LLM inference

Copilot CLI adds local Ollama model picker as MXC sandbox hits GA

Z-Lab's DFlash-2 boosts LLM throughput with diffusion-based drafting