Open source repositories tagged with #inference, ranked by health score.
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
SGLang is a high-performance serving framework for large language models and multimodal models.
A framework for efficient model inference with omni-modality models
A fast, lightweight, and extensible RWKV chat UI powered by Flutter. Offline-ready, multi-backend support, ideal for local RWKV inference.
Community maintained hardware plugin for vLLM on Ascend
On-device Speech AI for Apple Silicon
