#inference

open-edge-platform/dlstreamer

C++

91

health

Deep Learning Streamer (DL Streamer) Pipeline Framework is an open-source streaming media analytics framework, based on GStreamer* multimedia framework, for creating complex media analytics pipelines for the Cloud or at the Edge.

openvinotoolkit/openvino

C++

90

health

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

LMCache/LMCache

Python

89

health

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

raullenchai/Rapid-MLX

Python

89

health

The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.

kvcache-ai/Mooncake

C++

88

health

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation

C++

87

health

Deep learned, NVIDIA-accelerated 3D object pose estimation

timtoole02/Camelid

Rust

86

health

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

argmaxinc/argmax-oss-swift

Swift

85

health

On-device Speech AI for Apple Silicon

vllm-project/vllm-ascend

C++

83

health

Community maintained hardware plugin for vLLM on Ascend