#model-serving

ModelTC/LightLLM

Python

88

health

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

beclab/Olares

Go

88

health

Olares: An Open-Source Personal Cloud to Reclaim Your Data

vllm-project/vllm-omni

Python

86

health

A framework for efficient model inference with omni-modality models

vllm-project/vllm-ascend

C++

82

health

Community maintained hardware plugin for vLLM on Ascend