← Back to Discover
guqiong96

guqiong96/Lvllm

PythonApache-2.0active
89Health

LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

Stars457
Forks41
Open Issues1
Contributors41
Last Push0d ago

Health Breakdown

Activity
25
Community
25
Maintenance
14
Popularity
25
#cpu#decode#gpu#hybrid#inference#model#moe#numa#parallelism#prefill#vllm
View on GitHub ↗Issues (1) ↗Pull Requests ↗

Should you contribute to guqiong96/Lvllm?

guqiong96/Lvllm has a FoundDev health score of 89/100, which puts it in the active-and-maintained tier. The maintainer team is shipping recently, issues are being closed, and a PR you open this week has a realistic chance of being reviewed.

Last push was 0 days ago — that signals an actively maintained project. New issues are likely to get a maintainer response within days. The project is written primarily in Python, so prior Python experience will shorten ramp-up.

Licensed under Apache-2.0, a standard OSI-approved license — safe to contribute to under normal employer IP policies.

Community

guqiong96
guqiong96/Lvllm
PythonApache 2.0
89

LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

active
45741 contributors1 issues
0d ago

More Python repos

anasty17
anasty17/mirror-leech-telegram-bot
Official Repository: Telegram bot which can download direct links, torrents, nzb, google drive, telegram document, any file/folder from rclone supported clouds, all yt-dlp supported sites and jdownloader supported sites, then upload them to google drive, telegram cloud or to one of rclone supported clouds
4.3k100
Azure-Samples
Azure-Samples/azure-search-openai-demo
A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
7.8k95
commaai
commaai/opendbc
a Python API for your car
3.4k94