← Back to Discover
kekzl

kekzl/imp

C++MITactive
88Health

Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one docker command.

Stars124
Forks5
Open Issues0
Contributors5
Last Push0d ago

Health Breakdown

Activity
25
Community
25
Maintenance
13
Popularity
24
#ai-agents#blackwell#cuda#inference-engine#llm-inference#local-llm#mixture-of-experts#nvfp4#openai-api#openai-compatible#quantization#qwen#rtx-50-series#rtx-5090#rtx-pro-6000#safetensors#self-hosted#tool-calling#vllm
View on GitHub ↗Issues (0) ↗Pull Requests ↗

Should you contribute to kekzl/imp?

kekzl/imp has a FoundDev health score of 88/100, which puts it in the active-and-maintained tier. The maintainer team is shipping recently, issues are being closed, and a PR you open this week has a realistic chance of being reviewed.

Last push was 0 days ago — that signals an actively maintained project. New issues are likely to get a maintainer response within days. The project is written primarily in C++, so prior C++ experience will shorten ramp-up.

Licensed under MIT, a standard OSI-approved license — safe to contribute to under normal employer IP policies.

Community

kekzl
kekzl/imp
C++MIT
88

Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one docker command.

active
★ 1245 contributors
0d ago

More C++ repos

PX4
PX4/PX4-Autopilot
PX4 Autopilot Software
★ 12.9k99
openbmc
openbmc/bmcweb
A do everything Redfish, KVM, GUI, and DBus webserver for OpenBMC
★ 24097
plumed
plumed/plumed2
Development version of plumed 2
★ 52393