← Back to Discover
defilantech

defilantech/LLMKube

GoApache-2.0active
78Health

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Stars246
Forks41
Open Issues113
Contributors41
Last Push0d ago

Health Breakdown

Activity
25
Community
25
Maintenance
3
Popularity
25
#ai#apple-silicon#autoscaling#edge-computing#gguf#gpu#homelab#inference#kubernetes#kubernetes-operator#llama-cpp#llm#local-llm#metal#mlx#multi-gpu#nvidia#self-hosted#tgi#vllm
View on GitHub ↗Issues (113) ↗Pull Requests ↗Wiki ↗

Should you contribute to defilantech/LLMKube?

defilantech/LLMKube has a FoundDev health score of 78/100, which puts it in the active-and-maintained tier. The maintainer team is shipping recently, issues are being closed, and a PR you open this week has a realistic chance of being reviewed.

Last push was 0 days ago — that signals an actively maintained project. New issues are likely to get a maintainer response within days. The project is written primarily in Go, so prior Go experience will shorten ramp-up.

Licensed under Apache-2.0, a standard OSI-approved license — safe to contribute to under normal employer IP policies.

Community

defilantech78

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

active
★ 24641 contributors113 issues
0d ago

More Go repos

cosmos
cosmos/cosmos-sdk
Framework for building performant, customizable blockchains with native interoperability
★ 7.1k94
ethereum-optimism
ethereum-optimism/optimism
Optimism is Ethereum, scaled.
★ 6.5k92
kubernetes
kubernetes/kube-state-metrics
Add-on agent to generate and expose cluster-level metrics.
★ 6.2k92