#llmops

promptfoo/promptfoo

TypeScript

89

health

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

comet-ml/opik

Python

89

health

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

langwatch/langwatch

TypeScript

85

health

The platform for LLM evaluations and AI agent testing

traceloop/openllmetry

Python

85

health

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

ferro-labs/ai-gateway

Go

83

health

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

vllm-project/vllm-ascend

C++

82

health

Community maintained hardware plugin for vLLM on Ascend