L

llmops

🔧🔗https://github.com/BerriAI/litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq] 🕸️🔗docs.litellm.ai/docs/

🔧🔗https://github.com/langfuse/langfuse 🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, L

🔧🔗https://github.com/tensorzero/tensorzero TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, persist, and execute on your own infrastructure. burr.dagworks.io

Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.

A Blazing Fast AI Gateway. Route to 100+ LLMs with 1 fast & friendly API. https://github.com/Portkey-AI/gateway

🔧🔗https://github.com/langfuse/langfuse-js

🪢 Langfuse JS/TS SDKs - Instrument your LLM app and get detailed tracing/observability. Works with any LLM or framework

OpenLIT is an open-source LLM Observability tool built on OpenTelemetry. 📈🔥 Monitor GPU performance, LLM traces with input and output metadata, and metrics like cost, tokens, and user interactions along with complete APM for LLM Apps. 🖥️

https://github.com/distantmagic/paddler Stateful load balancer custom-tailored for llama.cpp

https://github.com/ComposioHQ/composio Composio equip's your AI agents & LLMs with 100+ high-quality integrations via function calling

🔧🔗https://github.com/langfuse/mcp-server-langfuse Model Context Protocol (MCP) Server for Langfuse Prompt Management. This server allows you to access and manage your Langfuse prompts through MCP

https://github.com/distantmagic/llmops-handbook Practical and advanced guide to LLMOps. It provides a solid understanding of large language models’ general concepts, deployment techniques, and software engineering practices.