Large Language Models

🔧🔗https://github.com/BerriAI/litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq] 🕸️🔗docs.litellm.ai/docs/

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, persist, and execute on your own infrastructure. burr.dagworks.io

Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.

🔧🔗https://github.com/Skyvern-AI/skyvern

Automate browser-based workflows with LLMs and Computer Vision 🕸️🔗 www.skyvern.com/

https://github.com/run-llama/LlamaIndexTS LlamaIndex is a data framework for your LLM applications

OpenLIT is an open-source LLM Observability tool built on OpenTelemetry. 📈🔥 Monitor GPU performance, LLM traces with input and output metadata, and metrics like cost, tokens, and user interactions along with complete APM for LLM Apps. 🖥️

https://github.com/distantmagic/paddler Stateful load balancer custom-tailored for llama.cpp

https://github.com/InternLM/MindSearch 🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT) 🔗 https://mindsearch.netlify.app/

https://github.com/mlc-ai/web-llm-chat Chat with AI large language models running natively in your browser. Enjoy private, server-free, seamless AI conversations.

https://github.com/InternLM/HuixiangDou HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance

https://github.com/InternLM/OpenAOE LLM Group Chat Framework: chat with multiple LLMs at the same time. 大模型群聊框架：同时与多个大语言模型聊天。