L

Large Language Models

🔧🔗https://github.com/langfuse/langfuse 🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, L

🔧🔗https://github.com/modelscope/evalscope A streamlined and customizable framework for efficient large model evaluation and performance benchmarking

🔧🔗https://github.com/getzep/graphiti

Build and query dynamic, temporally-aware Knowledge Graphs

Catwalk

🔧🔗https://github.com/charmbracelet/catwalk

🐈 A collection of LLM inference providers and models

https://github.com/joaomdmoura/crewAI Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

crewai.com

🔧🔗https://github.com/gpustack/gpustack

Manage GPU clusters for running LLMs

🔧🔗https://github.com/google/ml-metrics Ml-metrics provides performant and distributed friendly ML metrics implementations.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

huggingface.co/transformers https://github.com/huggingface/transformers

https://github.com/Lightning-AI/litgpt 20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale. 🔗 https://lightning.ai/

🔧🔗https://github.com/modelscope/data-juicer Making data higher-quality, juicier, and more digestible for foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

SGPT

🔧🔗https://github.com/tbckr/sgpt

SGPT is a command-line tool that provides a convenient way to interact with OpenAI models, enabling users to run queries, generate shell commands and produce code directly from the terminal.

https://github.com/InternLM/xtuner An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

A Blazing Fast AI Gateway. Route to 100+ LLMs with 1 fast & friendly API. https://github.com/Portkey-AI/gateway

Easy Edit

🔧🔗https://github.com/zjunlp/EasyEdit [ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

🔧🔗https://github.com/tensorzero/llmgym LLM Gym is a unified environment interface for developing and benchmarking LLM applications that learn from feedback. Think gym for LLM agents.

As the space of benchmar

🔧🔗https://github.com/redis/mcp-redis/ The official Redis MCP Server is a natural language interface designed for agentic applications to manage and search data in Redis efficiently

🔧🔗https://github.com/QwenLM/Qwen The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

🔧🔗https://github.com/modelscope/modelscope-agent

ModelScope-Agent: An agent framework connecting models in ModelScope with the world

LLMs from scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step 🔧🔗https://github.com/rasbt/LLMs-from-scratch

evals

🔧🔗https://github.com/maragudk/evals a CLI to work with LLM Evals