L

Large Language Models

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, persist, and execute on your own infrastructure. burr.dagworks.io

https://github.com/InternLM/xtuner An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

🔧🔗https://github.com/vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

🔧🔗https://github.com/google/adk-python An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.

🔧🔗https://github.com/vllm-project/vllm-ascend Community maintained hardware plugin for vLLM on Ascend

https://github.com/InternLM/lmdeploy LMDeploy is a toolkit for compressing, deploying, and serving LLMs. 🔗 lmdeploy.readthedocs.io/en/latest/

🔧🔗https://github.com/getzep/zep

Zep | The Memory Foundation For Your AI Stack

🔧🔗https://github.com/flashinfer-ai/flashinfer

FlashInfer: Kernel Library for LLM Serving

🕸️🔗flashinfer.ai

🔧🔗https://github.com/modelscope/ms-swift

SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning) Use PEFT or Full-parameter to finetune 400+ LLMs or 100+ MLLMs.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

huggingface.co/transformers https://github.com/huggingface/transformers

🔧🔗https://github.com/langfuse/langfuse 🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, L

🔧🔗https://github.com/llmware-ai/llmware

Unified framework for building enterprise RAG pipelines with small, specialized models

🕸️🔗https://llmware-ai.github.io/llmware/

🔧🔗https://github.com/getzep/graphiti

Build and query dynamic, temporally-aware Knowledge Graphs

🔧🔗https://github.com/gpustack/gpustack

Manage GPU clusters for running LLMs

🔧🔗https://github.com/modelscope/modelscope-agent

ModelScope-Agent: An agent framework connecting models in ModelScope with the world

🔧🔗https://github.com/containers/ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all

https://github.com/mlc-ai/web-llm High-performance In-browser LLM Inference Engine

🔧🔗https://github.com/EleutherAI/lm-evaluation-harness A framework for few-shot evaluation of language models.

🔧🔗https://github.com/google/ml-metrics Ml-metrics provides performant and distributed friendly ML metrics implementations.

🔧🔗https://github.com/modelscope/data-juicer Making data higher-quality, juicier, and more digestible for foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！