L

Large Language Models

🔧🔗https://github.com/vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

🔧🔗https://github.com/IBM/mcp-context-forge A Model Context Protocol (MCP) Gateway & Registry. Serves as a central management point for tools, resources, and prompts that can be accessed by MCP-compatible LLM applications. Converts REST API endpoints to MCP, composes virtual MCP servers with added security and observability, and converts between protocols (stdio, SSE, Streamable HTTP).

🔧🔗https://github.com/BerriAI/litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq] 🕸️🔗docs.litellm.ai/docs/

🔧🔗https://github.com/openai/codex Lightweight coding agent that runs in your terminal

MLX-lm

🔧🔗https://github.com/ml-explore/mlx-lm Run LLMs with MLX

https://github.com/joaomdmoura/crewAI Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

crewai.com

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

huggingface.co/transformers https://github.com/huggingface/transformers

🔧🔗https://github.com/vllm-project/vllm-ascend Community maintained hardware plugin for vLLM on Ascend

e2b Desktop sandbox

🔧🔗https://github.com/e2b-dev/desktop E2B Desktop Sandbox for LLMs. E2B Sandbox with desktop graphical environment that you can connect to any LLM for secure computer use.

🔧🔗https://github.com/modelscope/ms-swift

SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning) Use PEFT or Full-parameter to finetune 400+ LLMs or 100+ MLLMs.

🔧🔗https://github.com/flashinfer-ai/flashinfer

FlashInfer: Kernel Library for LLM Serving

🕸️🔗flashinfer.ai

🔧🔗https://github.com/Skyvern-AI/skyvern

Automate browser-based workflows with LLMs and Computer Vision 🕸️🔗 www.skyvern.com/

🔧🔗https://github.com/modelscope/evalscope A streamlined and customizable framework for efficient large model evaluation and performance benchmarking

🔧🔗https://github.com/google/adk-python An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.

🔧🔗https://github.com/gpustack/gpustack

Manage GPU clusters for running LLMs

https://github.com/InternLM/xtuner An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

🔧🔗https://github.com/tensorzero/tensorzero TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation

🔧🔗https://github.com/modelscope/data-juicer Making data higher-quality, juicier, and more digestible for foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

🔧🔗https://github.com/All-Hands-AI/OpenHands

🙌 OpenHands: Code Less, Make More

https://github.com/InternLM/lmdeploy LMDeploy is a toolkit for compressing, deploying, and serving LLMs. 🔗 lmdeploy.readthedocs.io/en/latest/