Llama

🔧🔗https://github.com/tensorzero/tensorzero TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation

🔧🔗https://github.com/sgl-project/sglang

SGLang is a fast serving framework for large language models and vision language models.

🔧🔗https://github.com/vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

🔧🔗https://github.com/modelscope/ms-swift

SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning) Use PEFT or Full-parameter to finetune 400+ LLMs or 100+ MLLMs.

🔧🔗https://github.com/hiyouga/LLaMA-Factory Unified Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

🔧🔗https://github.com/meta-llama/llama-stack

Composable building blocks to build Llama Apps

https://github.com/InternLM/lmdeploy LMDeploy is a toolkit for compressing, deploying, and serving LLMs. 🔗 lmdeploy.readthedocs.io/en/latest/

🔧🔗https://github.com/modelscope/data-juicer Making data higher-quality, juicier, and more digestible for foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

https://github.com/janhq/nitro.git now: https://github.com/janhq/cortex.git Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM, ONNX). Powers 👋 Jan

🔧🔗https://github.com/meta-llama/PurpleLlama

Set of tools to assess and improve LLM security.

A multi-user ChatGPT for any LLMs and vector database. Unlimited documents, messages, and storage in one privacy-focused app. Now available as a desktop application with a built-in LLM!

https://github.com/fishaudio/fish-speech Brand new TTS solution

https://github.com/Mozilla-Ocho/llamafile Distribute and run LLMs with a single file. llamafile.ai

🔧🔗https://github.com/meta-llama/llama-stack-client-node

NodeJS library for Llama Stack

🔧🔗https://github.com/meta-llama/llama-stack-client-python

Python SDK for Llama Stack

https://github.com/arcee-ai/mergekit Tools for merging pretrained large language models.

🔧🔗https://github.com/meta-llama/llama-models

Utilities intended for use with Llama models.

🔧🔗https://github.com/meta-llama/llama-stack-client-kotlin

Kotlin SDK for Llama Stack

🔧🔗https://github.com/meta-llama/llama-stack-apps

Agentic components of the Llama Stack APIs

🔧🔗https://github.com/andrewkchan/yalm

Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O