V

vllm

🔧🔗https://github.com/vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

🔧🔗https://github.com/vllm-project/llm-compressor Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

🔧🔗https://github.com/vllm-project/vllm-ascend Community maintained hardware plugin for vLLM on Ascend

🔧🔗https://github.com/containers/ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all

🔧🔗https://github.com/modelscope/ms-swift

SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning) Use PEFT or Full-parameter to finetune 400+ LLMs or 100+ MLLMs.

🔧🔗https://github.com/vllm-project/aibrix Cost-efficient and pluggable Infrastructure components for GenAI inference

🔧🔗https://github.com/vllm-project/vllm-spyre

Community maintained hardware plugin for vLLM on Spyre

🔧🔗https://github.com/vllm-project/production-stack

Scale from single vLLM instance to distributed vLLM deployment without changing any application code.

🔧🔗https://github.com/vllm-project/FlashMLA

Neuromancer 🔧🔗https://git.tomfos.tr/tom

Self-hosted, GPU-optimised GenAI platform providing a drop-in OpenAI-compatible API

🔧🔗https://github.com/vllm-project/dashboard

🔧🔗https://github.com/vllm-project/vllm_allocator_adaptor An adaptor to allow Python allocator for PyTorch pluggable allocator

🔧🔗 meta-llama/llama-recipes Scripts for fine-tuning Meta Llama with composable FSDP & PEFT methods to cover single/multi-node GPUs.