Vllm

Project information

🔧🔗https://github.com/vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm amd cuda inference pytorch transformer Llama gpt rocm model-serving tpu hpu mlops xpu inferentia Large Langua... llm-inference llmops

12,212 Commits
1,263 Branches
102 Tags

README
Apache License 2.0
CONTRIBUTING

Created on

March 17, 2024

Loading

🐾❤️ Strive to be the person your dogs believe you are ❤️🐾