T

transformer

🔧🔗https://github.com/vllm-project/vllm-ascend Community maintained hardware plugin for vLLM on Ascend

🔧🔗https://github.com/vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

🔧🔗https://github.com/sgl-project/sglang

SGLang is a fast serving framework for large language models and vision language models.

🔧🔗https://github.com/gpustack/llama-box

LM inference server implementation based on llama.cpp.

https://github.com/THUDM/SwissArmyTransformer SwissArmyTransformer is a flexible and powerful library to develop your own Transformer variants.