llm-inference

https://github.com/InternLM/lmdeploy LMDeploy is a toolkit for compressing, deploying, and serving LLMs. 🔗 lmdeploy.readthedocs.io/en/latest/

https://github.com/Lightning-AI/litgpt 20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale. 🔗 https://lightning.ai/

https://github.com/mistralai/mistral-inference Official inference library for Mistral models

mistral.ai/

https://github.com/huggingface/parler-tts Inference and training library for high-quality TTS models.

The Arcee client for executing domain-adpated language model routines

https://github.com/coqui-ai/inference-engine Coqui Inference Engine