distributed-training

OpenLIT is an open-source LLM Observability tool built on OpenTelemetry. 📈🔥 Monitor GPU performance, LLM traces with input and output metadata, and metrics like cost, tokens, and user interactions along with complete APM for LLM Apps. 🖥️

🔧🔗https://github.com/NVIDIA/apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

https://github.com/coqui-ai/snakepit 🐍 Coqui's machine learning job scheduler