M

metrics

🔧🔗https://github.com/google/ml-metrics Ml-metrics provides performant and distributed friendly ML metrics implementations.

https://github.com/Lightning-AI/torchmetrics Torchmetrics - Machine learning metrics for distributed, scalable PyTorch applications. 🔗 https://lightning.ai/docs/torchmetrics/

Cadvisor

🔧🔗https://github.com/google/cadvisor

OpenLIT is an open-source LLM Observability tool built on OpenTelemetry. 📈🔥 Monitor GPU performance, LLM traces with input and output metadata, and metrics like cost, tokens, and user interactions along with complete APM for LLM Apps. 🖥️

https://github.com/roboflow/supervision We write your reusable computer vision tools. 💜

Pyannote metrics

🔧🔗https://github.com/pyannote/pyannote-metrics

A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems

🔧🔗https://github.com/tubearchivist/tubearchivist-metrics

Optional service for the TubeArchivist project to export metrics for users who run prometheus/grafana

🔧🔗https://github.com/om-ai-lab/VL-CheckList Evaluating Vision & Language Pretraining Models with Objects, Attributes and Relations. [EMNLP 2022]