benchmark

https://github.com/THUDM/LongBench [ACL 2024] LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

https://github.com/THUDM/LongCite LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

https://github.com/princeton-nlp/SWE-bench [ICLR 2024] SWE-Bench: Can Language Models Resolve Real-world Github Issues? 🔗 https://www.swebench.com/

https://github.com/VictoriaMetrics/prometheus-benchmark Benchmark for Prometheus-compatible systems

🔧🔗https://github.com/cirruslabs/XcodeBenchmark-ForRunners

XcodeBenchmark measures the compilation time of a large codebase on iMac, MacBook, and Mac Pro

🔧🔗https://github.com/go101/go-benchmarks

Some benchmarks I wrote in writing the Go 101 book.

Repository for AI Model Benchmarking

https://github.com/princeton-nlp/CharXiv CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs

https://github.com/VictoriaMetrics/billy Billy benchmarks for VictoriaMetrics