B

benchmark

🔧🔗https://github.com/tensorzero/llmgym LLM Gym is a unified environment interface for developing and benchmarking LLM applications that learn from feedback. Think gym for LLM agents.

As the space of benchmar

https://github.com/princeton-nlp/SWE-bench [ICLR 2024] SWE-Bench: Can Language Models Resolve Real-world Github Issues? 🔗 https://www.swebench.com/

🔧🔗https://github.com/xlucn/fio-cdm

CrystalDiskMark-style disk speed test with fio.

Python version

https://github.com/princeton-nlp/CharXiv CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs

🔧🔗https://github.com/gin-gonic/FrameworkBenchmarks

Source code for the framework benchmarking project

Repository for AI Model Benchmarking

https://github.com/THUDM/LongBench [ACL 2024] LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

https://github.com/THUDM/LongCite LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA