L

long-context

https://github.com/InternLM/InternLM Official release of InternLM2.5 base and chat models. 1M context support https://internlm.intern-ai.org.cn/

https://github.com/forhaoliu/ringattention

Transformers with Arbitrarily Large Context

https://github.com/THUDM/LongBench [ACL 2024] LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

https://github.com/THUDM/LongCite LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

https://github.com/bytedance/ShadowKV ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference

https://github.com/THUDM/LongWriter LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs