cohere

🔧🔗https://github.com/cohere-ai/magikarp

This repository contains the code and extended results for the paper Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models https://arxiv.org/abs/2405.05417