Magikarp

Project information

🔧🔗https://github.com/cohere-ai/magikarp

This repository contains the code and extended results for the paper Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models https://arxiv.org/abs/2405.05417

cohere tokenization Large Langua...

28 Commits
25 Branches
0 Tags

README
Apache License 2.0

Created on

May 10, 2024

Loading

🐾❤️ Strive to be the person your dogs believe you are ❤️🐾