D

dataset

🔧🔗https://github.com/modelscope/data-juicer Making data higher-quality, juicier, and more digestible for foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

https://github.com/awesome-selfhosted/awesome-selfhosted-data machine-readable data for https://awesome-selfhosted.net

🔧🔗https://github.com/HumanSignal/label-studio

Label Studio is a multi-type data labeling and annotation tool with standardized output format

🔧🔗https://github.com/joanrod/ocr-vqganOCR-VQGAN, a discrete image encoder (tokenizer and detokenizer) for figure images in Paper2Fig100k dataset. Implementation of OCR Perceptual loss for clear text-within-im

🔧🔗https://github.com/MultiTonic/thinking-dataset

creating a thinking dataset

https://github.com/huggingface/dataspeech

https://github.com/coqui-ai/TTS-recipes 🐸TTS recipes for different datasets

https://github.com/lm-sys/llm-decontaminator Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"

https://github.com/Farama-Foundation/Minari A standard format for offline reinforcement learning datasets, with popular reference datasets and related utilities

https://github.com/run-llama/llama-datasets Github repo for storing LlamaDatasets