Explore projects
-
-
A high-throughput and memory-efficient inference and serving engine for LLMs
Updated -
https://github.com/InternLM/lmdeploy LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
🔗 lmdeploy.readthedocs.io/en/latest/Updated -
Port of OpenAI's Whisper model in C/C++
Updated -
Updated
-
Efficient framework-agnostic data loading
Updated -
MLX: An array framework for Apple silicon
Updated -
-
huggingface.co/transformers https://github.com/huggingface/transformers🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.Updated -
The open-source language model computer
Updated -
cortex.llamacpp is a high-efficiency C++ inference engine for edge computing. It is a dynamic library that can be loaded by any server at runtime.
Updated -
Updated
-
TVM for Tenstorrent ASICs
Updated -
https://github.com/AnswerDotAI/gpu.cpp A lightweight library for portable low-level GPU computation using WebGPU.
Updated