Explore projects

https://github.com/coturn/coturn

https://github.com/InternLM/lmdeploy LMDeploy is a toolkit for compressing, deploying, and serving LLMs. 🔗 lmdeploy.readthedocs.io/en/latest/

🔧🔗https://github.com/pytorch/pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

🔧🔗https://github.com/modelscope/dash-infer

DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including x86 and ARMv9.

https://github.com/janhq/cortex.cpp

https://github.com/janhq/nitro.git now: https://github.com/janhq/cortex.git Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM, ONNX). Powers 👋 Jan

🔧🔗https://github.com/bytedance/Elkeid Elkeid is an open source solution that can meet the security requirements of various workloads

Port of OpenAI's Whisper model in C/C++

Tensor library for machine learning

🔧🔗https://github.com/AppFlowy-IO/AppFlowy Bring projects, wikis, and teams together with AI. AppFlowy is an AI collaborative workspace where you achieve more without losing control of your data. The best

🔧🔗https://github.com/microsoft/quicreach

A 'ping' equivalent tool for QUIC.

https://github.com/mlc-ai/mlc-llm Universal LLM Deployment Engine with ML Compilation

A high-throughput and memory-efficient inference and serving engine for LLMs

🔧🔗https://github.com/ghostty-org/ghostty

👻 Ghostty is a fast, feature-rich, and cross-platform terminal emulator that uses platform-native UI and GPU acceleration.

User-Mode Driver for Tenstorrent hardware