Large Language Models

https://github.com/janhq/cortex Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM, ONNX). Powers 👋 Jan

🔧🔗https://github.com/llmware-ai/llmware

Unified framework for building enterprise RAG pipelines with small, specialized models

[🕸🔗https://llmware-ai.github.io/llmware/](https://llmware-ai.github.io/llmwa

https://github.com/mistralai/mistral-inference Official inference library for Mistral models

mistral.ai/

🔧🔗https://github.com/poloclub/mememo

A JavaScript library that brings vector search and RAG to your browser!

🔧🔗https://github.com/poloclub/llm-landscape

NeurIPS'24 - LLM Safety Landscape

https://github.com/InternLM/xtuner An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

Domain Adapted Language Modeling Toolkit - E2E RAG

https://github.com/topoteretes/PromethAI-Backend Open-source framework that gives you AI Agents that help you navigate decision-making, get personalized goals and execute them

www.prometh.ai

🔧🔗https://github.com/getzep/zepcli

zepcli - a command line tool for managing the Zep service.

https://github.com/mlc-ai/web-llm-chat Chat with AI large language models running natively in your browser. Enjoy private, server-free, seamless AI conversations.

https://github.com/janhq/cortex.tensorrt-llm Cortex.Tensorrt-LLM is a C++ inference library that can be loaded by any server at runtime. It submodules NVIDIA’s TensorRT-LLM for GPU accelerated inference on NVIDIA's GPUs.

https://github.com/THUDM/LongWriter LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

https://github.com/InternLM/HuixiangDou HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance

https://github.com/InternLM/InternLM-Math State-of-the-art bilingual open-sourced Math reasoning LLMs.

🔧🔗https://github.com/FoundationVision/LlamaGen

Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation 🔗arxiv.org/abs/2406.06525

🔧🔗https://github.com/FoundationVision/Groma

[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

🕸️🔗groma-mllm.github.io/

🔧🔗https://github.com/AlexCheema/veritas

Veritas is a library for verifying inference of large ML models on Mina.

https://github.com/THUDM/LongCite LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

🔧🔗https://github.com/AbanteAI/spice

Accellerant

🔧🔗https://github.com/IST-DASLab/marlin FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.