Cortex.Tensorrt Llm

Project information

https://github.com/janhq/cortex.tensorrt-llm Cortex.Tensorrt-LLM is a C++ inference library that can be loaded by any server at runtime. It submodules NVIDIA’s TensorRT-LLM for GPU accelerated inference on NVIDIA's GPUs.

nvidia jan tensorrt Large Langua... tensorrt-llm

75 Commits
49 Branches
19 Tags

README
Apache License 2.0

Created on

July 18, 2024

Loading

🐾❤️ Strive to be the person your dogs believe you are ❤️🐾