Q

quantization

🔧🔗https://github.com/vllm-project/llm-compressor Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

🔧🔗https://github.com/microsoft/BitBLAS

BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.