diff --git a/tools/benchmarks/README.md b/tools/benchmarks/README.md new file mode 100644 index 0000000000000000000000000000000000000000..b94739ebc7774f22f9b763d9fa6cfb23ae52c1f5 --- /dev/null +++ b/tools/benchmarks/README.md @@ -0,0 +1,4 @@ +# Benchmarks + +* inference - a folder contains benchmark scripts that apply a throughput analysis for Llama models inference on various backends including on-prem, cloud and on-device. +* llm_eval_harness - a folder contains a tool to evaluate fine-tuned Llama models including quantized models focusing on quality.