Skip to content
Snippets Groups Projects
Commit e804e2ba authored by Matthias Reso's avatar Matthias Reso
Browse files

Address more spell checker issues

parent 18d4c3c5
No related branches found
No related tags found
No related merge requests found
...@@ -4,7 +4,7 @@ To run fine-tuning on a single GPU, we will make use of two packages ...@@ -4,7 +4,7 @@ To run fine-tuning on a single GPU, we will make use of two packages
1- [PEFT](https://huggingface.co/blog/peft) methods and in specific using HuggingFace [PEFT](https://github.com/huggingface/peft)library. 1- [PEFT](https://huggingface.co/blog/peft) methods and in specific using HuggingFace [PEFT](https://github.com/huggingface/peft)library.
2- [bitandbytes](https://github.com/TimDettmers/bitsandbytes) int8 quantization. 2- [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) int8 quantization.
Given combination of PEFT and Int8 quantization, we would be able to fine_tune a Llama 2 7B model on one consumer grade GPU such as A10. Given combination of PEFT and Int8 quantization, we would be able to fine_tune a Llama 2 7B model on one consumer grade GPU such as A10.
......
# Serving a fine tuned Llama model with HuggingFace text-generation-inference server # Serving a fine tuned Llama model with HuggingFace text-generation-inference server
This document shows how to serve a fine tuned LLaMA mode with HuggingFace's text-generation-inference server. This option is currently only available for models that were trained using the LoRA method or without using the `--use_peft` argument. This document shows how to serve a fine tuned Llama mode with HuggingFace's text-generation-inference server. This option is currently only available for models that were trained using the LoRA method or without using the `--use_peft` argument.
## Step 0: Merging the weights (Only required if LoRA method was used) ## Step 0: Merging the weights (Only required if LoRA method was used)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment