diff --git a/README.md b/README.md index 918b5c51349ff01e6d84d4d3ec07be379833fd44..0be7fd184121abd206e1ef718116f3985317251f 100644 --- a/README.md +++ b/README.md @@ -4,15 +4,18 @@ The 'llama-recipes' repository is a companion to the [Meta Llama 3](https://gith <!-- markdown-link-check-enable --> > [!IMPORTANT] -> Meta Llama 3 has a new prompt template and special tokens (based on the tiktoken tokenizer). +> Meta Llama 3.1 has a new prompt template and special tokens. > | Token | Description | > |---|---| -> `<\|begin_of_text\|>` | This is equivalent to the BOS token. | -> `<\|end_of_text\|>` | This is equivalent to the EOS token. For multiturn-conversations it's usually unused. Instead, every message is terminated with `<\|eot_id\|>` instead.| -> `<\|eot_id\|>` | This token signifies the end of the message in a turn i.e. the end of a single message by a system, user or assistant role as shown below.| -> `<\|start_header_id\|>{role}<\|end_header_id\|>` | These tokens enclose the role for a particular message. The possible roles can be: system, user, assistant. | +> `<\|begin_of_text\|>` | Specifies the start of the prompt. | +> `<\|eot_id\|>` | This token signifies the end of a turn i.e. the end of the model's interaction either with the user or tool executor. | +> `<\|eom_id\|>` | End of Message. A message represents a possible stopping point where the model can inform the execution environment that a tool call needs to be made. | +> `<\|python_tag\|>` | A special tag used in the model’s response to signify a tool call. | +> `<\|finetune_right_pad_id\|>` | Used for padding text sequences in a batch to the same length. | +> `<\|start_header_id\|>{role}<\|end_header_id\|>` | These tokens enclose the role for a particular message. The possible roles can be: system, user, assistant and ipython. | +> `<\|end_of_text\|>` | This is equivalent to the EOS token. For multiturn-conversations it's usually unused, this token is expected to be generated only by the base models. | > -> A multiturn-conversation with Meta Llama 3 follows this prompt template: +> A multiturn-conversation with Meta Llama 3.1 that includes tool-calling follows this structure: > ``` > <|begin_of_text|><|start_header_id|>system<|end_header_id|> > @@ -20,13 +23,16 @@ The 'llama-recipes' repository is a companion to the [Meta Llama 3](https://gith > > {{ user_message_1 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|> > -> {{ model_answer_1 }}<|eot_id|><|start_header_id|>user<|end_header_id|> +> <|python_tag|>{{ model_tool_call_1 }}<|eom_id|><|start_header_id|>ipython<|end_header_id|> > -> {{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|> +> {{ tool_response }}<|eot_id|><|start_header_id|>assistant<|end_header_id|> +> +> {{model_response_based_on_tool_response}}<|eot_id|> > ``` > Each message gets trailed by an `<|eot_id|>` token before a new header is started, signaling a role change. > -> More details on the new tokenizer and prompt template can be found [here](https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3#special-tokens-used-with-meta-llama-3). +> More details on the new tokenizer and prompt template can be found [here](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1). + > > [!NOTE] > The llama-recipes repository was recently refactored to promote a better developer experience of using the examples. Some files have been moved to new locations. The `src/` folder has NOT been modified, so the functionality of this repo and package is not impacted.