diff --git a/README.md b/README.md index b0d14cd7345b49dd3d7dcd824164da8f48862036..148f0c56097ab03d64f69d3f890b61a85dd3978e 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,5 @@ # Llama Cookbook: The Official Guide to building with Llama Models -> Note: We recently did a refactor of the repo, [archive-main](https://github.com/meta-llama/llama-recipes/tree/archive-main) is a snapshot branch from before the refactor - Welcome to the official repository for helping you get started with [inference](./getting-started/inference/), [fine-tuning](./getting-started/finetuning) and [end-to-end use-cases](./end-to-end-use-cases) of building with the Llama Model family. The examples cover the most popular community approaches, popular use-cases and the latest Llama 3.2 Vision and Llama 3.2 Text, in this repository. @@ -10,26 +8,36 @@ The examples cover the most popular community approaches, popular use-cases and > Popular getting started links: > * [Build with Llama Notebook](./getting-started/build_with_Llama_3_2.ipynb) > * [Multimodal Inference with Llama 3.2 Vision](./getting-started/inference/local_inference/README.md#multimodal-inference) -> * [Inference on Llama Guard 1B + Multimodal inference on Llama Guard 11B-Vision](./end-to-end-use-cases/responsible_ai/llama_guard/ +> * [Inference on Llama Guard 1B + Multimodal inference on Llama Guard 11B-Vision](./end-to-end-use-cases/responsible_ai/llama_guard/) > [!TIP] > Popular end to end recipes: -> * [Finetune Llama 3.2 Vision](./getting-started/finetuning/finetune_vision_model.md)llama_guard_text_and_vision_inference.ipynb) +> * [Agentic Examples](./end-to-end-use-cases/agents/) +> * [NotebookLlama](./end-to-end-use-cases/NotebookLlama/) +> * [Browser Use with Llama](./end-to-end-use-cases/browser_use/) -> [!NOTE] -> Llama 3.2 follows the same prompt template as Llama 3.1, with a new special token `<|image|>` representing the input image for the multimodal models. -> -> More details on the prompt templates for image reasoning, tool-calling and code interpreter can be found [on the documentation website](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_2). +> Note: We recently did a refactor of the repo, [archive-main](https://github.com/meta-llama/llama-recipes/tree/archive-main) is a snapshot branch from before the refactor ## Repository Structure: - [3P Integrations](./3p-integrations): Getting Started Recipes and End to End Use-Cases from various Llama providers - [End to End Use Cases](./end-to-end-use-cases): As the name suggests, spanning various domains and applications - [Getting Started](./getting-started/): Reference for inferencing, fine-tuning and RAG examples +- [src](./src/): Contains the src for the original llama-recipes library along with some FAQs for fine-tuning. ## FAQ: +- Q: Prompt Template changes for Multi-Modality? + +A: Llama 3.2 follows the same prompt template as Llama 3.1, with a new special token `<|image|>` representing the input image for the multimodal models. + +More details on the prompt templates for image reasoning, tool-calling and code interpreter can be found [on the documentation website](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_2). + +- Q: I have some questions for Fine-Tuning, is there a section to address these? + +A: Checkout the Fine-Tuning FAQ [here](./src/docs/) + - Q: Some links are broken/folders are missing: A: We recently did a refactor of the repo, [archive-main](https://github.com/meta-llama/llama-recipes/tree/archive-main) is a snapshot branch from before the refactor diff --git a/UPDATES.md b/UPDATES.md index 0281eb3098bae6e63f5eabe7508c591f39c9d09c..74385fe63189366a38a2f43e0f901744a5cab507 100644 --- a/UPDATES.md +++ b/UPDATES.md @@ -20,4 +20,5 @@ Removed folders: - /flagged (Empty folder) - /recipes/quickstart/Running_Llama3_Anywhere (Redundant code) - /recipes/quickstart/inference/codellama (deprecated model) +- /recipes/quickstart/getting-to-know-llama-3.ipynb <!-- markdown-link-check-enable --> diff --git a/end-to-end-use-cases/README.md b/end-to-end-use-cases/README.md index 65308801059271dd20581c3bdb7256264d63a4cd..68284444e4f063fc20b84db71a6bb9be92d590ba 100644 --- a/end-to-end-use-cases/README.md +++ b/end-to-end-use-cases/README.md @@ -1,34 +1,57 @@ -## [Agentic Tutorial](./agents/): 101 and 201 tutorials on performing Tool Calling and building an Agentic Workflow using Llama Models +# End to End Use Applications using various Llama Models + +## [Agentic Tutorial](./agents/): + +### 101 and 201 tutorials on performing Tool Calling and building an Agentic Workflow using Llama Models 101 notebooks show how to apply Llama models and enable tool calling functionality, 201 notebook walks you through an end to end workflow of building an agent that can search two papers, fetch their details and find their differences. -## [Benchmarks](./benchmarks/): a folder contains benchmark scripts +## [Benchmarks](./benchmarks/): + +### A folder contains benchmark scripts The scripts apply a throughput analysis and introduction to `lm-evaluation-harness`, a tool to evaluate Llama models including quantized models focusing on quality -## [Browser Usage](./browser_use/): Demo of how to apply Llama models and use them for browsing the internet and completing tasks +## [Browser Usage](./browser_use/): -## [Automatic Triaging of Github Repositories](./github_triage/walkthrough.ipynb): Use Llama to automatically triage issues in an OSS repository and generate insights to improve community experience +### Demo of how to apply Llama models and use them for browsing the internet and completing tasks + +## [Automatic Triaging of Github Repositories](./github_triage/walkthrough.ipynb): + +### Use Llama to automatically triage issues in an OSS repository and generate insights to improve community experience This tool utilizes an off-the-shelf Llama model to analyze, generate insights, and create a report for better understanding of the state of a repository. It serves as a reference implementation for using Llama to develop custom reporting and data analytics applications. -## [VideoSummary](video_summary.ipynb): Ask Llama 3 to Summarize a Long YouTube Video (using Replicate or [OctoAI](../3p-integrations/octoai/video_summary.ipynb)) +## [VideoSummary](video_summary.ipynb): + +### Ask Llama 3 to Summarize a Long YouTube Video (using Replicate or [OctoAI](../3p-integrations/octoai/video_summary.ipynb)) This demo app uses Llama 3 to return a text summary of a YouTube video. It shows how to retrieve the caption of a YouTube video and how to ask Llama to summarize the content in different ways, from the simplest naive way that works for short text to more advanced methods of using LangChain's map_reduce and refine to overcome the 8K context length limit of Llama 3. -## [NBA2023-24](./coding/text2sql/quickstart.ipynb): Ask Llama 3 about Structured Data +## [NBA2023-24](./coding/text2sql/quickstart.ipynb): + +### Ask Llama 3 about Structured Data This demo app shows how to use LangChain and Llama 3 to let users ask questions about **structured** data stored in a SQL DB. As the 2023-24 NBA season is entering the playoff, we use the NBA roster info saved in a SQLite DB to show you how to ask Llama 3 questions about your favorite teams or players. -## [NotebookLlama](./NotebookLlama/): PDF to Podcast using Llama Models +## [NotebookLlama](./NotebookLlama/): + +### PDF to Podcast using Llama Models Workflow showcasing how to use multiple Llama models to go from any PDF to a Podcast and using open models to generate a multi-speaker podcast -## [live_data](live_data.ipynb): Ask Llama 3 about Live Data (using Replicate or [OctoAI](../3p-integrations/octoai/live_data.ipynb)) +## [live_data](live_data.ipynb): + +### Ask Llama 3 about Live Data (using Replicate or [OctoAI](../3p-integrations/octoai/live_data.ipynb)) This demo app shows how to perform live data augmented generation tasks with Llama 3, [LlamaIndex](https://github.com/run-llama/llama_index), another leading open-source framework for building LLM apps, and the [Tavily](https://tavily.com) live search API. -## [WhatsApp Chatbot](./customerservice_chatbots/whatsapp_chatbot/whatsapp_llama3.md): Building a Llama 3 Enabled WhatsApp Chatbot +## [WhatsApp Chatbot](./customerservice_chatbots/whatsapp_chatbot/whatsapp_llama3.md): +### Building a Llama 3 Enabled WhatsApp Chatbot This step-by-step tutorial shows how to use the [WhatsApp Business API](https://developers.facebook.com/docs/whatsapp/cloud-api/overview) to build a Llama 3 enabled WhatsApp chatbot. -## [Messenger Chatbot](./customerservice_chatbots/messenger_chatbot/messenger_llama3.md): Building a Llama 3 Enabled Messenger Chatbot +## [Messenger Chatbot](./customerservice_chatbots/messenger_chatbot/messenger_llama3.md): + +### Building a Llama 3 Enabled Messenger Chatbot This step-by-step tutorial shows how to use the [Messenger Platform](https://developers.facebook.com/docs/messenger-platform/overview) to build a Llama 3 enabled Messenger chatbot. ### RAG Chatbot Example (running [locally](./customerservice_chatbots/RAG_chatbot/RAG_Chatbot_Example.ipynb) or on [OctoAI](../3p-integrations/octoai/RAG_chatbot_example/RAG_chatbot_example.ipynb)) A complete example of how to build a Llama 3 chatbot hosted on your browser that can answer questions based on your own data using retrieval augmented generation (RAG). You can run Llama2 locally if you have a good enough GPU or on OctoAI if you follow the note [here](../README.md#octoai_note). -## [Sales Bot](./customerservice_chatbots/ai_agent_chatbot/SalesBot.ipynb): Sales Bot with Llama3 - A Summarization and RAG Use Case +## [Sales Bot](./customerservice_chatbots/ai_agent_chatbot/SalesBot.ipynb): + +### Sales Bot with Llama3 - A Summarization and RAG Use Case An summarization + RAG use case built around the Amazon product review Kaggle dataset to build a helpful Music Store Sales Bot. The summarization and RAG are built on top of Llama models hosted on OctoAI, and the vector database is hosted on Weaviate Cloud Services.