Skip to content
Snippets Groups Projects
Unverified Commit 69e3f85d authored by Jeff Tang's avatar Jeff Tang Committed by GitHub
Browse files

update text2sql demo to using Llama 3.3 (#822)

parents ccc0b533 012b10c2
Branches archive-main
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
recipes/use_cases/coding/text2sql/structured_llama.ipynb → recipes/use_cases/coding/text2sql/quickstart.ipynb
+ 334
− 0
View file @ 69e3f85d
%% Cell type:markdown id:e8cba0b6 tags: %% Cell type:markdown id:e8cba0b6 tags:
<a href="https://colab.research.google.com/github/meta-llama/llama-recipes/blob/main/recipes/use_cases/text2sql/StructuredLlama.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> <a href="https://colab.research.google.com/github/meta-llama/llama-recipes/blob/main/recipes/use_cases/coding/text2sql/quickstart.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
## Use Llama 3 to chat about structured data ## Quick Demo of Text2SQL Using Llama 3.3
This demo shows how to use LangChain with Llama 3 to query structured data, the 2023-24 NBA roster info, stored in a SQLite DB, and how to ask Llama 3 follow up question about the DB.
%% Cell type:markdown id:f839d07d tags: This demo shows how to use Llama 3.3 to answer questions about a SQLite DB.
We start by installing the necessary packages: We'll use LangChain and the Llama cloud provider [Together.ai](https://api.together.ai/), where you can easily get a free API key (or you can use any other Llama cloud provider or even Ollama running Llama locally - see [here](https://github.com/meta-llama/llama-recipes/tree/main/recipes/quickstart) for examples).
- [Replicate](https://replicate.com/) to host the Llama 3 model
- [langchain](https://python.langchain.com/docs/get_started/introduction) provides necessary RAG tools for this demo
**Note** We will be using [Replicate](https://replicate.com/meta/meta-llama-3-8b-instruct) to run the examples here. You will need to first sign in with Replicate with your github account, then create a free API token [here](https://replicate.com/account/api-tokens) that you can use for a while. You can also use other Llama 3 cloud providers such as [Groq](https://console.groq.com/), [Together](https://api.together.xyz/playground/language/meta-llama/Llama-3-8b-hf), or [Anyscale](https://app.endpoints.anyscale.com/playground) - see Section 2 of the Getting to Know Llama [notebook](https://github.com/meta-llama/llama-recipes/blob/main/recipes/quickstart/Getting_to_know_Llama.ipynb) for more information.
If you'd like to run Llama 3 locally for the benefits of privacy, no cost or no rate limit (some Llama 3 hosting providers set limits for free plan of queries or tokens per second or minute), see [Running Llama Locally](https://github.com/meta-llama/llama-recipes/blob/main/recipes/quickstart/Running_Llama2_Anywhere/Running_Llama_on_Mac_Windows_Linux.ipynb).
%% Cell type:code id:33fb3190-59fb-4edd-82dd-f20f6eab3e47 tags: %% Cell type:code id:33fb3190-59fb-4edd-82dd-f20f6eab3e47 tags:
``` python ``` python
!pip install langchain replicate !pip install -U langchain langchain-community langchain-together
``` ```
%% Cell type:code id:fa4562d3 tags: %% Cell type:code id:fa4562d3 tags:
``` python ``` python
from getpass import getpass
import os import os
from langchain_together import ChatTogether
REPLICATE_API_TOKEN = getpass() os.environ['TOGETHER_API_KEY'] = 'your_api_key'
os.environ["REPLICATE_API_TOKEN"] = REPLICATE_API_TOKEN
```
%% Cell type:markdown id:1e586b75 tags:
Next we call the Llama 3 8b chat model from Replicate. You can also use Llama 3 70b model by replacing the `model` name with "meta/meta-llama-3-70b-instruct". llm = ChatTogether(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
%% Cell type:code id:9dcd744c tags: temperature=0,
``` python
from langchain_community.llms import Replicate
llm = Replicate(
model="meta/meta-llama-3-8b-instruct",
model_kwargs={"temperature": 0.0, "top_p": 1, "max_new_tokens":500}
) )
``` ```
%% Cell type:markdown id:6d421ae7 tags: %% Cell type:markdown id:6d421ae7 tags:
To recreate the `nba_roster.db` file, run the two commands below: To recreate the `nba_roster.db` file, run the two commands below:
- `python txt2csv.py` to convert the `nba.txt` file to `nba_roster.csv`. The `nba.txt` file was created by scraping the NBA roster info from the web. - `python txt2csv.py` to convert the `nba.txt` file to `nba_roster.csv`. The `nba.txt` file was created by scraping the NBA roster info from the web.
- `python csv2db.py` to convert `nba_roster.csv` to `nba_roster.db`. - `python csv2db.py` to convert `nba_roster.csv` to `nba_roster.db`.
%% Cell type:code id:56f0360e-fca3-49a8-9a70-0416f84e15fc tags:
``` python
# uncomment if you don't want to create the db yourself
#! wget https://github.com/meta-llama/llama-recipes/raw/3649841b426999fdc61c30a9fc8721106bec769b/recipes/use_cases/coding/text2sql/nba_roster.db
```
%% Cell type:code id:3bb99f39-cd7a-4db6-91dd-02f3bf80347c tags: %% Cell type:code id:3bb99f39-cd7a-4db6-91dd-02f3bf80347c tags:
``` python ``` python
from langchain_community.utilities import SQLDatabase from langchain_community.utilities import SQLDatabase
# Note: to run in Colab, you need to upload the nba_roster.db file in the repo to the Colab folder first. # Note: to run in Colab, you need to upload the nba_roster.db file in the repo to the Colab folder first.
db = SQLDatabase.from_uri("sqlite:///nba_roster.db", sample_rows_in_table_info=0) db = SQLDatabase.from_uri("sqlite:///nba_roster.db", sample_rows_in_table_info=0)
def get_schema(): def get_schema():
return db.get_table_info() return db.get_table_info()
``` ```
%% Cell type:code id:8d793ce7-324b-4861-926c-54973d7c9b43 tags: %% Cell type:code id:8d793ce7-324b-4861-926c-54973d7c9b43 tags:
``` python ``` python
question = "What team is Klay Thompson on?" question = "What team is Stephen Curry on?"
prompt = f"""Based on the table schema below, write a SQL query that would answer the user's question; just return the SQL query and nothing else. prompt = f"""Based on the table schema below, write a SQL query that would answer the user's question; just return the SQL query and nothing else.
Scheme: Scheme:
{get_schema()} {get_schema()}
Question: {question} Question: {question}
SQL Query:""" SQL Query:"""
print(prompt) print(prompt)
``` ```
%% Output
Based on the table schema below, write a SQL query that would answer the user's question; just return the SQL query and nothing else.
Scheme:
CREATE TABLE nba_roster (
"Team" TEXT,
"NAME" TEXT,
"Jersey" TEXT,
"POS" TEXT,
"AGE" INTEGER,
"HT" TEXT,
"WT" TEXT,
"COLLEGE" TEXT,
"SALARY" TEXT
)
Question: What team is Stephen Curry on?
SQL Query:
%% Cell type:code id:70776558 tags: %% Cell type:code id:70776558 tags:
``` python ``` python
answer = llm.invoke(prompt) answer = llm.invoke(prompt).content
print(answer) print(answer)
``` ```
%% Output
SELECT Team FROM nba_roster WHERE NAME = 'Stephen Curry'
%% Cell type:markdown id:afcf423a tags: %% Cell type:markdown id:afcf423a tags:
If you don't have the "just return the SQL query and nothing else" in the prompt above, or even with it but asking Llama 2 which doesn't follow instructions as well as Llama 3, you'll likely get more text other than the SQL query back in the answer. ***Note:*** If you don't have the "just return the SQL query and nothing else" in the prompt above, you'll likely get more text other than the SQL query back in the answer, making some extra post-processing necessary before running the db query below.
%% Cell type:code id:62472ce6-794b-4a61-b88c-a1e031e28e4e tags: %% Cell type:code id:62472ce6-794b-4a61-b88c-a1e031e28e4e tags:
``` python ``` python
# note this is a dangerous operation and for demo purpose only; in production app you'll need to safe-guard any DB operation # note this is a dangerous operation and for demo purpose only; in production app you'll need to safe-guard any DB operation
result = db.run(answer) result = db.run(answer)
result
``` ```
%% Output
"[('Golden State Warriors',)]"
%% Cell type:code id:39ed4bc3 tags: %% Cell type:code id:39ed4bc3 tags:
``` python ``` python
# how about a follow up question # how about a follow up question
follow_up = "What's his salary?" follow_up = "What's his salary?"
print(llm.invoke(follow_up)) print(llm.invoke(follow_up).content)
``` ```
%% Cell type:markdown id:98b2c523 tags: %% Output
I don't have enough information to determine whose salary you are referring to. Could you please provide more context or specify the person you are asking about?
Since we did not pass any context along with the follow-up to the model it did not know who "his" is and just picked LeBron James. %% Cell type:markdown id:98b2c523 tags:
Let's try to fix it by adding context to the follow-up prompt. Since we did not pass any context along with the follow-up to Llama, it doesn't know the answer. Let's try to fix it by adding context to the follow-up prompt.
%% Cell type:code id:0c305278-29d2-4e88-9b3d-ad67c94ce0f2 tags: %% Cell type:code id:0c305278-29d2-4e88-9b3d-ad67c94ce0f2 tags:
``` python ``` python
prompt = f"""Based on the table schema, question, SQL query, and SQL response below, write a new SQL response; be concise, just output the SQL response. prompt = f"""Based on the table schema, question, SQL query, and SQL response below, write a new SQL response; be concise, just output the SQL response.
Scheme: Scheme:
{get_schema()} {get_schema()}
Question: {follow_up} Question: {follow_up}
SQL Query: {question} SQL Query: {question}
SQL Result: {result} SQL Result: {result}
New SQL Response: New SQL Response:
""" """
print(prompt) print(prompt)
``` ```
%% Output
Based on the table schema, question, SQL query, and SQL response below, write a new SQL response; be concise, just output the SQL response.
Scheme:
CREATE TABLE nba_roster (
"Team" TEXT,
"NAME" TEXT,
"Jersey" TEXT,
"POS" TEXT,
"AGE" INTEGER,
"HT" TEXT,
"WT" TEXT,
"COLLEGE" TEXT,
"SALARY" TEXT
)
Question: What's his salary?
SQL Query: What team is Stephen Curry on?
SQL Result: [('Golden State Warriors',)]
New SQL Response:
%% Cell type:code id:03739b96-e607-4fa9-bc5c-df118198dc7f tags: %% Cell type:code id:03739b96-e607-4fa9-bc5c-df118198dc7f tags:
``` python ``` python
new_answer = llm.invoke(prompt) new_answer = llm.invoke(prompt).content
print(new_answer) print(new_answer)
``` ```
%% Output
SELECT SALARY FROM nba_roster WHERE NAME = "Stephen Curry"
%% Cell type:markdown id:c782abb6-3b44-45be-8694-70fc29b82523 tags: %% Cell type:markdown id:c782abb6-3b44-45be-8694-70fc29b82523 tags:
Because we have "be concise, just output the SQL response", Llama 3 is able to just generate the SQL statement; otherwise output parsing will be needed. Because we have "be concise, just output the SQL response", Llama 3 is able to just generate the SQL statement; otherwise output parsing will be needed.
%% Cell type:code id:6ecfca53-be7e-4668-bad1-5ca7571817d7 tags: %% Cell type:code id:6ecfca53-be7e-4668-bad1-5ca7571817d7 tags:
``` python ``` python
db.run(new_answer) db.run(new_answer)
``` ```
%% Output
"[('$51,915,615',)]"
%% Cell type:code id:9d79bbb1-e91d-4b56-b6ef-98c94ff414d0 tags:
``` python
```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

:feet::heart: Strive to be the person your dogs believe you are :heart::feet: