naming tweaks

993fea07 · Thierry Moreau · 1ca4a9c5 · 993fea07
Commit 993fea07 authored 1 year ago by Thierry Moreau
--- a/demo_apps/OctoAI_API_examples/Llama2_Gradio.ipynb
+++ b/demo_apps/OctoAI_API_examples/Llama2_Gradio.ipynb
@@ -89,8 +89,8 @@
    "        history_langchain_format.append(HumanMessage(content=human))\n",
    "        history_langchain_format.append(AIMessage(content=ai))\n",
    "    history_langchain_format.append(HumanMessage(content=message))\n",
-    "    gpt_response = llm(message) #history_langchain_format)\n",
-    "    return gpt_response#.content\n",
+    "    llm_response = llm(message, history_langchain_format)\n",
+    "    return llm_response.content\n",
    "\n",
    "gr.ChatInterface(predict).launch()"
   ]

 %% Cell type:markdown id:47a9adb3 tags:

 ## This demo app shows how to query Llama 2 using the Gradio UI.

 Since we are using OctoAI in this example, you'll need to obtain an OctoAI token:

 - You will need to first sign into [OctoAI](https://octoai.cloud/) with your Github or Google account
 - Then create a free API token [here](https://octo.ai/docs/getting-started/how-to-create-an-octoai-access-token) that you can use for a while (a month or $10 in OctoAI credits, whichever one runs out first)

 **Note** After the free trial ends, you will need to enter billing info to continue to use Llama2 hosted on OctoAI.

 To run this example:
 - Run the notebook
 - Set up your OCTOAI API token and enter it when prompted
 - Enter your question and click Submit

 In the notebook or a browser with URL http://127.0.0.1:7860 you should see a UI with your answer.

 Let's start by installing the necessary packages:
 - langchain provides necessary RAG tools for this demo
 - octoai-sdk allows us to use OctoAI Llama 2 endpoint
 - gradio is used for the UI elements

 And setting up the OctoAI token.

 %% Cell type:code id:6ae4f858-6ef7-49d9-b45b-1ef79d0217a0 tags:

 ``` python
 !pip install langchain octoai-sdk gradio
 ```

 %% Cell type:code id:3306c11d-ed82-41c5-a381-15fb5c07d307 tags:

 ``` python
 from getpass import getpass
 import os

 OCTOAI_API_TOKEN = getpass()
 os.environ["OCTOAI_API_TOKEN"] = OCTOAI_API_TOKEN
 ```

 %% Cell type:code id:928041cc tags:

 ``` python
 from langchain.schema import AIMessage, HumanMessage
 import gradio as gr
 from langchain.llms.octoai_endpoint import OctoAIEndpoint

 llama2_13b = "llama-2-13b-chat-fp16"

 llm = OctoAIEndpoint(
    endpoint_url="https://text.octoai.run/v1/chat/completions",
    model_kwargs={
        "model": llama2_13b,
        "messages": [
            {
                "role": "system",
                "content": "You are a helpful, respectful and honest assistant."
            }
        ],
        "max_tokens": 500,
        "top_p": 1,
        "temperature": 0.01
    },
 )


 def predict(message, history):
    history_langchain_format = []
    for human, ai in history:
        history_langchain_format.append(HumanMessage(content=human))
        history_langchain_format.append(AIMessage(content=ai))
    history_langchain_format.append(HumanMessage(content=message))
-    gpt_response = llm(message) #history_langchain_format)
-    return gpt_response#.content
+    llm_response = llm(message, history_langchain_format)
+    return llm_response.content

 gr.ChatInterface(predict).launch()
 ```