Skip to content
Snippets Groups Projects
Unverified Commit 523ccc29 authored by Andrei Fajardo's avatar Andrei Fajardo Committed by GitHub
Browse files

Fix condense chat engine nb — use stream_chat instead of just chat (#11033)

parent 3356cf78
Branches
Tags
No related merge requests found
%% Cell type:markdown id:8f1a9a96 tags:
<a href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/customization/streaming/chat_engine_condense_question_stream_response.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
%% Cell type:markdown id:18e20fbc-056b-44ac-b1fc-2d34b8e99bcc tags:
# Streaming for Chat Engine - Condense Question Mode
%% Cell type:markdown id:1778820a-4c58-4b2f-8b1a-3d6f1f49995d tags:
Load documents, build the VectorStoreIndex
%% Cell type:code id:a9ac125a-79df-452d-9f58-ac4f30997acf tags:
``` python
import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
```
%% Output
INFO:numexpr.utils:Note: NumExpr detected 12 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
Note: NumExpr detected 12 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
INFO:numexpr.utils:NumExpr defaulting to 8 threads.
NumExpr defaulting to 8 threads.
/Users/suo/miniconda3/envs/llama/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.7) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.
warnings.warn(
%% Cell type:markdown id:8f8bcf7a tags:
Download Data
%% Cell type:code id:8b8d3a56 tags:
``` python
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'
```
%% Output
--2024-02-20 11:00:23-- https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 75042 (73K) [text/plain]
Saving to: ‘data/paul_graham/paul_graham_essay.txt’
data/paul_graham/pa 100%[===================>] 73.28K --.-KB/s in 0.05s
2024-02-20 11:00:23 (1.59 MB/s) - ‘data/paul_graham/paul_graham_essay.txt’ saved [75042/75042]
%% Cell type:code id:c4a6b55f tags:
``` python
# load documents
documents = SimpleDirectoryReader("./data/paul_graham").load_data()
```
%% Cell type:code id:fa67c282 tags:
``` python
index = VectorStoreIndex.from_documents(documents)
```
%% Output
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
%% Cell type:markdown id:63a4259d-89b5-49f8-b158-9eba5353d6f5 tags:
Chat with your data
%% Cell type:code id:825b5bb3-37ff-4886-be2c-264584ca9eab tags:
``` python
chat_engine = index.as_chat_engine(
chat_mode="condense_question", streaming=True
)
response_stream = chat_engine.chat("What did Paul Graham do after YC?")
response_stream = chat_engine.stream_chat("What did Paul Graham do after YC?")
```
%% Output
INFO:llama_index.chat_engine.condense_question:Querying with: What was the next step in Paul Graham's career after his involvement with Y Combinator?
Querying with: What was the next step in Paul Graham's career after his involvement with Y Combinator?
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
INFO:llama_index.core.chat_engine.condense_question:Querying with: What did Paul Graham do after his time at Y Combinator?
Querying with: What did Paul Graham do after his time at Y Combinator?
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
%% Cell type:code id:d8fa4310-4dc5-4787-a073-755d2e0b4887 tags:
``` python
response_stream.print_response_stream()
```
%% Output
Paul Graham's next step in his career after his involvement with Y Combinator was to take up painting. He spent most of the rest of 2014 painting and then in March 2015 he started working on Lisp again.
Paul Graham decided to hand over Y Combinator to someone else after his time there. He asked Jessica if she wanted to be president, but she declined. Eventually, they recruited Sam Altman to take over as president. Paul Graham, along with Robert, retired from Y Combinator, while Jessica and Trevor became ordinary partners.
%% Cell type:markdown id:67021e64-8665-4338-9fb4-c0f1d6361092 tags:
Ask a follow up question
%% Cell type:code id:f6181319-5d76-48c4-a5d4-23c6e9bc5ccb tags:
``` python
response_stream = chat_engine.chat("What about after that?")
response_stream = chat_engine.stream_chat("What about after that?")
```
%% Output
INFO:llama_index.chat_engine.condense_question:Querying with: What did Paul Graham do after he started working on Lisp again in March 2015?
Querying with: What did Paul Graham do after he started working on Lisp again in March 2015?
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
INFO:llama_index.core.chat_engine.condense_question:Querying with: What did Paul Graham do after handing over Y Combinator to Sam Altman and retiring from the company?
Querying with: What did Paul Graham do after handing over Y Combinator to Sam Altman and retiring from the company?
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
%% Cell type:code id:95045f5b-7964-4872-bc91-809d9debf1f5 tags:
``` python
response_stream.print_response_stream()
```
%% Output
Paul Graham spent the rest of 2015 writing essays and working on the new dialect of Lisp he called Arc. He also looked for an apartment to buy and started to plan a second still life painting from the same objects.
After handing over Y Combinator to Sam Altman and retiring from the company, Paul Graham started his own investment firm with Jessica.
%% Cell type:code id:72cc02dd-90b7-4d63-bdb2-e4c4666f87ef tags:
``` python
response_stream = chat_engine.chat("Can you tell me more?")
response_stream = chat_engine.stream_chat("Can you tell me more?")
```
%% Output
INFO:llama_index.chat_engine.condense_question:Querying with: What did Paul Graham do after he started working on the new dialect of Lisp he called Arc in 2015?
Querying with: What did Paul Graham do after he started working on the new dialect of Lisp he called Arc in 2015?
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
INFO:llama_index.core.chat_engine.condense_question:Querying with: Can you tell me more about Paul Graham's investment firm that he started with Jessica after retiring from Y Combinator?
Querying with: Can you tell me more about Paul Graham's investment firm that he started with Jessica after retiring from Y Combinator?
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
%% Cell type:code id:d4f8efbb-fcb0-4c58-b92b-d2264a7e7103 tags:
``` python
response_stream.print_response_stream()
```
%% Output
Paul Graham worked on the new dialect of Lisp he called Arc for four years, from March 26, 2015 to October 12, 2019. During this time, he wrote the new Lisp, called Bel, in Arc. He also wrote essays and took his children to the coast on a sunny day in 2015. In the summer of 2016, he and his family moved to England. Finally, in the fall of 2019, he finished the project.
Paul Graham started an investment firm with Jessica after retiring from Y Combinator. They decided to create their own investment firm to implement the ideas they had been discussing. Paul funded the firm, allowing Jessica to quit her job and work for it, while also bringing on Robert and Trevor as partners. The firm was structured as an angel firm, which was a novel concept at the time. They aimed not only to make seed investments but also to provide comprehensive support to startups, similar to the help they had received when starting their own company. The investment firm was not organized as a fund and was funded with their own money. The distinctive aspect of their approach was the batch model, where they funded multiple startups at once and provided intensive support over a three-month period.
%% Cell type:markdown id:c2c68de8-af58-4f7e-8759-19fc072873fd tags:
Reset conversation state
%% Cell type:code id:d13cf082-1a91-43c5-8bad-76fa45be96f9 tags:
``` python
chat_engine.reset()
```
%% Cell type:code id:627de435-d195-4dad-b314-a68e731979a9 tags:
``` python
response_stream = chat_engine.chat("What about after that?")
response_stream = chat_engine.stream_chat("What about after that?")
```
%% Output
INFO:llama_index.chat_engine.condense_question:Querying with: What happens after the current situation?
Querying with: What happens after the current situation?
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
INFO:llama_index.core.chat_engine.condense_question:Querying with: What will happen after that?
Querying with: What will happen after that?
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
%% Cell type:code id:75ef9e31-3cdb-4129-92f7-e61be201ea36 tags:
``` python
response_stream.print_response_stream()
```
%% Output
After the current situation, the narrator resumes painting and experimenting with a new kind of still life. He also resumes his old life in New York, now that he is rich. He is able to take taxis and eat in restaurants, which is exciting for a while. He also starts to connect with other people who are trying to paint in New York.
After that, the individual started working on Lisp again in March 2015. The distinctive aspect of Lisp is that its core is a language defined by writing an interpreter in itself. Initially intended as a formal model of computation, Lisp evolved into a programming language as well. The individual's interest in Lisp stemmed from its power and elegance derived from its origins as a model of computation.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment