Skip to content
Snippets Groups Projects
Commit 6b38f126 authored by Joshua Briggs's avatar Joshua Briggs
Browse files

fix: updates semantic version on these docs 

notes:
01 - when loading a doc im creating a new instance of rl instead of using the same rl, you might want to check this to make sure its correct.
05 - cannot get the LLM to run with the semantic router
07 - compatibility issues with the encoder, also think the dependency semantic_router[vision] needs updating
parent 69b49329
No related branches found
No related tags found
No related merge requests found
Expand all Hide whitespace changes
Inline Side-by-side
docs/01-save-load-from-file.ipynb
+ 68
− 32
View file @ 6b38f126
%% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/01-save-load-from-file.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/01-save-load-from-file.ipynb)
%% Cell type:markdown id: tags:
# Route Layers from File
Here we will show how to save routers to YAML or JSON files, and how to load a route layer from file.
%% Cell type:markdown id: tags:
## Getting Started
%% Cell type:markdown id: tags:
We start by installing the library:
%% Cell type:code id: tags:
``` python
!pip install -qU semantic-router
!pip install -qU "semantic-router==0.1.0.dev3"
```
%% Cell type:markdown id: tags:
## Define Route
%% Cell type:markdown id: tags:
First let's create a list of routes:
%% Cell type:code id: tags:
``` python
from semantic_router import Route
politics = Route(
name="politics",
utterances=[
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"don't you just love the president" "don't you just hate the president",
"they're going to destroy this country!",
"they will save the country!",
],
)
chitchat = Route(
name="chitchat",
utterances=[
"how's the weather today?",
"how are things going?",
"lovely weather today",
"the weather is horrendous",
"let's go to the chippy",
],
)
routes = [politics, chitchat]
```
%% Output
c:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
c:\Users\Joshu\OneDrive\Documents\Aurelio\agents-course\07-pratical-ai\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
%% Cell type:markdown id: tags:
We define a route layer using these routes and using the Cohere encoder.
%% Cell type:code id: tags:
``` python
import os
from getpass import getpass
from semantic_router import RouteLayer
from semantic_router.encoders import CohereEncoder
from semantic_router.routers import SemanticRouter
# dashboard.cohere.ai
os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass(
"Enter Cohere API Key: "
)
encoder = CohereEncoder()
rl = RouteLayer(encoder=encoder, routes=routes)
rl = SemanticRouter(encoder=encoder, routes=routes, auto_sync="local")
```
%% Output
2024-05-07 15:03:35 INFO semantic_router.utils.logger local
2025-01-06 11:59:56 - semantic_router.utils.logger - WARNING - base.py:356 - _get_index() - No index provided. Using default LocalIndex.
2025-01-06 11:59:56 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 11:59:57 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 11:59:57 - semantic_router.utils.logger - WARNING - local.py:148 - _write_config() - No config is written for LocalIndex.
%% Cell type:markdown id: tags:
## Test Route
%% Cell type:code id: tags:
``` python
rl("isn't politics the best thing ever")
```
%% Output
2025-01-06 12:00:03 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:00:08 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:00:42 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:03:49 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:03:49 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:04:00 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:04:09 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:04:38 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:04:38 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
RouteChoice(name='politics', function_call=None, similarity_score=None)
%% Cell type:code id: tags:
``` python
rl("how's the weather today?")
```
%% Output
RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags:
## Save To JSON
%% Cell type:markdown id: tags:
To save our route layer we call the `to_json` method:
%% Cell type:code id: tags:
``` python
rl.to_json("layer.json")
```
%% Output
2024-05-07 15:03:37 INFO semantic_router.utils.logger Saving route config to layer.json
2025-01-06 12:00:14 - semantic_router.utils.logger - INFO - base.py:211 - to_file() - Saving route config to layer.json
%% Cell type:markdown id: tags:
## Loading from JSON
%% Cell type:markdown id: tags:
We can view the router file we just saved to see what information is stored.
%% Cell type:code id: tags:
``` python
import json
with open("layer.json", "r") as f:
layer_json = json.load(f)
print(layer_json)
```
%% Output
{'encoder_type': 'cohere', 'encoder_name': 'embed-english-v3.0', 'routes': [{'name': 'politics', 'utterances': ["isn't politics the best thing ever", "why don't you tell me about your political opinions", "don't you just love the presidentdon't you just hate the president", "they're going to destroy this country!", 'they will save the country!'], 'description': None, 'function_schemas': None, 'llm': None, 'score_threshold': 0.3}, {'name': 'chitchat', 'utterances': ["how's the weather today?", 'how are things going?', 'lovely weather today', 'the weather is horrendous', "let's go to the chippy"], 'description': None, 'function_schemas': None, 'llm': None, 'score_threshold': 0.3}]}
{'encoder_type': 'cohere', 'encoder_name': 'embed-english-v3.0', 'routes': [{'name': 'politics', 'utterances': ["isn't politics the best thing ever", "why don't you tell me about your political opinions", "don't you just love the presidentdon't you just hate the president", "they're going to destroy this country!", 'they will save the country!'], 'description': None, 'function_schemas': None, 'llm': None, 'score_threshold': 0.3, 'metadata': {}}, {'name': 'chitchat', 'utterances': ["how's the weather today?", 'how are things going?', 'lovely weather today', 'the weather is horrendous', "let's go to the chippy"], 'description': None, 'function_schemas': None, 'llm': None, 'score_threshold': 0.3, 'metadata': {}}]}
%% Cell type:markdown id: tags:
It tells us our encoder type, encoder name, and routes. This is everything we need to initialize a new router. To do so, we use the `from_json` method.
%% Cell type:code id: tags:
``` python
rl = RouteLayer.from_json("layer.json")
rl = SemanticRouter.from_json("layer.json")
```
%% Output
2024-05-07 15:03:37 INFO semantic_router.utils.logger Loading route config from layer.json
2024-05-07 15:03:37 INFO semantic_router.utils.logger local
2025-01-06 12:00:30 - semantic_router.utils.logger - INFO - base.py:96 - from_file() - Loading route config from layer.json
2025-01-06 12:00:31 - semantic_router.utils.logger - WARNING - base.py:356 - _get_index() - No index provided. Using default LocalIndex.
%% Cell type:markdown id: tags:
We can confirm that our layer has been initialized with the expected attributes by viewing the `RouteLayer` object:
%% Cell type:code id: tags:
``` python
print(
f"""{rl.encoder.type=}
{rl.encoder.name=}
{rl.routes=}"""
)
```
%% Output
rl.encoder.type='cohere'
rl.encoder.name='embed-english-v3.0'
rl.routes=[Route(name='politics', utterances=["isn't politics the best thing ever", "why don't you tell me about your political opinions", "don't you just love the presidentdon't you just hate the president", "they're going to destroy this country!", 'they will save the country!'], description=None, function_schemas=None, llm=None, score_threshold=0.3), Route(name='chitchat', utterances=["how's the weather today?", 'how are things going?', 'lovely weather today', 'the weather is horrendous', "let's go to the chippy"], description=None, function_schemas=None, llm=None, score_threshold=0.3)]
rl.routes=[Route(name='politics', utterances=["isn't politics the best thing ever", "why don't you tell me about your political opinions", "don't you just love the presidentdon't you just hate the president", "they're going to destroy this country!", 'they will save the country!'], description=None, function_schemas=None, llm=None, score_threshold=0.3, metadata={}), Route(name='chitchat', utterances=["how's the weather today?", 'how are things going?', 'lovely weather today', 'the weather is horrendous', "let's go to the chippy"], description=None, function_schemas=None, llm=None, score_threshold=0.3, metadata={})]
%% Cell type:code id: tags:
``` python
new_rl = SemanticRouter(encoder=rl.encoder, routes=rl.routes, auto_sync="local")
```
%% Output
2025-01-06 12:04:38 - semantic_router.utils.logger - WARNING - base.py:356 - _get_index() - No index provided. Using default LocalIndex.
2025-01-06 12:04:38 - semantic_router.utils.logger - WARNING - local.py:148 - _write_config() - No config is written for LocalIndex.
%% Cell type:markdown id: tags:
---
%% Cell type:markdown id: tags:
## Test Route Again
%% Cell type:code id: tags:
``` python
rl("isn't politics the best thing ever")
new_rl("isn't politics the best thing ever")
```
%% Output
RouteChoice(name='politics', function_call=None, similarity_score=None)
%% Cell type:code id: tags:
``` python
rl("how's the weather today?")
new_rl("how's the weather today?")
```
%% Output
RouteChoice(name='chitchat', function_call=None, similarity_score=None)
......
docs/02-dynamic-routes.ipynb
+ 116
− 75
View file @ 6b38f126
This diff is collapsed.
docs/03-basic-langchain-agent.ipynb
+ 153
− 71
View file @ 6b38f126
This diff is collapsed.
docs/06-threshold-optimization.ipynb
+ 47
− 29
View file @ 6b38f126
%% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/06-threshold-optimization.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/06-threshold-optimization)
%% Cell type:markdown id: tags:
# Route Threshold Optimization
%% Cell type:markdown id: tags:
Route score thresholds are what defines whether a route should be chosen. If the score we identify for any given route is higher than the `Route.score_threshold` it passes, otherwise it does not and _either_ another route is chosen, or we return _no_ route.
Given that this one `score_threshold` parameter can define the choice of a route, it's important to get it right — but it's incredibly inefficient to do so manually. Instead, we can use the `fit` and `evaluate` methods of our `RouteLayer`. All we must do is pass a smaller number of _(utterance, target route)_ examples to our methods, and with `fit` we will often see dramatically improved performance within seconds — we will see how to measure that performance gain with `evaluate`.
%% Cell type:code id: tags:
``` python
!pip install -qU "semantic-router[local]"
!pip install -qU "semantic-router==0.1.0.dev3"
```
%% Cell type:markdown id: tags:
## Define RouteLayer
As usual we will define our `RouteLayer`. The `RouteLayer` requires just `routes` and an `encoder`. If using dynamic routes you must also define an `llm` (or use the OpenAI default).
We will start by defining four routes; _politics_, _chitchat_, _mathematics_, and _biology_.
%% Cell type:code id: tags:
``` python
from semantic_router import Route
# we could use this as a guide for our chatbot to avoid political conversations
politics = Route(
name="politics",
utterances=[
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"don't you just love the president" "don't you just hate the president",
"they're going to destroy this country!",
"they will save the country!",
],
)
# this could be used as an indicator to our chatbot to switch to a more
# conversational prompt
chitchat = Route(
name="chitchat",
utterances=[
"Did you watch the game last night?",
"what's your favorite type of music?",
"Have you read any good books lately?",
"nice weather we're having",
"Do you have any plans for the weekend?",
],
)
# we can use this to switch to an agent with more math tools, prompting, and LLMs
mathematics = Route(
name="mathematics",
utterances=[
"can you explain the concept of a derivative?",
"What is the formula for the area of a triangle?",
"how do you solve a system of linear equations?",
"What is the concept of a prime number?",
"Can you explain the Pythagorean theorem?",
],
)
# we can use this to switch to an agent with more biology knowledge
biology = Route(
name="biology",
utterances=[
"what is the process of osmosis?",
"can you explain the structure of a cell?",
"What is the role of RNA?",
"What is genetic mutation?",
"Can you explain the process of photosynthesis?",
],
)
# we place all of our decisions together into single list
routes = [politics, chitchat, mathematics, biology]
```
%% Output
c:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
c:\Users\Joshu\OneDrive\Documents\Aurelio\agents-course\07-pratical-ai\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
%% Cell type:markdown id: tags:
For our encoder we will use the local `HuggingFaceEncoder`. Other popular encoders include `CohereEncoder`, `FastEmbedEncoder`, `OpenAIEncoder`, and `AzureOpenAIEncoder`.
%% Cell type:code id: tags:
``` python
from semantic_router.encoders import HuggingFaceEncoder
encoder = HuggingFaceEncoder()
```
%% Output
c:\Users\Joshu\OneDrive\Documents\Aurelio\agents-course\07-pratical-ai\.venv\Lib\site-packages\huggingface_hub\file_download.py:140: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\Joshu\.cache\huggingface\hub\models--sentence-transformers--all-MiniLM-L6-v2. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
warnings.warn(message)
%% Cell type:markdown id: tags:
Now we initialize our `RouteLayer`.
%% Cell type:code id: tags:
``` python
from semantic_router.layer import RouteLayer
from semantic_router.routers import SemanticRouter
rl = RouteLayer(encoder=encoder, routes=routes)
rl = SemanticRouter(encoder=encoder, routes=routes, auto_sync="local")
```
%% Output
2024-05-07 15:53:24 INFO semantic_router.utils.logger local
2025-01-06 12:29:52 - semantic_router.utils.logger - WARNING - base.py:356 - _get_index() - No index provided. Using default LocalIndex.
2025-01-06 12:29:52 - semantic_router.utils.logger - WARNING - local.py:148 - _write_config() - No config is written for LocalIndex.
%% Cell type:markdown id: tags:
By default, we should get reasonable performance:
%% Cell type:code id: tags:
``` python
for utterance in [
"don't you love politics?",
"how's the weather today?",
"What's DNA?",
"I'm interested in learning about llama 2",
]:
print(f"{utterance} -> {rl(utterance).name}")
```
%% Output
don't you love politics? -> politics
how's the weather today? -> chitchat
What's DNA? -> biology
I'm interested in learning about llama 2 -> None
%% Cell type:markdown id: tags:
We can evaluate the performance of our route layer using the `evaluate` method. All we need is to pass a list of utterances and target route labels:
%% Cell type:code id: tags:
``` python
test_data = [
("don't you love politics?", "politics"),
("how's the weather today?", "chitchat"),
("What's DNA?", "biology"),
("I'm interested in learning about llama 2", None),
]
# unpack the test data
X, y = zip(*test_data)
# evaluate using the default thresholds
accuracy = rl.evaluate(X=X, y=y)
print(f"Accuracy: {accuracy*100:.2f}%")
```
%% Output
Generating embeddings: 100%|██████████| 1/1 [00:00<00:00, 76.91it/s]
Generating embeddings: 100%|██████████| 1/1 [00:00<00:00, 71.19it/s]
Accuracy: 100.00%
%% Cell type:markdown id: tags:
On this small subset we get perfect accuracy — but what if we try we a larger, more robust dataset?
_Hint: try using GPT-4 or another LLM to generate some examples for your own use-cases. The more accurate examples you provide, the better you can expect the routes to perform on your actual use-case._
%% Cell type:code id: tags:
``` python
test_data = [
# politics
("What's your opinion on the current government?", "politics"),
("Who do you think will win the next election?", "politics"),
("What are your thoughts on the new policy?", "politics"),
("How do you feel about the political situation?", "politics"),
("Do you agree with the president's actions?", "politics"),
("What's your stance on the political debate?", "politics"),
("How do you see the future of our country?", "politics"),
("What do you think about the opposition party?", "politics"),
("Do you believe the government is doing enough?", "politics"),
("What's your opinion on the political scandal?", "politics"),
("Do you think the new law will make a difference?", "politics"),
("What are your thoughts on the political reform?", "politics"),
("Do you agree with the government's foreign policy?", "politics"),
# chitchat
("What's the weather like?", "chitchat"),
("It's a beautiful day today.", "chitchat"),
("How's your day going?", "chitchat"),
("It's raining cats and dogs.", "chitchat"),
("Let's grab a coffee.", "chitchat"),
("What's up?", "chitchat"),
("It's a bit chilly today.", "chitchat"),
("How's it going?", "chitchat"),
("Nice weather we're having.", "chitchat"),
("It's a bit windy today.", "chitchat"),
("Let's go for a walk.", "chitchat"),
("How's your week been?", "chitchat"),
("It's quite sunny today.", "chitchat"),
("How are you feeling?", "chitchat"),
("It's a bit cloudy today.", "chitchat"),
# mathematics
("What is the Pythagorean theorem?", "mathematics"),
("Can you solve this quadratic equation?", "mathematics"),
("What is the derivative of x squared?", "mathematics"),
("Explain the concept of integration.", "mathematics"),
("What is the area of a circle?", "mathematics"),
("How do you calculate the volume of a sphere?", "mathematics"),
("What is the difference between a vector and a scalar?", "mathematics"),
("Explain the concept of a matrix.", "mathematics"),
("What is the Fibonacci sequence?", "mathematics"),
("How do you calculate permutations?", "mathematics"),
("What is the concept of probability?", "mathematics"),
("Explain the binomial theorem.", "mathematics"),
("What is the difference between discrete and continuous data?", "mathematics"),
("What is a complex number?", "mathematics"),
("Explain the concept of limits.", "mathematics"),
# biology
("What is photosynthesis?", "biology"),
("Explain the process of cell division.", "biology"),
("What is the function of mitochondria?", "biology"),
("What is DNA?", "biology"),
("What is the difference between prokaryotic and eukaryotic cells?", "biology"),
("What is an ecosystem?", "biology"),
("Explain the theory of evolution.", "biology"),
("What is a species?", "biology"),
("What is the role of enzymes?", "biology"),
("What is the circulatory system?", "biology"),
("Explain the process of respiration.", "biology"),
("What is a gene?", "biology"),
("What is the function of the nervous system?", "biology"),
("What is homeostasis?", "biology"),
("What is the difference between a virus and a bacteria?", "biology"),
("What is the role of the immune system?", "biology"),
# add some None routes to prevent excessively small thresholds
("What is the capital of France?", None),
("how many people live in the US?", None),
("when is the best time to visit Bali?", None),
("how do I learn a language", None),
("tell me an interesting fact", None),
("what is the best programming language?", None),
("I'm interested in learning about llama 2", None),
]
```
%% Cell type:code id: tags:
``` python
# unpack the test data
X, y = zip(*test_data)
# evaluate using the default thresholds
accuracy = rl.evaluate(X=X, y=y)
print(f"Accuracy: {accuracy*100:.2f}%")
```
%% Output
Generating embeddings: 100%|██████████| 1/1 [00:00<00:00, 9.23it/s]
Generating embeddings: 100%|██████████| 1/1 [00:00<00:00, 5.57it/s]
Accuracy: 34.85%
%% Cell type:markdown id: tags:
Ouch, that's not so good! Fortunately, we can easily improve our performance here.
%% Cell type:markdown id: tags:
## Route Layer Optimization
%% Cell type:markdown id: tags:
Our optimization works by finding the best route thresholds for each `Route` in our `RouteLayer`. We can see the current, default thresholds by calling the `get_thresholds` method:
%% Cell type:code id: tags:
``` python
route_thresholds = rl.get_thresholds()
print("Default route thresholds:", route_thresholds)
```
%% Output
Default route thresholds: {'politics': 0.5, 'chitchat': 0.5, 'mathematics': 0.5, 'biology': 0.5}
%% Cell type:markdown id: tags:
These are all preset route threshold values. Fortunately, it's very easy to optimize these — we simply call the `fit` method and provide our training utterances `X`, and target route labels `y`:
%% Cell type:code id: tags:
``` python
# Call the fit method
rl.fit(X=X, y=y)
```
%% Output
Generating embeddings: 100%|██████████| 1/1 [00:00<00:00, 9.21it/s]
Training: 100%|██████████| 500/500 [00:01<00:00, 419.45it/s, acc=0.89]
Generating embeddings: 100%|██████████| 1/1 [00:00<00:00, 6.28it/s]
Training: 100%|██████████| 500/500 [00:03<00:00, 147.86it/s, acc=0.91]
%% Cell type:markdown id: tags:
Let's see what our new thresholds look like:
%% Cell type:code id: tags:
``` python
route_thresholds = rl.get_thresholds()
print("Updated route thresholds:", route_thresholds)
```
%% Output
Updated route thresholds: {'politics': 0.05050505050505051, 'chitchat': 0.32323232323232326, 'mathematics': 0.18181818181818182, 'biology': 0.21212121212121213}
Updated route thresholds: {'politics': 0.25558616467707385, 'chitchat': 0.24022038567493115, 'mathematics': 0.22222222222222224, 'biology': 0.24242424242424243}
%% Cell type:markdown id: tags:
These are vastly different thresholds to what we were seeing before — it's worth noting that _optimal_ values for different encoders can vary greatly. For example, OpenAI's Ada 002 model, when used with our encoders will tend to output much larger numbers in the `0.5` to `0.8` range.
After training we have a final performance of:
%% Cell type:code id: tags:
``` python
accuracy = rl.evaluate(X=X, y=y)
print(f"Accuracy: {accuracy*100:.2f}%")
```
%% Output
Generating embeddings: 100%|██████████| 1/1 [00:00<00:00, 8.89it/s]
Generating embeddings: 100%|██████████| 1/1 [00:00<00:00, 6.53it/s]
Accuracy: 89.39%
Accuracy: 90.91%
%% Cell type:markdown id: tags:
That is _much_ better. If we wanted to optimize this further we can focus on adding more utterances to our existing routes, analyzing _where_ exactly our failures are, and modifying our routes around those. This extended optimzation process is much more manual, but with it we can continue optimizing routes to get even better performance.
......
docs/08-async-dynamic-routes.ipynb
+ 125
− 59
View file @ 6b38f126
%% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/08-async-dynamic-routes.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/08-async-dynamic-routes.ipynb)
%% Cell type:markdown id: tags:
# Dynamic Routes
%% Cell type:markdown id: tags:
In semantic-router there are two types of routes that can be chosen. Both routes belong to the `Route` object, the only difference between them is that _static_ routes return a `Route.name` when chosen, whereas _dynamic_ routes use an LLM call to produce parameter input values.
For example, a _static_ route will tell us if a query is talking about mathematics by returning the route name (which could be `"math"` for example). A _dynamic_ route does the same thing, but it also extracts key information from the input utterance to be used in a function associated with that route.
For example we could provide a dynamic route with associated utterances:
```
"what is x to the power of y?"
"what is 9 to the power of 4?"
"calculate the result of base x and exponent y"
"calculate the result of base 10 and exponent 3"
"return x to the power of y"
```
and we could also provide the route with a schema outlining key features of the function:
```
def power(base: float, exponent: float) -> float:
"""Raise base to the power of exponent.
Args:
base (float): The base number.
exponent (float): The exponent to which the base is raised.
Returns:
float: The result of base raised to the power of exponent.
"""
return base ** exponent
```
Then, if the users input utterance is "What is 2 to the power of 3?", the route will be triggered, as the input utterance is semantically similar to the route utterances. Furthermore, the route utilizes an LLM to identify that `base=2` and `expoenent=3`. These values are returned in such a way that they can be used in the above `power` function. That is, the dynamic router automates the process of calling relevant functions from natural language inputs.
***⚠️ Note: We have a fully local version of dynamic routes available at [docs/05-local-execution.ipynb](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb). The local 05 version tends to outperform the OpenAI version we demo in this notebook, so we'd recommend trying [05](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb)!***
%% Cell type:markdown id: tags:
## Installing the Library
%% Cell type:code id: tags:
``` python
!pip install -qU \
"semantic-router==0.0.54" \
"semantic-router==0.1.0.dev3" \
tzdata
```
%% Cell type:markdown id: tags:
## Initializing Routes and RouteLayer
%% Cell type:markdown id: tags:
Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes.
%% Cell type:code id: tags:
``` python
from semantic_router import Route
politics = Route(
name="politics",
utterances=[
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"don't you just love the president" "don't you just hate the president",
"they're going to destroy this country!",
"they will save the country!",
],
)
chitchat = Route(
name="chitchat",
utterances=[
"how's the weather today?",
"how are things going?",
"lovely weather today",
"the weather is horrendous",
"let's go to the chippy",
],
)
routes = [politics, chitchat]
```
%% Cell type:markdown id: tags:
We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`.
%% Cell type:code id: tags:
``` python
import os
from getpass import getpass
from semantic_router import RouteLayer
from semantic_router.routers import SemanticRouter
from semantic_router.encoders import CohereEncoder, OpenAIEncoder
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass(
"Enter OpenAI API Key: "
)
# encoder = CohereEncoder()
encoder = OpenAIEncoder()
rl = RouteLayer(encoder=encoder, routes=routes)
rl = SemanticRouter(encoder=encoder, routes=routes, auto_sync="local")
```
%% Output
2025-01-06 12:40:02 - semantic_router.utils.logger - WARNING - base.py:356 - _get_index() - No index provided. Using default LocalIndex.
2025-01-06 12:40:02 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:40:03 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:40:03 - semantic_router.utils.logger - WARNING - local.py:148 - _write_config() - No config is written for LocalIndex.
%% Cell type:markdown id: tags:
We run the solely static routes layer:
%% Cell type:code id: tags:
``` python
await rl.acall("how's the weather today?")
```
%% Output
2025-01-06 12:40:06 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags:
## Creating a Dynamic Route
%% Cell type:markdown id: tags:
As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide the `function_schemas` as a list. Each function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly.
%% Cell type:code id: tags:
``` python
from datetime import datetime
from zoneinfo import ZoneInfo
def get_time(timezone: str) -> str:
"""Finds the current time in a specific timezone.
:param timezone: The timezone to find the current time in, should
be a valid timezone from the IANA Time Zone Database like
"America/New_York" or "Europe/London". Do NOT put the place
name itself like "rome", or "new york", you must provide
the IANA format.
:type timezone: str
:return: The current time in the specified timezone."""
now = datetime.now(ZoneInfo(timezone))
return now.strftime("%H:%M")
```
%% Cell type:code id: tags:
``` python
get_time("America/New_York")
```
%% Output
'02:44'
'07:40'
%% Cell type:markdown id: tags:
To get the function schema we can use the `get_schema` function from the `function_call` module.
%% Cell type:code id: tags:
``` python
from semantic_router.llms.openai import get_schemas_openai
schemas = get_schemas_openai([get_time])
schemas
```
%% Output
[{'type': 'function',
'function': {'name': 'get_time',
'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.',
'parameters': {'type': 'object',
'properties': {'timezone': {'type': 'string',
'description': 'The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.'}},
'required': ['timezone']}}}]
%% Cell type:markdown id: tags:
We use this to define our dynamic route:
%% Cell type:code id: tags:
``` python
time_route = Route(
name="get_time",
utterances=[
"what is the time in new york city?",
"what is the time in london?",
"I live in Rome, what time is it?",
],
function_schemas=schemas,
)
```
%% Cell type:markdown id: tags:
Add the new route to our `layer`:
%% Cell type:code id: tags:
``` python
rl.add(time_route)
```
%% Output
2024-07-19 14:44:29 INFO semantic_router.utils.logger Adding `get_time` route
2025-01-06 12:40:16 - semantic_router.utils.logger - WARNING - base.py:172 - _read_config() - This method should be implemented by subclasses.
2025-01-06 12:40:16 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:40:16 - semantic_router.utils.logger - WARNING - local.py:148 - _write_config() - No config is written for LocalIndex.
%% Cell type:markdown id: tags:
Now we can ask our layer a time related question to trigger our new dynamic route.
%% Cell type:code id: tags:
``` python
response = await rl.acall("what is the time in new york city?")
response
```
%% Output
2024-07-19 14:44:32 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default
2024-07-19 14:44:34 INFO semantic_router.utils.logger OpenAI => Function Inputs: [{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}]
2025-01-06 12:40:18 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:40:18 - semantic_router.utils.logger - WARNING - base.py:488 - acall() - No LLM provided for dynamic route, will use OpenAI LLM default
2025-01-06 12:40:19 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
RouteChoice(name='get_time', function_call=[{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}], similarity_score=None)
%% Cell type:code id: tags:
``` python
for call in response.function_call:
if call["function_name"] == "get_time":
args = call["arguments"]
result = get_time(**args)
print(result)
```
%% Output
02:44
07:40
%% Cell type:markdown id: tags:
Our dynamic route provides both the route itself _and_ the input parameters required to use the route.
%% Cell type:markdown id: tags:
## Dynamic Routes with Multiple Functions
%% Cell type:markdown id: tags:
---
%% Cell type:markdown id: tags:
Routes can be assigned multiple functions. Then, when that particular Route is selected by the Route Layer, a number of those functions might be invoked due to the users utterance containing relevant information that fits their arguments.
%% Cell type:markdown id: tags:
Let's define a Route that has multiple functions.
%% Cell type:code id: tags:
``` python
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
# Function with one argument
def get_time(timezone: str) -> str:
"""Finds the current time in a specific timezone.
:param timezone: The timezone to find the current time in, should
be a valid timezone from the IANA Time Zone Database like
"America/New_York" or "Europe/London". Do NOT put the place
name itself like "rome", or "new york", you must provide
the IANA format.
:type timezone: str
:return: The current time in the specified timezone."""
now = datetime.now(ZoneInfo(timezone))
return now.strftime("%H:%M")
def get_time_difference(timezone1: str, timezone2: str) -> str:
"""Calculates the time difference between two timezones.
:param timezone1: The first timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".
:param timezone2: The second timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".
:type timezone1: str
:type timezone2: str
:return: The time difference in hours between the two timezones."""
# Get the current time in UTC
now_utc = datetime.utcnow().replace(tzinfo=ZoneInfo("UTC"))
# Convert the UTC time to the specified timezones
tz1_time = now_utc.astimezone(ZoneInfo(timezone1))
tz2_time = now_utc.astimezone(ZoneInfo(timezone2))
# Calculate the difference in offsets from UTC
tz1_offset = tz1_time.utcoffset().total_seconds()
tz2_offset = tz2_time.utcoffset().total_seconds()
# Calculate the difference in hours
hours_difference = (tz2_offset - tz1_offset) / 3600
return f"The time difference between {timezone1} and {timezone2} is {hours_difference} hours."
# Function with three arguments
def convert_time(time: str, from_timezone: str, to_timezone: str) -> str:
"""Converts a specific time from one timezone to another.
:param time: The time to convert in HH:MM format.
:param from_timezone: The original timezone of the time, should be a valid IANA timezone.
:param to_timezone: The target timezone for the time, should be a valid IANA timezone.
:type time: str
:type from_timezone: str
:type to_timezone: str
:return: The converted time in the target timezone.
:raises ValueError: If the time format or timezone strings are invalid.
Example:
convert_time("12:30", "America/New_York", "Asia/Tokyo") -> "03:30"
"""
try:
# Use today's date to avoid historical timezone issues
today = datetime.now().date()
datetime_string = f"{today} {time}"
time_obj = datetime.strptime(datetime_string, "%Y-%m-%d %H:%M").replace(
tzinfo=ZoneInfo(from_timezone)
)
converted_time = time_obj.astimezone(ZoneInfo(to_timezone))
formatted_time = converted_time.strftime("%H:%M")
return formatted_time
except Exception as e:
raise ValueError(f"Error converting time: {e}")
```
%% Cell type:code id: tags:
``` python
functions = [get_time, get_time_difference, convert_time]
```
%% Cell type:code id: tags:
``` python
# Generate schemas for all functions
from semantic_router.llms.openai import get_schemas_openai
schemas = get_schemas_openai(functions)
schemas
```
%% Output
[{'type': 'function',
'function': {'name': 'get_time',
'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.',
'parameters': {'type': 'object',
'properties': {'timezone': {'type': 'string',
'description': 'The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.'}},
'required': ['timezone']}}},
{'type': 'function',
'function': {'name': 'get_time_difference',
'description': 'Calculates the time difference between two timezones.\n:param timezone1: The first timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".\n:param timezone2: The second timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".\n:type timezone1: str\n:type timezone2: str\n:return: The time difference in hours between the two timezones.',
'parameters': {'type': 'object',
'properties': {'timezone1': {'type': 'string',
'description': 'The first timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".'},
'timezone2': {'type': 'string',
'description': 'The second timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".'}},
'required': ['timezone1', 'timezone2']}}},
{'type': 'function',
'function': {'name': 'convert_time',
'description': 'Converts a specific time from one timezone to another.\n:param time: The time to convert in HH:MM format.\n:param from_timezone: The original timezone of the time, should be a valid IANA timezone.\n:param to_timezone: The target timezone for the time, should be a valid IANA timezone.\n:type time: str\n:type from_timezone: str\n:type to_timezone: str\n:return: The converted time in the target timezone.\n:raises ValueError: If the time format or timezone strings are invalid.\n\nExample:\n convert_time("12:30", "America/New_York", "Asia/Tokyo") -> "03:30"',
'parameters': {'type': 'object',
'properties': {'time': {'type': 'string',
'description': 'The time to convert in HH:MM format.'},
'from_timezone': {'type': 'string',
'description': 'The original timezone of the time, should be a valid IANA timezone.'},
'to_timezone': {'type': 'string',
'description': 'The target timezone for the time, should be a valid IANA timezone.'}},
'required': ['time', 'from_timezone', 'to_timezone']}}}]
%% Cell type:code id: tags:
``` python
# Define the dynamic route with multiple functions
multi_function_route = Route(
name="timezone_management",
utterances=[
# Utterances for get_time function
"what is the time in New York?",
"current time in Berlin?",
"tell me the time in Moscow right now",
"can you show me the current time in Tokyo?",
"please provide the current time in London",
# Utterances for get_time_difference function
"how many hours ahead is Tokyo from London?",
"time difference between Sydney and Cairo",
"what's the time gap between Los Angeles and New York?",
"how much time difference is there between Paris and Sydney?",
"calculate the time difference between Dubai and Toronto",
# Utterances for convert_time function
"convert 15:00 from New York time to Berlin time",
"change 09:00 from Paris time to Moscow time",
"adjust 20:00 from Rome time to London time",
"convert 12:00 from Madrid time to Chicago time",
"change 18:00 from Beijing time to Los Angeles time"
# All three functions
"What is the time in Seattle? What is the time difference between Mumbai and Tokyo? What is 5:53 Toronto time in Sydney time?",
],
function_schemas=schemas,
)
```
%% Cell type:code id: tags:
``` python
routes = [politics, chitchat, multi_function_route]
```
%% Cell type:code id: tags:
``` python
rl2 = RouteLayer(encoder=encoder, routes=routes)
rl2 = SemanticRouter(encoder=encoder, routes=routes, auto_sync="local")
```
%% Output
2025-01-06 12:40:41 - semantic_router.utils.logger - WARNING - base.py:356 - _get_index() - No index provided. Using default LocalIndex.
2025-01-06 12:40:41 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:40:42 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:40:42 - semantic_router.utils.logger - WARNING - local.py:148 - _write_config() - No config is written for LocalIndex.
%% Cell type:markdown id: tags:
### Function to Parse Route Layer Responses
%% Cell type:code id: tags:
``` python
def parse_response(response: str):
for call in response.function_call:
args = call["arguments"]
if call["function_name"] == "get_time":
result = get_time(**args)
print(result)
if call["function_name"] == "get_time_difference":
result = get_time_difference(**args)
print(result)
if call["function_name"] == "convert_time":
result = convert_time(**args)
print(result)
```
%% Cell type:markdown id: tags:
### Checking that Politics Non-Dynamic Route Still Works
%% Cell type:code id: tags:
``` python
response = await rl2.acall("What is your political leaning?")
response
```
%% Output
2025-01-06 12:40:46 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
RouteChoice(name='politics', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags:
### Checking that Chitchat Non-Dynamic Route Still Works
%% Cell type:code id: tags:
``` python
response = await rl2.acall("Hello bot, how are you today?")
response
```
%% Output
2025-01-06 12:40:48 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags:
### Testing the `multi_function_route` - The `get_time` Function
%% Cell type:code id: tags:
``` python
response = await rl2.acall("what is the time in New York?")
response
```
%% Output
2024-07-19 14:45:02 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default
2024-07-19 14:45:03 INFO semantic_router.utils.logger OpenAI => Function Inputs: [{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}]
2025-01-06 12:40:49 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:40:49 - semantic_router.utils.logger - WARNING - base.py:488 - acall() - No LLM provided for dynamic route, will use OpenAI LLM default
2025-01-06 12:40:51 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
RouteChoice(name='timezone_management', function_call=[{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}], similarity_score=None)
%% Cell type:code id: tags:
``` python
parse_response(response)
```
%% Output
02:45
07:40
%% Cell type:markdown id: tags:
### Testing the `multi_function_route` - The `get_time_difference` Function
%% Cell type:code id: tags:
``` python
response = await rl2.acall(
"What is the time difference between Los Angeles and Istanbul?"
)
response
```
%% Output
2024-07-19 14:45:07 INFO semantic_router.utils.logger OpenAI => Function Inputs: [{'function_name': 'get_time_difference', 'arguments': {'timezone1': 'America/Los_Angeles', 'timezone2': 'Europe/Istanbul'}}]
2025-01-06 12:40:53 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:40:54 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
RouteChoice(name='timezone_management', function_call=[{'function_name': 'get_time_difference', 'arguments': {'timezone1': 'America/Los_Angeles', 'timezone2': 'Europe/Istanbul'}}], similarity_score=None)
%% Cell type:code id: tags:
``` python
parse_response(response)
```
%% Output
The time difference between America/Los_Angeles and Europe/Istanbul is 10.0 hours.
The time difference between America/Los_Angeles and Europe/Istanbul is 11.0 hours.
C:\Users\Joshu\AppData\Local\Temp\ipykernel_336\3683005204.py:28: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
now_utc = datetime.utcnow().replace(tzinfo=ZoneInfo("UTC"))
%% Cell type:markdown id: tags:
### Testing the `multi_function_route` - The `convert_time` Function
%% Cell type:code id: tags:
``` python
response = await rl2.acall(
"What is 23:02 Dubai time in Tokyo time? Please and thank you."
)
response
```
%% Output
2024-07-19 14:45:10 INFO semantic_router.utils.logger OpenAI => Function Inputs: [{'function_name': 'convert_time', 'arguments': {'time': '23:02', 'from_timezone': 'Asia/Dubai', 'to_timezone': 'Asia/Tokyo'}}]
2025-01-06 12:40:55 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:40:57 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
RouteChoice(name='timezone_management', function_call=[{'function_name': 'convert_time', 'arguments': {'time': '23:02', 'from_timezone': 'Asia/Dubai', 'to_timezone': 'Asia/Tokyo'}}], similarity_score=None)
%% Cell type:code id: tags:
``` python
parse_response(response)
```
%% Output
04:02
%% Cell type:markdown id: tags:
### The Cool Bit - Testing `multi_function_route` - Multiple Functions at Once
%% Cell type:code id: tags:
``` python
response = await rl2.acall(
"""
What is the time in Prague?
What is the time difference between Frankfurt and Beijing?
What is 5:53 Lisbon time in Bangkok time?
"""
)
```
%% Output
2024-07-19 14:45:15 INFO semantic_router.utils.logger OpenAI => Function Inputs: [{'function_name': 'get_time', 'arguments': {'timezone': 'Europe/Prague'}}, {'function_name': 'get_time_difference', 'arguments': {'timezone1': 'Europe/Berlin', 'timezone2': 'Asia/Shanghai'}}, {'function_name': 'convert_time', 'arguments': {'time': '05:53', 'from_timezone': 'Europe/Lisbon', 'to_timezone': 'Asia/Bangkok'}}]
2025-01-06 12:40:58 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-01-06 12:41:09 - httpx - INFO - _client.py:1740 - _send_single_request() - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
%% Cell type:code id: tags:
``` python
parse_response(response)
```
%% Output
08:45
The time difference between Europe/Berlin and Asia/Shanghai is 6.0 hours.
11:53
13:41
The time difference between Europe/Berlin and Asia/Shanghai is 7.0 hours.
12:53
C:\Users\Joshu\AppData\Local\Temp\ipykernel_336\3683005204.py:28: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
now_utc = datetime.utcnow().replace(tzinfo=ZoneInfo("UTC"))
......
docs/09-route-filter.ipynb
+ 33
− 28
View file @ 6b38f126
%% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/09-route-filter.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/00-introduction.ipynb)
%% Cell type:markdown id: tags:
# Semantic Router Filter
%% Cell type:markdown id: tags:
The Semantic Router library can be used as a super fast route making layer on top of LLMs. That means rather than waiting on a slow agent to decide what to do, we can use the magic of semantic vector space to make routes. Cutting route making time down from seconds to milliseconds.
%% Cell type:markdown id: tags:
## Getting Started
%% Cell type:markdown id: tags:
We start by installing the library:
%% Cell type:code id: tags:
``` python
!pip install -qU semantic-router
!pip install -qU "semantic-router==0.1.0.dev3"
```
%% Cell type:markdown id: tags:
We start by defining a dictionary mapping routes to example phrases that should trigger those routes.
%% Cell type:code id: tags:
``` python
from semantic_router import Route
politics = Route(
name="politics",
utterances=[
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"don't you just love the president",
"don't you just hate the president",
"they're going to destroy this country!",
"they will save the country!",
],
)
```
%% Output
c:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
%% Cell type:markdown id: tags:
Let's define another for good measure:
%% Cell type:code id: tags:
``` python
chitchat = Route(
name="chitchat",
utterances=[
"how's the weather today?",
"how are things going?",
"lovely weather today",
"the weather is horrendous",
"let's go to the chippy",
],
)
routes = [politics, chitchat]
```
%% Cell type:markdown id: tags:
Now we initialize our embedding model:
%% Cell type:code id: tags:
``` python
import os
from getpass import getpass
from semantic_router.encoders import CohereEncoder, OpenAIEncoder
os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass(
"Enter Cohere API Key: "
)
# os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass(
# "Enter OpenAI API Key: "
# )
encoder = CohereEncoder()
# encoder = OpenAIEncoder()
```
%% Cell type:markdown id: tags:
Now we define the `RouteLayer`. When called, the route layer will consume text (a query) and output the category (`Route`) it belongs to — to initialize a `RouteLayer` we need our `encoder` model and a list of `routes`.
%% Cell type:code id: tags:
``` python
from semantic_router.layer import RouteLayer
from semantic_router.routers import SemanticRouter
rl = RouteLayer(encoder=encoder, routes=routes)
rl = SemanticRouter(encoder=encoder, routes=routes, auto_sync="local")
```
%% Output
2024-05-07 16:02:43 INFO semantic_router.utils.logger local
2025-01-06 12:42:33 - semantic_router.utils.logger - WARNING - base.py:356 - _get_index() - No index provided. Using default LocalIndex.
2025-01-06 12:42:33 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:42:34 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:42:34 - semantic_router.utils.logger - WARNING - local.py:148 - _write_config() - No config is written for LocalIndex.
%% Cell type:markdown id: tags:
Now we can test it:
%% Cell type:code id: tags:
``` python
rl("don't you love politics?")
```
%% Output
2025-01-06 12:42:37 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:42:39 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:42:42 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:42:46 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
2025-01-06 12:42:48 - httpx - INFO - _client.py:1025 - _send_single_request() - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
RouteChoice(name='politics', function_call=None, similarity_score=None)
%% Cell type:code id: tags:
``` python
rl("how's the weather today?")
```
%% Output
RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags:
Both are classified accurately, what if we send a query that is unrelated to our existing `Route` objects?
%% Cell type:code id: tags:
``` python
rl("I'm interested in learning about llama 2")
```
%% Output
RouteChoice(name=None, function_call=None, similarity_score=None)
%% Cell type:markdown id: tags:
In this case, we return `None` because no matches were identified.
%% Cell type:markdown id: tags:
# Demonstrating the Filter Feature
Now, let's demonstrate the filter feature. We can specify a subset of routes to consider when making a classification. This can be useful if we want to restrict the scope of possible routes based on some context.
For example, let's say we only want to consider the "chitchat" route for a particular query:
%% Cell type:code id: tags:
``` python
rl("don't you love politics?", route_filter=["chitchat"])
```
%% Output
RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags:
Even though the query might be more related to the "politics" route, it will be classified as "chitchat" because we've restricted the routes to consider.
Similarly, we can restrict it to the "politics" route:
%% Cell type:code id: tags:
``` python
rl("how's the weather today?", route_filter=["politics"])
```
%% Output
RouteChoice(name=None, function_call=None, similarity_score=None)
%% Cell type:markdown id: tags:
In this case, it will return None because the query doesn't match the "politics" route well enough to pass the threshold.
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

:feet::heart: Strive to be the person your dogs believe you are :heart::feet: