Skip to content
Snippets Groups Projects
Unverified Commit 07658aa6 authored by Siraj R Aizlewood's avatar Siraj R Aizlewood
Browse files

Linting.

parent e926b5cc
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
docs/02-dynamic-routes.ipynb
+ 38
− 41
View file @ 07658aa6
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Dynamic Routes # Dynamic Routes
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In semantic-router there are two types of routes that can be chosen. Both routes belong to the `Route` object, the only difference between them is that _static_ routes return a `Route.name` when chosen, whereas _dynamic_ routes use an LLM call to produce parameter input values. In semantic-router there are two types of routes that can be chosen. Both routes belong to the `Route` object, the only difference between them is that _static_ routes return a `Route.name` when chosen, whereas _dynamic_ routes use an LLM call to produce parameter input values.
For example, a _static_ route will tell us if a query is talking about mathematics by returning the route name (which could be `"math"` for example). A _dynamic_ route can generate additional values, so it may decide a query is talking about maths, but it can also generate Python code that we can later execute to answer the user's query, this output may look like `"math", "import math; output = math.sqrt(64)`. For example, a _static_ route will tell us if a query is talking about mathematics by returning the route name (which could be `"math"` for example). A _dynamic_ route can generate additional values, so it may decide a query is talking about maths, but it can also generate Python code that we can later execute to answer the user's query, this output may look like `"math", "import math; output = math.sqrt(64)`.
***⚠️ Note: We have a fully local version of dynamic routes available at [docs/05-local-execution.ipynb](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb). The local 05 version tends to outperform the OpenAI version we demo in this notebook, so we'd recommend trying [05](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb)!*** ***⚠️ Note: We have a fully local version of dynamic routes available at [docs/05-local-execution.ipynb](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb). The local 05 version tends to outperform the OpenAI version we demo in this notebook, so we'd recommend trying [05](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb)!***
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Installing the Library ## Installing the Library
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
!pip install -qU semantic-router==0.0.34 !pip install -qU semantic-router==0.0.34
!pip install tzdata
``` ```
%% Output
[notice] A new release of pip is available: 23.1.2 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
Requirement already satisfied: tzdata in c:\users\siraj\documents\personal\work\aurelio\virtual environments\semantic_router\lib\site-packages (2024.1)
[notice] A new release of pip is available: 23.1.2 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Initializing Routes and RouteLayer ## Initializing Routes and RouteLayer
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes. Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router import Route from semantic_router import Route
politics = Route( politics = Route(
name="politics", name="politics",
utterances=[ utterances=[
"isn't politics the best thing ever", "isn't politics the best thing ever",
"why don't you tell me about your political opinions", "why don't you tell me about your political opinions",
"don't you just love the president" "don't you just hate the president", "don't you just love the president" "don't you just hate the president",
"they're going to destroy this country!", "they're going to destroy this country!",
"they will save the country!", "they will save the country!",
], ],
) )
chitchat = Route( chitchat = Route(
name="chitchat", name="chitchat",
utterances=[ utterances=[
"how's the weather today?", "how's the weather today?",
"how are things going?", "how are things going?",
"lovely weather today", "lovely weather today",
"the weather is horrendous", "the weather is horrendous",
"let's go to the chippy", "let's go to the chippy",
], ],
) )
routes = [politics, chitchat] routes = [politics, chitchat]
``` ```
%% Output %% Output
c:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html c:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm from .autonotebook import tqdm as notebook_tqdm
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`. We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import os import os
from getpass import getpass from getpass import getpass
from semantic_router import RouteLayer from semantic_router import RouteLayer
from semantic_router.encoders import CohereEncoder, OpenAIEncoder from semantic_router.encoders import CohereEncoder, OpenAIEncoder
# dashboard.cohere.ai # dashboard.cohere.ai
# os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass( # os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass(
# "Enter Cohere API Key: " # "Enter Cohere API Key: "
# ) # )
# platform.openai.com # platform.openai.com
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass( os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass(
"Enter OpenAI API Key: " "Enter OpenAI API Key: "
) )
# encoder = CohereEncoder() # encoder = CohereEncoder()
encoder = OpenAIEncoder() encoder = OpenAIEncoder()
rl = RouteLayer(encoder=encoder, routes=routes) rl = RouteLayer(encoder=encoder, routes=routes)
``` ```
%% Output %% Output
2024-04-27 02:19:43 INFO semantic_router.utils.logger local 2024-04-29 01:50:52 INFO semantic_router.utils.logger local
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We run the solely static routes layer: We run the solely static routes layer:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl("how's the weather today?") rl("how's the weather today?")
``` ```
%% Output %% Output
RouteChoice(name='chitchat', function_call=None, similarity_score=None) RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Creating a Dynamic Route ## Creating a Dynamic Route
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide a `function_schema`. The function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly. As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide a `function_schema`. The function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from datetime import datetime from datetime import datetime
from zoneinfo import ZoneInfo from zoneinfo import ZoneInfo
def get_time(timezone: str) -> str: def get_time(timezone: str) -> str:
"""Finds the current time in a specific timezone. """Finds the current time in a specific timezone.
:param timezone: The timezone to find the current time in, should :param timezone: The timezone to find the current time in, should
be a valid timezone from the IANA Time Zone Database like be a valid timezone from the IANA Time Zone Database like
"America/New_York" or "Europe/London". Do NOT put the place "America/New_York" or "Europe/London". Do NOT put the place
name itself like "rome", or "new york", you must provide name itself like "rome", or "new york", you must provide
the IANA format. the IANA format.
:type timezone: str :type timezone: str
:return: The current time in the specified timezone.""" :return: The current time in the specified timezone."""
now = datetime.now(ZoneInfo(timezone)) now = datetime.now(ZoneInfo(timezone))
return now.strftime("%H:%M") return now.strftime("%H:%M")
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
get_time("America/New_York") get_time("America/New_York")
``` ```
%% Output %% Output
'18:19' '17:50'
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To get the function schema we can use the `get_schema` function from the `function_call` module. To get the function schema we can use the `get_schema` function from the `function_call` module.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router.utils.function_call import get_schema from semantic_router.utils.function_call import get_schema
schema = get_schema(get_time) schema = get_schema(get_time)
schema schema
``` ```
%% Output %% Output
{'name': 'get_time', {'name': 'get_time',
'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.', 'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.',
'signature': '(timezone: str) -> str', 'signature': '(timezone: str) -> str',
'output': "<class 'str'>"} 'output': "<class 'str'>"}
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We use this to define our dynamic route: We use this to define our dynamic route:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
time_route = Route( time_route = Route(
name="get_time", name="get_time",
utterances=[ utterances=[
"what is the time in new york city?", "what is the time in new york city?",
"what is the time in london?", "what is the time in london?",
"I live in Rome, what time is it?", "I live in Rome, what time is it?",
], ],
function_schema=schema, function_schema=schema,
) )
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
time_route.llm time_route.llm
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Add the new route to our `layer`: Add the new route to our `layer`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl.add(time_route) rl.add(time_route)
``` ```
%% Output %% Output
2024-04-27 02:19:45 INFO semantic_router.utils.logger Adding `get_time` route 2024-04-29 01:50:53 INFO semantic_router.utils.logger Adding `get_time` route
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
time_route.llm time_route.llm
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now we can ask our layer a time related question to trigger our new dynamic route. Now we can ask our layer a time related question to trigger our new dynamic route.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
out = rl("what is the time in new york city?") out = rl("what is the time in new york city?")
get_time(**out.function_call) get_time(**out.function_call)
``` ```
%% Output %% Output
2024-04-27 02:19:45 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable. 2024-04-29 01:50:54 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable.
2024-04-27 02:19:45 INFO semantic_router.utils.logger Extracting function input... 2024-04-29 01:50:54 INFO semantic_router.utils.logger Extracting function input...
2024-04-29 01:50:55 INFO semantic_router.utils.logger LLM output: {
##################################################
tools
None
##################################################
2024-04-27 02:19:46 INFO semantic_router.utils.logger LLM output: {
"timezone": "America/New_York" "timezone": "America/New_York"
} }
2024-04-27 02:19:46 INFO semantic_router.utils.logger Function inputs: {'timezone': 'America/New_York'} 2024-04-29 01:50:55 INFO semantic_router.utils.logger Function inputs: {'timezone': 'America/New_York'}
##################################################
completion.choices[0].message.tool_calls
None
##################################################
##################################################
extracted_inputs
{'timezone': 'America/New_York'}
##################################################
'18:19' '17:50'
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
time_route.llm time_route.llm
``` ```
%% Output %% Output
OpenAILLM(name='gpt-3.5-turbo', client=<openai.OpenAI object at 0x00000152CAD11ED0>, temperature=0.01, max_tokens=200) OpenAILLM(name='gpt-3.5-turbo', client=<openai.OpenAI object at 0x00000129AFF58190>, temperature=0.01, max_tokens=200)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Our dynamic route provides both the route itself _and_ the input parameters required to use the route. Our dynamic route provides both the route itself _and_ the input parameters required to use the route.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
--- ---
......
docs/10-dynamic-routes-via-openai-function-calling.ipynb
+ 39
− 29
View file @ 07658aa6
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Dynamic Routes via OpenAI Function Calling # Dynamic Routes via OpenAI Function Calling
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In this Notebook we showcase an alternative to dynamic routes where an `openai_function_schema` is specified or the Route, and in doing so, OpenAIs "Function Calling" API is utilized. In this Notebook we showcase an alternative to dynamic routes where an `openai_function_schema` is specified or the Route, and in doing so, OpenAIs "Function Calling" API is utilized.
The usual Semantic Router procedure is used to determine which Route is most appropriate for the input utterance. Then, rather than using a regular LLM call with careful prompt engineering to allow the LLM to extract function arguments from the input, OpenAIs Function Calling is leveraged to try and obtain function arguments more reliably. The usual Semantic Router procedure is used to determine which Route is most appropriate for the input utterance. Then, rather than using a regular LLM call with careful prompt engineering to allow the LLM to extract function arguments from the input, OpenAIs Function Calling is leveraged to try and obtain function arguments more reliably.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Installing the Library ## Installing the Library
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
!pip install -qU semantic-router==0.0.34 !pip install -qU semantic-router==0.0.34
``` ```
%% Output
[notice] A new release of pip is available: 23.1.2 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Initializing Routes and RouteLayer ## Initializing Routes and RouteLayer
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes. Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router import Route from semantic_router import Route
politics = Route( politics = Route(
name="politics", name="politics",
utterances=[ utterances=[
"isn't politics the best thing ever", "isn't politics the best thing ever",
"why don't you tell me about your political opinions", "why don't you tell me about your political opinions",
"don't you just love the president" "don't you just hate the president", "don't you just love the president" "don't you just hate the president",
"they're going to destroy this country!", "they're going to destroy this country!",
"they will save the country!", "they will save the country!",
], ],
) )
chitchat = Route( chitchat = Route(
name="chitchat", name="chitchat",
utterances=[ utterances=[
"how's the weather today?", "how's the weather today?",
"how are things going?", "how are things going?",
"lovely weather today", "lovely weather today",
"the weather is horrendous", "the weather is horrendous",
"let's go to the chippy", "let's go to the chippy",
], ],
) )
routes = [politics, chitchat] routes = [politics, chitchat]
``` ```
%% Output %% Output
c:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html c:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_2\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm from .autonotebook import tqdm as notebook_tqdm
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`. We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import os import os
from getpass import getpass from getpass import getpass
from semantic_router import RouteLayer from semantic_router import RouteLayer
from semantic_router.encoders import CohereEncoder, OpenAIEncoder from semantic_router.encoders import CohereEncoder, OpenAIEncoder
# dashboard.cohere.ai # dashboard.cohere.ai
# os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass( # os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass(
# "Enter Cohere API Key: " # "Enter Cohere API Key: "
# ) # )
# platform.openai.com # platform.openai.com
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass( os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass(
"Enter OpenAI API Key: " "Enter OpenAI API Key: "
) )
# encoder = CohereEncoder() # encoder = CohereEncoder()
encoder = OpenAIEncoder() encoder = OpenAIEncoder()
rl = RouteLayer(encoder=encoder, routes=routes) rl = RouteLayer(encoder=encoder, routes=routes)
``` ```
%% Output %% Output
2024-04-28 22:54:05 INFO semantic_router.utils.logger local 2024-04-29 01:48:49 INFO semantic_router.utils.logger local
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We run the solely static routes layer: We run the solely static routes layer:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl("how's the weather today?") rl("how's the weather today?")
``` ```
%% Output %% Output
RouteChoice(name='chitchat', function_call=None, similarity_score=None) RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Creating a Dynamic Route ## Creating a Dynamic Route
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide a `function_schema`. The function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly. As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide a `function_schema`. The function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from datetime import datetime from datetime import datetime
from zoneinfo import ZoneInfo from zoneinfo import ZoneInfo
def get_time(timezone: str) -> str: def get_time(timezone: str) -> str:
"""Finds the current time in a specific timezone. """Finds the current time in a specific timezone.
:param timezone: The timezone to find the current time in, should :param timezone: The timezone to find the current time in, should
be a valid timezone from the IANA Time Zone Database like be a valid timezone from the IANA Time Zone Database like
"America/New_York" or "Europe/London". Do NOT put the place "America/New_York" or "Europe/London". Do NOT put the place
name itself like "rome", or "new york", you must provide name itself like "rome", or "new york", you must provide
the IANA format. the IANA format.
:type timezone: str :type timezone: str
:return: The current time in the specified timezone.""" :return: The current time in the specified timezone."""
now = datetime.now(ZoneInfo(timezone)) now = datetime.now(ZoneInfo(timezone))
return now.strftime("%H:%M") return now.strftime("%H:%M")
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
get_time("America/New_York") get_time("America/New_York")
``` ```
%% Output %% Output
'14:54' '17:48'
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To get the function schema we can use the `get_schema` function from the `function_call` module. To get the function schema we can use the `get_schema` function from the `function_call` module.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router.utils.function_call import get_schema from semantic_router.utils.function_call import get_schema
schema = get_schema(get_time) schema = get_schema(get_time)
schema schema
``` ```
%% Output %% Output
{'name': 'get_time', {'name': 'get_time',
'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.', 'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.',
'signature': '(timezone: str) -> str', 'signature': '(timezone: str) -> str',
'output': "<class 'str'>"} 'output': "<class 'str'>"}
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router.utils.function_call import get_schema_openai_func_calling from semantic_router.utils.function_call import get_schema_openai_func_calling
openai_function_schema = get_schema_openai_func_calling(get_time) openai_function_schema = get_schema_openai_func_calling(get_time)
openai_function_schema openai_function_schema
``` ```
%% Output %% Output
{'type': 'function', {'type': 'function',
'function': {'name': 'get_time', 'function': {'name': 'get_time',
'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.', 'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.',
'parameters': {'type': 'object', 'parameters': {'type': 'object',
'properties': {'timezone': {'type': 'string', 'properties': {'timezone': {'type': 'string',
'description': 'The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.'}}, 'description': 'The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.'}},
'required': ['timezone']}}} 'required': ['timezone']}}}
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We use this to define our dynamic route: We use this to define our dynamic route:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
time_route = Route( time_route = Route(
name="get_time", name="get_time",
utterances=[ utterances=[
"what is the time in new york city?", "what is the time in new york city?",
"what is the time in london?", "what is the time in london?",
"I live in Rome, what time is it?", "I live in Rome, what time is it?",
], ],
# function_schema=schema, # function_schema=schema,
openai_function_schema=openai_function_schema, openai_function_schema=openai_function_schema,
) )
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Add the new route to our `layer`: Add the new route to our `layer`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl.add(time_route) rl.add(time_route)
``` ```
%% Output %% Output
2024-04-28 22:54:06 INFO semantic_router.utils.logger Adding `get_time` route 2024-04-29 01:48:50 INFO semantic_router.utils.logger Adding `get_time` route
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now we can ask our layer a time related question to trigger our new dynamic route. Now we can ask our layer a time related question to trigger our new dynamic route.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
out = rl("what is the time in new york city?") out = rl("what is the time in new york city?")
out out
``` ```
%% Output %% Output
2024-04-28 22:54:07 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable. 2024-04-29 01:48:52 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable.
RouteChoice(name='get_time', function_call={'timezone': 'America/New_York'}, similarity_score=None) RouteChoice(name='get_time', function_call={'timezone': 'America/New_York'}, similarity_score=None)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
get_time(**out.function_call) get_time(**out.function_call)
``` ```
%% Output %% Output
'14:54' '17:48'
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Our dynamic route provides both the route itself _and_ the input parameters required to use the route. Our dynamic route provides both the route itself _and_ the input parameters required to use the route.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now let's just verify that the other route selection still work correctly, starting with chitchat. Now let's just verify that the other route selection still work correctly, starting with chitchat.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
out = rl("What's the weather like?") out = rl("What's the weather like?")
out out
``` ```
%% Output %% Output
RouteChoice(name='chitchat', function_call=None, similarity_score=None) RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now, an utterance that shouldn't fall into any of the routes defined above. Now, an utterance that shouldn't fall into any of the routes defined above.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
out = rl("Tell me how to bake cake?") out = rl("Tell me how to bake cake?")
out out
``` ```
%% Output %% Output
RouteChoice(name=None, function_call=None, similarity_score=None) RouteChoice(name=None, function_call=None, similarity_score=None)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
--- ---
......
semantic_router/llms/openai.py
+ 11
− 18
View file @ 07658aa6
import os import os
from typing import Any, List, Optional from typing import List, Optional, Any
import openai import openai
...@@ -8,6 +8,7 @@ from semantic_router.schema import Message ...@@ -8,6 +8,7 @@ from semantic_router.schema import Message
from semantic_router.utils.defaults import EncoderDefault from semantic_router.utils.defaults import EncoderDefault
from semantic_router.utils.logger import logger from semantic_router.utils.logger import logger
import json import json
from openai.types.chat import ChatCompletionMessageToolCall
class OpenAILLM(BaseLLM): class OpenAILLM(BaseLLM):
client: Optional[openai.OpenAI] client: Optional[openai.OpenAI]
...@@ -36,12 +37,12 @@ class OpenAILLM(BaseLLM): ...@@ -36,12 +37,12 @@ class OpenAILLM(BaseLLM):
self.temperature = temperature self.temperature = temperature
self.max_tokens = max_tokens self.max_tokens = max_tokens
def __call__(self, messages: List[Message], function_schema: dict = None) -> str: def __call__(self, messages: List[Message], openai_function_schema: Optional[dict[str, Any]] = None) -> str:
if self.client is None: if self.client is None:
raise ValueError("OpenAI client is not initialized.") raise ValueError("OpenAI client is not initialized.")
try: try:
if function_schema: if openai_function_schema:
tools = [function_schema] tools = [openai_function_schema]
else: else:
tools = None tools = None
completion = self.client.chat.completions.create( completion = self.client.chat.completions.create(
...@@ -49,34 +50,27 @@ class OpenAILLM(BaseLLM): ...@@ -49,34 +50,27 @@ class OpenAILLM(BaseLLM):
messages=[m.to_openai() for m in messages], messages=[m.to_openai() for m in messages],
temperature=self.temperature, temperature=self.temperature,
max_tokens=self.max_tokens, max_tokens=self.max_tokens,
tools=tools, tools=tools, # type: ignore # MyPy expecting Iterable[ChatCompletionToolParam] | NotGiven, but dict is accepted by OpenAI.
) )
output = completion.choices[0].message.content output = completion.choices[0].message.content
if function_schema: if openai_function_schema:
return completion.choices[0].message.tool_calls return completion.choices[0].message.tool_calls
# tool_calls = completion.choices[0].message.tool_calls
# if not tool_calls:
# raise Exception("No tool calls available in the completion response.")
# tool_call = tool_calls[0]
# arguments_json = tool_call.function.arguments
# arguments_dict = json.loads(arguments_json)
# return arguments_dict
if not output: if not output:
raise Exception("No output generated") raise Exception("No output generated")
return output return output
except Exception as e: except Exception as e:
logger.error(f"LLM error: {e}") logger.error(f"LLM error: {e}")
raise Exception(f"LLM error: {e}") from e raise Exception(f"LLM error: {e}") from e
#
def extract_function_inputs_openai(self, query: str, function_schema: dict) -> dict:
def extract_function_inputs_openai(self, query: str, openai_function_schema: dict[str, Any]) -> dict:
messages = [] messages = []
system_prompt = "You are an intelligent AI. Given a command or request from the user, call the function to complete the request." system_prompt = "You are an intelligent AI. Given a command or request from the user, call the function to complete the request."
messages.append(Message(role="system", content=system_prompt)) messages.append(Message(role="system", content=system_prompt))
messages.append(Message(role="user", content=query)) messages.append(Message(role="user", content=query))
output = self(messages=messages, function_schema=function_schema) output = self(messages=messages, openai_function_schema=openai_function_schema)
if not output: if not output:
raise Exception("No output generated for extract function input") raise Exception("No output generated for extract function input")
if len(output) != 1: if len(output) != 1:
...@@ -85,4 +79,3 @@ class OpenAILLM(BaseLLM): ...@@ -85,4 +79,3 @@ class OpenAILLM(BaseLLM):
arguments_json = tool_call.function.arguments arguments_json = tool_call.function.arguments
function_inputs = json.loads(arguments_json) function_inputs = json.loads(arguments_json)
return function_inputs return function_inputs
\ No newline at end of file
semantic_router/route.py
+ 17
− 13
View file @ 07658aa6
...@@ -67,23 +67,27 @@ class Route(BaseModel): ...@@ -67,23 +67,27 @@ class Route(BaseModel):
"LLM is required for dynamic routes. Please ensure the `llm` " "LLM is required for dynamic routes. Please ensure the `llm` "
"attribute is set." "attribute is set."
) )
elif query is None: if query is None or type(query) != str:
raise ValueError( raise ValueError(
"Query is required for dynamic routes. Please ensure the `query` " "Query is required for dynamic routes. Please ensure the `query` "
"argument is passed." "argument is passed."
) )
if self.function_schema: if self.function_schema:
extracted_inputs = self.llm.extract_function_inputs( extracted_inputs = self.llm.extract_function_inputs(
query=query, function_schema=self.function_schema query=query,
) function_schema=self.function_schema
func_call = extracted_inputs )
elif self.openai_function_schema: func_call = extracted_inputs
if not isinstance(self.llm, OpenAILLM): elif self.openai_function_schema: # Logically must be self.openai_function_schema, but keeps MyPy happy.
raise TypeError("LLM must be an instance of OpenAILLM for openai_function_schema.") if not isinstance(self.llm, OpenAILLM):
extracted_inputs = self.llm.extract_function_inputs_openai( raise TypeError(
query=query, function_schema=self.openai_function_schema "LLM must be an instance of OpenAILLM for openai_function_schema."
) )
func_call = extracted_inputs extracted_inputs = self.llm.extract_function_inputs_openai(
query=query,
openai_function_schema=self.openai_function_schema
)
func_call = extracted_inputs
else: else:
# otherwise we just pass None for the call # otherwise we just pass None for the call
func_call = None func_call = None
......
semantic_router/utils/function_call.py
+ 19
− 15
View file @ 07658aa6
...@@ -8,6 +8,7 @@ from semantic_router.schema import Message, RouteChoice ...@@ -8,6 +8,7 @@ from semantic_router.schema import Message, RouteChoice
from semantic_router.utils.logger import logger from semantic_router.utils.logger import logger
import re import re
def get_schema(item: Union[BaseModel, Callable]) -> Dict[str, Any]: def get_schema(item: Union[BaseModel, Callable]) -> Dict[str, Any]:
if isinstance(item, BaseModel): if isinstance(item, BaseModel):
signature_parts = [] signature_parts = []
...@@ -39,6 +40,7 @@ def get_schema(item: Union[BaseModel, Callable]) -> Dict[str, Any]: ...@@ -39,6 +40,7 @@ def get_schema(item: Union[BaseModel, Callable]) -> Dict[str, Any]:
} }
return schema return schema
def convert_param_type_to_json_type(param_type: str) -> str: def convert_param_type_to_json_type(param_type: str) -> str:
if param_type == "int": if param_type == "int":
return "number" return "number"
...@@ -55,48 +57,50 @@ def convert_param_type_to_json_type(param_type: str) -> str: ...@@ -55,48 +57,50 @@ def convert_param_type_to_json_type(param_type: str) -> str:
else: else:
return "object" return "object"
def get_schema_openai_func_calling(item: Callable) -> Dict[str, Any]: def get_schema_openai_func_calling(item: Callable) -> Dict[str, Any]:
if not callable(item): if not callable(item):
raise ValueError("Provided item must be a callable function.") raise ValueError("Provided item must be a callable function.")
docstring = inspect.getdoc(item) docstring = inspect.getdoc(item)
signature = inspect.signature(item) signature = inspect.signature(item)
schema = { schema = {
"type": "function", "type": "function",
"function": { "function": {
"name": item.__name__, "name": item.__name__,
"description": docstring if docstring else "No description available.", "description": docstring if docstring else "No description available.",
"parameters": { "parameters": {"type": "object", "properties": {}, "required": []},
"type": "object", },
"properties": {},
"required": []
}
}
} }
for param_name, param in signature.parameters.items(): for param_name, param in signature.parameters.items():
param_type = param.annotation.__name__ if param.annotation != inspect.Parameter.empty else "Any" param_type = (
param.annotation.__name__
if param.annotation != inspect.Parameter.empty
else "Any"
)
param_description = "No description available." param_description = "No description available."
param_required = param.default is inspect.Parameter.empty param_required = param.default is inspect.Parameter.empty
# Attempt to extract the parameter description from the docstring # Attempt to extract the parameter description from the docstring
if docstring: if docstring:
param_doc_regex = re.compile(rf":param {param_name}:(.*?)\n(?=:\w|$)", re.S) param_doc_regex = re.compile(rf":param {param_name}:(.*?)\n(?=:\w|$)", re.S)
match = param_doc_regex.search(docstring) match = param_doc_regex.search(docstring)
if match: if match:
param_description = match.group(1).strip() param_description = match.group(1).strip()
schema["function"]["parameters"]["properties"][param_name] = { schema["function"]["parameters"]["properties"][param_name] = {
"type": convert_param_type_to_json_type(param_type), "type": convert_param_type_to_json_type(param_type),
"description": param_description "description": param_description,
} }
if param_required: if param_required:
schema["function"]["parameters"]["required"].append(param_name) schema["function"]["parameters"]["required"].append(param_name)
return schema return schema
# TODO: Add route layer object to the input, solve circular import issue # TODO: Add route layer object to the input, solve circular import issue
async def route_and_execute( async def route_and_execute(
query: str, llm: BaseLLM, functions: List[Callable], layer query: str, llm: BaseLLM, functions: List[Callable], layer
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

:feet::heart: Strive to be the person your dogs believe you are :heart::feet: