Skip to content
Snippets Groups Projects
Unverified Commit 8d6d9b53 authored by Siraj R Aizlewood's avatar Siraj R Aizlewood
Browse files

Fixing pytests.

parent 0760d913
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
docs/02-dynamic-routes.ipynb
+ 14
− 26
View file @ 8d6d9b53
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Dynamic Routes # Dynamic Routes
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In semantic-router there are two types of routes that can be chosen. Both routes belong to the `Route` object, the only difference between them is that _static_ routes return a `Route.name` when chosen, whereas _dynamic_ routes use an LLM call to produce parameter input values. In semantic-router there are two types of routes that can be chosen. Both routes belong to the `Route` object, the only difference between them is that _static_ routes return a `Route.name` when chosen, whereas _dynamic_ routes use an LLM call to produce parameter input values.
For example, a _static_ route will tell us if a query is talking about mathematics by returning the route name (which could be `"math"` for example). A _dynamic_ route does the same thing, but it also extracts key information from the input utterance to be used in a function associated with that route. For example, a _static_ route will tell us if a query is talking about mathematics by returning the route name (which could be `"math"` for example). A _dynamic_ route does the same thing, but it also extracts key information from the input utterance to be used in a function associated with that route.
For example we could provide a dynamic route with associated utterances: For example we could provide a dynamic route with associated utterances:
``` ```
"what is x to the power of y?" "what is x to the power of y?"
"what is 9 to the power of 4?" "what is 9 to the power of 4?"
"calculate the result of base x and exponent y" "calculate the result of base x and exponent y"
"calculate the result of base 10 and exponent 3" "calculate the result of base 10 and exponent 3"
"return x to the power of y" "return x to the power of y"
``` ```
and we could also provide the route with a schema outlining key features of the function: and we could also provide the route with a schema outlining key features of the function:
``` ```
def power(base: float, exponent: float) -> float: def power(base: float, exponent: float) -> float:
"""Raise base to the power of exponent. """Raise base to the power of exponent.
Args: Args:
base (float): The base number. base (float): The base number.
exponent (float): The exponent to which the base is raised. exponent (float): The exponent to which the base is raised.
Returns: Returns:
float: The result of base raised to the power of exponent. float: The result of base raised to the power of exponent.
""" """
return base ** exponent return base ** exponent
``` ```
Then, if the users input utterance is "What is 2 to the power of 3?", the route will be triggered, as the input utterance is semantically similar to the route utterances. Furthermore, the route utilizes an LLM to identify that `base=2` and `expoenent=3`. These values are returned in such a way that they can be used in the above `power` function. That is, the dynamic router automates the process of calling relevant functions from natural language inputs. Then, if the users input utterance is "What is 2 to the power of 3?", the route will be triggered, as the input utterance is semantically similar to the route utterances. Furthermore, the route utilizes an LLM to identify that `base=2` and `expoenent=3`. These values are returned in such a way that they can be used in the above `power` function. That is, the dynamic router automates the process of calling relevant functions from natural language inputs.
***⚠️ Note: We have a fully local version of dynamic routes available at [docs/05-local-execution.ipynb](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb). The local 05 version tends to outperform the OpenAI version we demo in this notebook, so we'd recommend trying [05](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb)!*** ***⚠️ Note: We have a fully local version of dynamic routes available at [docs/05-local-execution.ipynb](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb). The local 05 version tends to outperform the OpenAI version we demo in this notebook, so we'd recommend trying [05](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb)!***
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Installing the Library ## Installing the Library
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
!pip install tzdata !pip install tzdata
!pip install -qU semantic-router !pip install -qU semantic-router
``` ```
%% Output %% Output
Requirement already satisfied: tzdata in c:\users\siraj\documents\personal\work\aurelio\virtual environments\semantic_router_3\lib\site-packages (2024.1) Requirement already satisfied: tzdata in c:\users\siraj\documents\personal\work\aurelio\virtual environments\semantic_router_3\lib\site-packages (2024.1)
WARNING: Ignoring invalid distribution ~ (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~illow (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~rotobuf (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~ (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~illow (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~rotobuf (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~ (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~illow (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~rotobuf (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~ (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~illow (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
WARNING: Ignoring invalid distribution ~rotobuf (C:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages)
[notice] A new release of pip is available: 23.1.2 -> 24.0 [notice] A new release of pip is available: 23.1.2 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip [notice] To update, run: python.exe -m pip install --upgrade pip
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Initializing Routes and RouteLayer ## Initializing Routes and RouteLayer
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes. Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router import Route from semantic_router import Route
politics = Route( politics = Route(
name="politics", name="politics",
utterances=[ utterances=[
"isn't politics the best thing ever", "isn't politics the best thing ever",
"why don't you tell me about your political opinions", "why don't you tell me about your political opinions",
"don't you just love the president" "don't you just hate the president", "don't you just love the president" "don't you just hate the president",
"they're going to destroy this country!", "they're going to destroy this country!",
"they will save the country!", "they will save the country!",
], ],
) )
chitchat = Route( chitchat = Route(
name="chitchat", name="chitchat",
utterances=[ utterances=[
"how's the weather today?", "how's the weather today?",
"how are things going?", "how are things going?",
"lovely weather today", "lovely weather today",
"the weather is horrendous", "the weather is horrendous",
"let's go to the chippy", "let's go to the chippy",
], ],
) )
routes = [politics, chitchat] routes = [politics, chitchat]
``` ```
%% Output %% Output
c:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html c:\Users\Siraj\Documents\Personal\Work\Aurelio\Virtual Environments\semantic_router_3\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm from .autonotebook import tqdm as notebook_tqdm
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`. We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import os import os
from getpass import getpass from getpass import getpass
from semantic_router import RouteLayer from semantic_router import RouteLayer
from semantic_router.encoders import CohereEncoder, OpenAIEncoder from semantic_router.encoders import CohereEncoder, OpenAIEncoder
# dashboard.cohere.ai # dashboard.cohere.ai
# os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass( # os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass(
# "Enter Cohere API Key: " # "Enter Cohere API Key: "
# ) # )
# platform.openai.com # platform.openai.com
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass( os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass(
"Enter OpenAI API Key: " "Enter OpenAI API Key: "
) )
# encoder = CohereEncoder() # encoder = CohereEncoder()
encoder = OpenAIEncoder() encoder = OpenAIEncoder()
rl = RouteLayer(encoder=encoder, routes=routes) rl = RouteLayer(encoder=encoder, routes=routes)
``` ```
%% Output %% Output
2024-05-07 15:12:16 INFO semantic_router.utils.logger local 2024-05-08 01:57:55 INFO semantic_router.utils.logger local
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We run the solely static routes layer: We run the solely static routes layer:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl("how's the weather today?") rl("how's the weather today?")
``` ```
%% Output %% Output
RouteChoice(name='chitchat', function_call=None, similarity_score=None) RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Creating a Dynamic Route ## Creating a Dynamic Route
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide the `function_schemas` as a list. Each function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly. As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide the `function_schemas` as a list. Each function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from datetime import datetime from datetime import datetime
from zoneinfo import ZoneInfo from zoneinfo import ZoneInfo
def get_time(timezone: str) -> str: def get_time(timezone: str) -> str:
"""Finds the current time in a specific timezone. """Finds the current time in a specific timezone.
:param timezone: The timezone to find the current time in, should :param timezone: The timezone to find the current time in, should
be a valid timezone from the IANA Time Zone Database like be a valid timezone from the IANA Time Zone Database like
"America/New_York" or "Europe/London". Do NOT put the place "America/New_York" or "Europe/London". Do NOT put the place
name itself like "rome", or "new york", you must provide name itself like "rome", or "new york", you must provide
the IANA format. the IANA format.
:type timezone: str :type timezone: str
:return: The current time in the specified timezone.""" :return: The current time in the specified timezone."""
now = datetime.now(ZoneInfo(timezone)) now = datetime.now(ZoneInfo(timezone))
return now.strftime("%H:%M") return now.strftime("%H:%M")
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
get_time("America/New_York") get_time("America/New_York")
``` ```
%% Output %% Output
'07:12' '17:57'
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To get the function schema we can use the `get_schema` function from the `function_call` module. To get the function schema we can use the `get_schema` function from the `function_call` module.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router.llms.openai import get_schemas_openai from semantic_router.llms.openai import get_schemas_openai
schemas = get_schemas_openai([get_time]) schemas = get_schemas_openai([get_time])
schemas schemas
``` ```
%% Output %% Output
[{'type': 'function', [{'type': 'function',
'function': {'name': 'get_time', 'function': {'name': 'get_time',
'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.', 'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.',
'parameters': {'type': 'object', 'parameters': {'type': 'object',
'properties': {'timezone': {'type': 'string', 'properties': {'timezone': {'type': 'string',
'description': 'The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.'}}, 'description': 'The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.'}},
'required': ['timezone']}}}] 'required': ['timezone']}}}]
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We use this to define our dynamic route: We use this to define our dynamic route:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
time_route = Route( time_route = Route(
name="get_time", name="get_time",
utterances=[ utterances=[
"what is the time in new york city?", "what is the time in new york city?",
"what is the time in london?", "what is the time in london?",
"I live in Rome, what time is it?", "I live in Rome, what time is it?",
], ],
function_schemas=schemas, function_schemas=schemas,
) )
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
time_route.llm time_route.llm
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Add the new route to our `layer`: Add the new route to our `layer`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl.add(time_route) rl.add(time_route)
``` ```
%% Output %% Output
2024-05-07 15:12:18 INFO semantic_router.utils.logger Adding `get_time` route 2024-05-08 01:57:56 INFO semantic_router.utils.logger Adding `get_time` route
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
time_route.llm time_route.llm
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now we can ask our layer a time related question to trigger our new dynamic route. Now we can ask our layer a time related question to trigger our new dynamic route.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
response = rl("what is the time in new york city?") response = rl("what is the time in new york city?")
response response
``` ```
%% Output %% Output
2024-05-07 15:12:19 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable. 2024-05-08 01:57:57 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable.
2024-05-07 15:12:20 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}] 2024-05-08 01:57:58 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}]
RouteChoice(name='get_time', function_call=[{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}], similarity_score=None) RouteChoice(name='get_time', function_call=[{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}], similarity_score=None)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
print(response.function_call) print(response.function_call)
``` ```
%% Output %% Output
[{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}] [{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}]
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import json import json
for call in response.function_call: for call in response.function_call:
if call['function_name'] == 'get_time': if call['function_name'] == 'get_time':
args = call['arguments'] args = call['arguments']
result = get_time(**args) result = get_time(**args)
print(result) print(result)
``` ```
%% Output %% Output
07:12 17:57
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Our dynamic route provides both the route itself _and_ the input parameters required to use the route. Our dynamic route provides both the route itself _and_ the input parameters required to use the route.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Dynamic Routes with Multiple Functions ## Dynamic Routes with Multiple Functions
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
--- ---
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Routes can be assigned multiple functions. Then, when that particular Route is selected by the Route Layer, a number of those functions might be invoked due to the users utterance containing relevant information that fits their arguments. Routes can be assigned multiple functions. Then, when that particular Route is selected by the Route Layer, a number of those functions might be invoked due to the users utterance containing relevant information that fits their arguments.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Let's define a Route that has multiple functions. Let's define a Route that has multiple functions.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from datetime import datetime, timedelta from datetime import datetime, timedelta
from zoneinfo import ZoneInfo from zoneinfo import ZoneInfo
# Function with one argument # Function with one argument
def get_time(timezone: str) -> str: def get_time(timezone: str) -> str:
"""Finds the current time in a specific timezone. """Finds the current time in a specific timezone.
:param timezone: The timezone to find the current time in, should :param timezone: The timezone to find the current time in, should
be a valid timezone from the IANA Time Zone Database like be a valid timezone from the IANA Time Zone Database like
"America/New_York" or "Europe/London". Do NOT put the place "America/New_York" or "Europe/London". Do NOT put the place
name itself like "rome", or "new york", you must provide name itself like "rome", or "new york", you must provide
the IANA format. the IANA format.
:type timezone: str :type timezone: str
:return: The current time in the specified timezone.""" :return: The current time in the specified timezone."""
now = datetime.now(ZoneInfo(timezone)) now = datetime.now(ZoneInfo(timezone))
return now.strftime("%H:%M") return now.strftime("%H:%M")
def get_time_difference(timezone1: str, timezone2: str) -> str: def get_time_difference(timezone1: str, timezone2: str) -> str:
"""Calculates the time difference between two timezones. """Calculates the time difference between two timezones.
:param timezone1: The first timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London". :param timezone1: The first timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".
:param timezone2: The second timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London". :param timezone2: The second timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".
:type timezone1: str :type timezone1: str
:type timezone2: str :type timezone2: str
:return: The time difference in hours between the two timezones.""" :return: The time difference in hours between the two timezones."""
# Get the current time in UTC # Get the current time in UTC
now_utc = datetime.utcnow().replace(tzinfo=ZoneInfo('UTC')) now_utc = datetime.utcnow().replace(tzinfo=ZoneInfo('UTC'))
# Convert the UTC time to the specified timezones # Convert the UTC time to the specified timezones
tz1_time = now_utc.astimezone(ZoneInfo(timezone1)) tz1_time = now_utc.astimezone(ZoneInfo(timezone1))
tz2_time = now_utc.astimezone(ZoneInfo(timezone2)) tz2_time = now_utc.astimezone(ZoneInfo(timezone2))
# Calculate the difference in offsets from UTC # Calculate the difference in offsets from UTC
tz1_offset = tz1_time.utcoffset().total_seconds() tz1_offset = tz1_time.utcoffset().total_seconds()
tz2_offset = tz2_time.utcoffset().total_seconds() tz2_offset = tz2_time.utcoffset().total_seconds()
# Calculate the difference in hours # Calculate the difference in hours
hours_difference = (tz2_offset - tz1_offset) / 3600 hours_difference = (tz2_offset - tz1_offset) / 3600
return f"The time difference between {timezone1} and {timezone2} is {hours_difference} hours." return f"The time difference between {timezone1} and {timezone2} is {hours_difference} hours."
# Function with three arguments # Function with three arguments
def convert_time(time: str, from_timezone: str, to_timezone: str) -> str: def convert_time(time: str, from_timezone: str, to_timezone: str) -> str:
"""Converts a specific time from one timezone to another. """Converts a specific time from one timezone to another.
:param time: The time to convert in HH:MM format. :param time: The time to convert in HH:MM format.
:param from_timezone: The original timezone of the time, should be a valid IANA timezone. :param from_timezone: The original timezone of the time, should be a valid IANA timezone.
:param to_timezone: The target timezone for the time, should be a valid IANA timezone. :param to_timezone: The target timezone for the time, should be a valid IANA timezone.
:type time: str :type time: str
:type from_timezone: str :type from_timezone: str
:type to_timezone: str :type to_timezone: str
:return: The converted time in the target timezone. :return: The converted time in the target timezone.
:raises ValueError: If the time format or timezone strings are invalid. :raises ValueError: If the time format or timezone strings are invalid.
Example: Example:
convert_time("12:30", "America/New_York", "Asia/Tokyo") -> "03:30" convert_time("12:30", "America/New_York", "Asia/Tokyo") -> "03:30"
""" """
try: try:
# Use today's date to avoid historical timezone issues # Use today's date to avoid historical timezone issues
today = datetime.now().date() today = datetime.now().date()
datetime_string = f"{today} {time}" datetime_string = f"{today} {time}"
time_obj = datetime.strptime(datetime_string, "%Y-%m-%d %H:%M").replace(tzinfo=ZoneInfo(from_timezone)) time_obj = datetime.strptime(datetime_string, "%Y-%m-%d %H:%M").replace(tzinfo=ZoneInfo(from_timezone))
converted_time = time_obj.astimezone(ZoneInfo(to_timezone)) converted_time = time_obj.astimezone(ZoneInfo(to_timezone))
formatted_time = converted_time.strftime("%H:%M") formatted_time = converted_time.strftime("%H:%M")
return formatted_time return formatted_time
except Exception as e: except Exception as e:
raise ValueError(f"Error converting time: {e}") raise ValueError(f"Error converting time: {e}")
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
functions = [get_time, get_time_difference, convert_time] functions = [get_time, get_time_difference, convert_time]
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# Generate schemas for all functions # Generate schemas for all functions
from semantic_router.llms.openai import get_schemas_openai from semantic_router.llms.openai import get_schemas_openai
schemas = get_schemas_openai(functions) schemas = get_schemas_openai(functions)
schemas schemas
``` ```
%% Output %% Output
[{'type': 'function', [{'type': 'function',
'function': {'name': 'get_time', 'function': {'name': 'get_time',
'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.', 'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.',
'parameters': {'type': 'object', 'parameters': {'type': 'object',
'properties': {'timezone': {'type': 'string', 'properties': {'timezone': {'type': 'string',
'description': 'The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.'}}, 'description': 'The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.'}},
'required': ['timezone']}}}, 'required': ['timezone']}}},
{'type': 'function', {'type': 'function',
'function': {'name': 'get_time_difference', 'function': {'name': 'get_time_difference',
'description': 'Calculates the time difference between two timezones.\n:param timezone1: The first timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".\n:param timezone2: The second timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".\n:type timezone1: str\n:type timezone2: str\n:return: The time difference in hours between the two timezones.', 'description': 'Calculates the time difference between two timezones.\n:param timezone1: The first timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".\n:param timezone2: The second timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".\n:type timezone1: str\n:type timezone2: str\n:return: The time difference in hours between the two timezones.',
'parameters': {'type': 'object', 'parameters': {'type': 'object',
'properties': {'timezone1': {'type': 'string', 'properties': {'timezone1': {'type': 'string',
'description': 'The first timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".'}, 'description': 'The first timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".'},
'timezone2': {'type': 'string', 'timezone2': {'type': 'string',
'description': 'The second timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".'}}, 'description': 'The second timezone, should be a valid timezone from the IANA Time Zone Database like "America/New_York" or "Europe/London".'}},
'required': ['timezone1', 'timezone2']}}}, 'required': ['timezone1', 'timezone2']}}},
{'type': 'function', {'type': 'function',
'function': {'name': 'convert_time', 'function': {'name': 'convert_time',
'description': 'Converts a specific time from one timezone to another.\n:param time: The time to convert in HH:MM format.\n:param from_timezone: The original timezone of the time, should be a valid IANA timezone.\n:param to_timezone: The target timezone for the time, should be a valid IANA timezone.\n:type time: str\n:type from_timezone: str\n:type to_timezone: str\n:return: The converted time in the target timezone.\n:raises ValueError: If the time format or timezone strings are invalid.\n\nExample:\n convert_time("12:30", "America/New_York", "Asia/Tokyo") -> "03:30"', 'description': 'Converts a specific time from one timezone to another.\n:param time: The time to convert in HH:MM format.\n:param from_timezone: The original timezone of the time, should be a valid IANA timezone.\n:param to_timezone: The target timezone for the time, should be a valid IANA timezone.\n:type time: str\n:type from_timezone: str\n:type to_timezone: str\n:return: The converted time in the target timezone.\n:raises ValueError: If the time format or timezone strings are invalid.\n\nExample:\n convert_time("12:30", "America/New_York", "Asia/Tokyo") -> "03:30"',
'parameters': {'type': 'object', 'parameters': {'type': 'object',
'properties': {'time': {'type': 'string', 'properties': {'time': {'type': 'string',
'description': 'The time to convert in HH:MM format.'}, 'description': 'The time to convert in HH:MM format.'},
'from_timezone': {'type': 'string', 'from_timezone': {'type': 'string',
'description': 'The original timezone of the time, should be a valid IANA timezone.'}, 'description': 'The original timezone of the time, should be a valid IANA timezone.'},
'to_timezone': {'type': 'string', 'to_timezone': {'type': 'string',
'description': 'The target timezone for the time, should be a valid IANA timezone.'}}, 'description': 'The target timezone for the time, should be a valid IANA timezone.'}},
'required': ['time', 'from_timezone', 'to_timezone']}}}] 'required': ['time', 'from_timezone', 'to_timezone']}}}]
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# Define the dynamic route with multiple functions # Define the dynamic route with multiple functions
multi_function_route = Route( multi_function_route = Route(
name="timezone_management", name="timezone_management",
utterances=[ utterances=[
# Utterances for get_time function # Utterances for get_time function
"what is the time in New York?", "what is the time in New York?",
"current time in Berlin?", "current time in Berlin?",
"tell me the time in Moscow right now", "tell me the time in Moscow right now",
"can you show me the current time in Tokyo?", "can you show me the current time in Tokyo?",
"please provide the current time in London", "please provide the current time in London",
# Utterances for get_time_difference function # Utterances for get_time_difference function
"how many hours ahead is Tokyo from London?", "how many hours ahead is Tokyo from London?",
"time difference between Sydney and Cairo", "time difference between Sydney and Cairo",
"what's the time gap between Los Angeles and New York?", "what's the time gap between Los Angeles and New York?",
"how much time difference is there between Paris and Sydney?", "how much time difference is there between Paris and Sydney?",
"calculate the time difference between Dubai and Toronto", "calculate the time difference between Dubai and Toronto",
# Utterances for convert_time function # Utterances for convert_time function
"convert 15:00 from New York time to Berlin time", "convert 15:00 from New York time to Berlin time",
"change 09:00 from Paris time to Moscow time", "change 09:00 from Paris time to Moscow time",
"adjust 20:00 from Rome time to London time", "adjust 20:00 from Rome time to London time",
"convert 12:00 from Madrid time to Chicago time", "convert 12:00 from Madrid time to Chicago time",
"change 18:00 from Beijing time to Los Angeles time" "change 18:00 from Beijing time to Los Angeles time"
# All three functions # All three functions
"What is the time in Seattle? What is the time difference between Mumbai and Tokyo? What is 5:53 Toronto time in Sydney time?" "What is the time in Seattle? What is the time difference between Mumbai and Tokyo? What is 5:53 Toronto time in Sydney time?"
], ],
function_schemas=schemas function_schemas=schemas
) )
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
routes = [politics, chitchat, multi_function_route] routes = [politics, chitchat, multi_function_route]
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl2 = RouteLayer(encoder=encoder, routes=routes) rl2 = RouteLayer(encoder=encoder, routes=routes)
``` ```
%% Output %% Output
2024-05-07 15:12:20 INFO semantic_router.utils.logger local 2024-05-08 01:57:58 INFO semantic_router.utils.logger local
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Function to Parse Route Layer Responses ### Function to Parse Route Layer Responses
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
def parse_response(response: str): def parse_response(response: str):
for call in response.function_call: for call in response.function_call:
args = call['arguments'] args = call['arguments']
if call['function_name'] == 'get_time': if call['function_name'] == 'get_time':
result = get_time(**args) result = get_time(**args)
print(result) print(result)
if call['function_name'] == 'get_time_difference': if call['function_name'] == 'get_time_difference':
result = get_time_difference(**args) result = get_time_difference(**args)
print(result) print(result)
if call['function_name'] == 'convert_time': if call['function_name'] == 'convert_time':
result = convert_time(**args) result = convert_time(**args)
print(result) print(result)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Checking that Politics Non-Dynamic Route Still Works ### Checking that Politics Non-Dynamic Route Still Works
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
response = rl2("What is your political leaning?") response = rl2("What is your political leaning?")
response response
``` ```
%% Output %% Output
RouteChoice(name='politics', function_call=None, similarity_score=None) RouteChoice(name='politics', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Checking that Chitchat Non-Dynamic Route Still Works ### Checking that Chitchat Non-Dynamic Route Still Works
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
response = rl2("Hello bot, how are you today?") response = rl2("Hello bot, how are you today?")
response response
``` ```
%% Output %% Output
RouteChoice(name='chitchat', function_call=None, similarity_score=None) RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Testing the `multi_function_route` - The `get_time` Function ### Testing the `multi_function_route` - The `get_time` Function
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
response = rl2("what is the time in New York?") response = rl2("what is the time in New York?")
response response
``` ```
%% Output %% Output
2024-05-07 15:12:22 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable. 2024-05-08 01:58:00 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable.
2024-05-07 15:12:23 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}] 2024-05-08 01:58:01 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}]
RouteChoice(name='timezone_management', function_call=[{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}], similarity_score=None) RouteChoice(name='timezone_management', function_call=[{'function_name': 'get_time', 'arguments': {'timezone': 'America/New_York'}}], similarity_score=None)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
parse_response(response) parse_response(response)
``` ```
%% Output %% Output
07:12 17:58
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Testing the `multi_function_route` - The `get_time_difference` Function ### Testing the `multi_function_route` - The `get_time_difference` Function
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
response = rl2("What is the time difference between Los Angeles and Istanbul?") response = rl2("What is the time difference between Los Angeles and Istanbul?")
response response
``` ```
%% Output %% Output
2024-05-07 15:12:24 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'get_time_difference', 'arguments': {'timezone1': 'America/Los_Angeles', 'timezone2': 'Europe/Istanbul'}}] 2024-05-08 01:58:02 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'get_time_difference', 'arguments': {'timezone1': 'America/Los_Angeles', 'timezone2': 'Europe/Istanbul'}}]
RouteChoice(name='timezone_management', function_call=[{'function_name': 'get_time_difference', 'arguments': {'timezone1': 'America/Los_Angeles', 'timezone2': 'Europe/Istanbul'}}], similarity_score=None) RouteChoice(name='timezone_management', function_call=[{'function_name': 'get_time_difference', 'arguments': {'timezone1': 'America/Los_Angeles', 'timezone2': 'Europe/Istanbul'}}], similarity_score=None)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
parse_response(response) parse_response(response)
``` ```
%% Output %% Output
The time difference between America/Los_Angeles and Europe/Istanbul is 10.0 hours. The time difference between America/Los_Angeles and Europe/Istanbul is 10.0 hours.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Testing the `multi_function_route` - The `convert_time` Function ### Testing the `multi_function_route` - The `convert_time` Function
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
response = rl2("What is 23:02 Dubai time in Tokyo time? Please and thank you.") response = rl2("What is 23:02 Dubai time in Tokyo time? Please and thank you.")
response response
``` ```
%% Output %% Output
2024-05-07 15:12:26 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'convert_time', 'arguments': {'time': '23:02', 'from_timezone': 'Asia/Dubai', 'to_timezone': 'Asia/Tokyo'}}] 2024-05-08 01:58:04 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'convert_time', 'arguments': {'time': '23:02', 'from_timezone': 'Asia/Dubai', 'to_timezone': 'Asia/Tokyo'}}]
RouteChoice(name='timezone_management', function_call=[{'function_name': 'convert_time', 'arguments': {'time': '23:02', 'from_timezone': 'Asia/Dubai', 'to_timezone': 'Asia/Tokyo'}}], similarity_score=None) RouteChoice(name='timezone_management', function_call=[{'function_name': 'convert_time', 'arguments': {'time': '23:02', 'from_timezone': 'Asia/Dubai', 'to_timezone': 'Asia/Tokyo'}}], similarity_score=None)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
parse_response(response) parse_response(response)
``` ```
%% Output %% Output
04:02 04:02
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### The Cool Bit - Testing `multi_function_route` - Multiple Functions at Once ### The Cool Bit - Testing `multi_function_route` - Multiple Functions at Once
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
response = rl2(""" response = rl2("""
What is the time in Prague? What is the time in Prague?
What is the time difference between Frankfurt and Beijing? What is the time difference between Frankfurt and Beijing?
What is 5:53 Lisbon time in Bangkok time? What is 5:53 Lisbon time in Bangkok time?
""" """
) )
``` ```
%% Output %% Output
2024-05-07 15:12:29 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'get_time', 'arguments': {'timezone': 'Europe/Prague'}}, {'function_name': 'get_time_difference', 'arguments': {'timezone1': 'Europe/Berlin', 'timezone2': 'Asia/Shanghai'}}, {'function_name': 'convert_time', 'arguments': {'time': '05:53', 'from_timezone': 'Europe/Lisbon', 'to_timezone': 'Asia/Bangkok'}}] 2024-05-08 01:58:07 INFO semantic_router.utils.logger Function inputs: [{'function_name': 'get_time', 'arguments': {'timezone': 'Europe/Prague'}}, {'function_name': 'get_time_difference', 'arguments': {'timezone1': 'Europe/Berlin', 'timezone2': 'Asia/Shanghai'}}, {'function_name': 'convert_time', 'arguments': {'time': '05:53', 'from_timezone': 'Europe/Lisbon', 'to_timezone': 'Asia/Bangkok'}}]
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
response response
``` ```
%% Output %% Output
RouteChoice(name='timezone_management', function_call=[{'function_name': 'get_time', 'arguments': {'timezone': 'Europe/Prague'}}, {'function_name': 'get_time_difference', 'arguments': {'timezone1': 'Europe/Berlin', 'timezone2': 'Asia/Shanghai'}}, {'function_name': 'convert_time', 'arguments': {'time': '05:53', 'from_timezone': 'Europe/Lisbon', 'to_timezone': 'Asia/Bangkok'}}], similarity_score=None) RouteChoice(name='timezone_management', function_call=[{'function_name': 'get_time', 'arguments': {'timezone': 'Europe/Prague'}}, {'function_name': 'get_time_difference', 'arguments': {'timezone1': 'Europe/Berlin', 'timezone2': 'Asia/Shanghai'}}, {'function_name': 'convert_time', 'arguments': {'time': '05:53', 'from_timezone': 'Europe/Lisbon', 'to_timezone': 'Asia/Bangkok'}}], similarity_score=None)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
parse_response(response) parse_response(response)
``` ```
%% Output %% Output
13:12 23:58
The time difference between Europe/Berlin and Asia/Shanghai is 6.0 hours. The time difference between Europe/Berlin and Asia/Shanghai is 6.0 hours.
11:53 11:53
......
semantic_router/llms/base.py
+ 4
− 4
View file @ 8d6d9b53
...@@ -84,9 +84,9 @@ Your task is to output JSON representing the input arguments of a Python functio ...@@ -84,9 +84,9 @@ Your task is to output JSON representing the input arguments of a Python functio
This is the Python function's schema: This is the Python function's schema:
### FUNCTION_SCHEMA Start ### ### FUNCTION_SCHEMAS Start ###
{function_schemas} {function_schemas}
### FUNCTION_SCHEMA End ### ### FUNCTION_SCHEMAS End ###
This is the input query. This is the input query.
...@@ -94,9 +94,9 @@ This is the input query. ...@@ -94,9 +94,9 @@ This is the input query.
{query} {query}
### QUERY End ### ### QUERY End ###
The arguments that you need to provide values for, together with their datatypes, are stated in "signature" in the FUNCTION_SCHEMA. The arguments that you need to provide values for, together with their datatypes, are stated in "signature" in the FUNCTION_SCHEMAS.
The values these arguments must take are made clear by the QUERY. The values these arguments must take are made clear by the QUERY.
Use the FUNCTION_SCHEMA "description" too, as this might provide helpful clues about the arguments and their values. Use the FUNCTION_SCHEMAS "description" too, as this might provide helpful clues about the arguments and their values.
Return only JSON, stating the argument names and their corresponding values. Return only JSON, stating the argument names and their corresponding values.
### FORMATTING_INSTRUCTIONS Start ### ### FORMATTING_INSTRUCTIONS Start ###
......
semantic_router/llms/openai.py
+ 1
− 1
View file @ 8d6d9b53
...@@ -96,7 +96,7 @@ class OpenAILLM(BaseLLM): ...@@ -96,7 +96,7 @@ class OpenAILLM(BaseLLM):
# Collecting multiple tool calls information # Collecting multiple tool calls information
output = str( output = str(
self._extract_tool_calls_info(tool_calls) self._extract_tool_calls_info(tool_calls)
) # str in keepign with base type. ) # str in keeping with base type.
else: else:
content = completion.choices[0].message.content content = completion.choices[0].message.content
if content is None: if content is None:
......
tests/unit/llms/test_llm_base.py
+ 4
− 4
View file @ 8d6d9b53
...@@ -16,15 +16,15 @@ class TestBaseLLM: ...@@ -16,15 +16,15 @@ class TestBaseLLM:
base_llm("test") base_llm("test")
def test_base_llm_is_valid_inputs_valid_input_pass(self, base_llm): def test_base_llm_is_valid_inputs_valid_input_pass(self, base_llm):
test_schema = { test_schemas = [{
"name": "get_time", "name": "get_time",
"description": 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.', "description": 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London". Do NOT put the place\n name itself like "rome", or "new york", you must provide\n the IANA format.\n:type timezone: str\n:return: The current time in the specified timezone.',
"signature": "(timezone: str) -> str", "signature": "(timezone: str) -> str",
"output": "<class 'str'>", "output": "<class 'str'>",
} }]
test_inputs = {"timezone": "America/New_York"} test_inputs = [{"timezone": "America/New_York"}]
assert base_llm._is_valid_inputs(test_inputs, test_schema) is True assert base_llm._is_valid_inputs(test_inputs, test_schemas) is True
@pytest.mark.skip(reason="TODO: bug in is_valid_inputs") @pytest.mark.skip(reason="TODO: bug in is_valid_inputs")
def test_base_llm_is_valid_inputs_valid_input_fail(self, base_llm): def test_base_llm_is_valid_inputs_valid_input_fail(self, base_llm):
......
tests/unit/llms/test_llm_llamacpp.py
+ 2
− 2
View file @ 8d6d9b53
...@@ -61,7 +61,7 @@ class TestLlamaCppLLM: ...@@ -61,7 +61,7 @@ class TestLlamaCppLLM:
test_query = "What time is it in America/New_York?" test_query = "What time is it in America/New_York?"
llamacpp_llm.extract_function_inputs( llamacpp_llm.extract_function_inputs(
query=test_query, function_schema=test_schema query=test_query, function_schemas=[test_schema]
) )
def test_llamacpp_extract_function_inputs_invalid(self, llamacpp_llm, mocker): def test_llamacpp_extract_function_inputs_invalid(self, llamacpp_llm, mocker):
...@@ -82,5 +82,5 @@ class TestLlamaCppLLM: ...@@ -82,5 +82,5 @@ class TestLlamaCppLLM:
test_query = "What time is it in America/New_York?" test_query = "What time is it in America/New_York?"
llamacpp_llm.extract_function_inputs( llamacpp_llm.extract_function_inputs(
query=test_query, function_schema=test_schema query=test_query, function_schemas=[test_schema]
) )
tests/unit/llms/test_llm_openai.py
+ 65
− 50
View file @ 8d6d9b53
...@@ -60,27 +60,29 @@ class TestOpenAILLM: ...@@ -60,27 +60,29 @@ class TestOpenAILLM:
"""Sample function for testing.""" """Sample function for testing."""
return f"param1: {param1}, param2: {param2}" return f"param1: {param1}, param2: {param2}"
expected_schema = { expected_schema = [
"type": "function", {
"function": { "type": "function",
"name": "sample_function", "function": {
"description": "Sample function for testing.", "name": "sample_function",
"parameters": { "description": "Sample function for testing.",
"type": "object", "parameters": {
"properties": { "type": "object",
"param1": { "properties": {
"type": "number", "param1": {
"description": "No description available.", "type": "number",
}, "description": "No description available.",
"param2": { },
"type": "string", "param2": {
"description": "No description available.", "type": "string",
"description": "No description available.",
},
}, },
"required": ["param1"],
}, },
"required": ["param1"],
}, },
}, }
} ]
schema = get_schemas_openai([sample_function]) schema = get_schemas_openai([sample_function])
assert schema == expected_schema, "Schema did not match expected output." assert schema == expected_schema, "Schema did not match expected output."
...@@ -89,19 +91,38 @@ class TestOpenAILLM: ...@@ -89,19 +91,38 @@ class TestOpenAILLM:
with pytest.raises(ValueError): with pytest.raises(ValueError):
get_schemas_openai([non_callable]) get_schemas_openai([non_callable])
# def test_openai_llm_call_with_function_schema(self, openai_llm, mocker):
# mock_completion = mocker.MagicMock()
# mock_completion.choices[0].message.tool_calls = [
# mocker.MagicMock(function=mocker.MagicMock(arguments="result"))
# ]
# mocker.patch.object(
# openai_llm.client.chat.completions, "create", return_value=mock_completion
# )
# llm_input = [Message(role="user", content="test")]
# function_schemas = [{"type": "function", "name": "sample_function"}]
# output = openai_llm(llm_input, function_schemas)
# assert (
# output == "result"
# ), "Output did not match expected result with function schema"
def test_openai_llm_call_with_function_schema(self, openai_llm, mocker): def test_openai_llm_call_with_function_schema(self, openai_llm, mocker):
# Mocking the tool call with valid JSON arguments and setting the function name explicitly
mock_function = mocker.MagicMock(arguments='{"timezone":"America/New_York"}')
mock_function.name = "sample_function" # Set the function name explicitly here
mock_tool_call = mocker.MagicMock(function=mock_function)
mock_completion = mocker.MagicMock() mock_completion = mocker.MagicMock()
mock_completion.choices[0].message.tool_calls = [ mock_completion.choices[0].message.tool_calls = [mock_tool_call]
mocker.MagicMock(function=mocker.MagicMock(arguments="result"))
]
mocker.patch.object( mocker.patch.object(
openai_llm.client.chat.completions, "create", return_value=mock_completion openai_llm.client.chat.completions, "create", return_value=mock_completion
) )
llm_input = [Message(role="user", content="test")] llm_input = [Message(role="user", content="test")]
function_schemas = [{"type": "function", "name": "sample_function"}] function_schemas = [{"type": "function", "name": "sample_function"}]
output = openai_llm(llm_input, function_schemas) output = openai_llm(llm_input, function_schemas)
assert ( assert (
output == "result" output == "[{'function_name': 'sample_function', 'arguments': {'timezone': 'America/New_York'}}]"
), "Output did not match expected result with function schema" ), "Output did not match expected result with function schema"
def test_openai_llm_call_with_invalid_tool_calls(self, openai_llm, mocker): def test_openai_llm_call_with_invalid_tool_calls(self, openai_llm, mocker):
...@@ -137,42 +158,38 @@ class TestOpenAILLM: ...@@ -137,42 +158,38 @@ class TestOpenAILLM:
openai_llm(llm_input, function_schemas) openai_llm(llm_input, function_schemas)
expected_error_message = ( expected_error_message = (
"LLM error: Invalid output, expected arguments to be specified." "LLM error: Invalid output, expected arguments to be specified for each tool call."
) )
actual_error_message = str(exc_info.value) actual_error_message = str(exc_info.value)
assert ( assert (
expected_error_message in actual_error_message expected_error_message in actual_error_message
), f"Expected error message: '{expected_error_message}', but got: '{actual_error_message}'" ), f"Expected error message: '{expected_error_message}', but got: '{actual_error_message}'"
def test_openai_llm_call_with_multiple_tools_specified(self, openai_llm, mocker):
mock_completion = mocker.MagicMock()
mock_completion.choices[0].message.tool_calls = [
mocker.MagicMock(),
mocker.MagicMock(),
]
mocker.patch.object(
openai_llm.client.chat.completions, "create", return_value=mock_completion
)
llm_input = [Message(role="user", content="test")]
function_schemas = [{"type": "function", "name": "sample_function"}]
with pytest.raises(Exception) as exc_info:
openai_llm(llm_input, function_schemas)
expected_error_message = (
"LLM error: Invalid output, expected a single tool to be specified."
)
actual_error_message = str(exc_info.value)
assert (
expected_error_message in actual_error_message
), f"Expected error message: '{expected_error_message}', but got: '{actual_error_message}'"
def test_extract_function_inputs(self, openai_llm, mocker): def test_extract_function_inputs(self, openai_llm, mocker):
query = "fetch user data" query = "fetch user data"
function_schemas = [{"function": "get_user_data", "args": ["user_id"]}] function_schemas = [
{
"type": "function",
"function": {
"name": "get_user_data",
"description": "Function to fetch user data.",
"parameters": {
"type": "object",
"properties": {
"user_id": {
"type": "string",
"description": "The ID of the user."
}
},
"required": ["user_id"]
}
}
}
]
# Mock the __call__ method to return a JSON string as expected # Mock the __call__ method to return a JSON string as expected
mocker.patch.object(OpenAILLM, "__call__", return_value='{"user_id": "123"}') mocker.patch.object(OpenAILLM, "__call__", return_value='[{"function_name": "get_user_data", "arguments": {"user_id": "123"}}]')
result = openai_llm.extract_function_inputs(query, function_schemas) result = openai_llm.extract_function_inputs(query, function_schemas)
# Ensure the __call__ method is called with the correct parameters # Ensure the __call__ method is called with the correct parameters
...@@ -188,6 +205,4 @@ class TestOpenAILLM: ...@@ -188,6 +205,4 @@ class TestOpenAILLM:
) )
# Check if the result is as expected # Check if the result is as expected
assert result == { assert result == [{"function_name": "get_user_data", "arguments": {"user_id": "123"}}], "The function inputs should match the expected dictionary."
"user_id": "123" \ No newline at end of file
}, "The function inputs should match the expected dictionary."
tests/unit/test_route.py
+ 8
− 8
View file @ 8d6d9b53
...@@ -63,12 +63,12 @@ class MockLLM(BaseLLM): ...@@ -63,12 +63,12 @@ class MockLLM(BaseLLM):
class TestRoute: class TestRoute:
def test_value_error_in_route_call(self): def test_value_error_in_route_call(self):
function_schema = {"name": "test_function", "type": "function"} function_schemas = [{"name": "test_function", "type": "function"}]
route = Route( route = Route(
name="test_function", name="test_function",
utterances=["utterance1", "utterance2"], utterances=["utterance1", "utterance2"],
function_schema=function_schema, function_schemas=function_schemas,
) )
with pytest.raises(ValueError): with pytest.raises(ValueError):
...@@ -76,9 +76,9 @@ class TestRoute: ...@@ -76,9 +76,9 @@ class TestRoute:
def test_generate_dynamic_route(self): def test_generate_dynamic_route(self):
mock_llm = MockLLM(name="test") mock_llm = MockLLM(name="test")
function_schema = {"name": "test_function", "type": "function"} function_schemas = {"name": "test_function", "type": "function"}#
route = Route._generate_dynamic_route( route = Route._generate_dynamic_route(
llm=mock_llm, function_schema=function_schema llm=mock_llm, function_schemas=function_schemas, route_name="test_route"
) )
assert route.name == "test_function" assert route.name == "test_function"
assert route.utterances == [ assert route.utterances == [
...@@ -107,8 +107,8 @@ class TestRoute: ...@@ -107,8 +107,8 @@ class TestRoute:
# } # }
# </config> # </config>
# """ # """
# function_schema = {"name": "test_function", "type": "function"} # function_schemas = [{"name": "test_function", "type": "function"}]
# route = await Route._generate_dynamic_route(function_schema) # route = await Route._generate_dynamic_route(function_schemas)
# assert route.name == "test_function" # assert route.name == "test_function"
# assert route.utterances == [ # assert route.utterances == [
# "example_utterance_1", # "example_utterance_1",
...@@ -124,7 +124,7 @@ class TestRoute: ...@@ -124,7 +124,7 @@ class TestRoute:
"name": "test", "name": "test",
"utterances": ["utterance"], "utterances": ["utterance"],
"description": None, "description": None,
"function_schema": None, "function_schemas": None,
"llm": None, "llm": None,
"score_threshold": None, "score_threshold": None,
} }
...@@ -144,7 +144,7 @@ class TestRoute: ...@@ -144,7 +144,7 @@ class TestRoute:
"""Test function docstring""" """Test function docstring"""
pass pass
dynamic_route = Route.from_dynamic_route(llm=mock_llm, entity=test_function) dynamic_route = Route.from_dynamic_route(llm=mock_llm, entities=[test_function], route_name="test_route")
assert dynamic_route.name == "test_function" assert dynamic_route.name == "test_function"
assert dynamic_route.utterances == [ assert dynamic_route.utterances == [
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

:feet::heart: Strive to be the person your dogs believe you are :heart::feet: