{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "UxqB7_Ieur0s"
},
"source": [
"[](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EduhQaNAur0u"
},
"source": [
"# Dynamic Routes via OpenAI Function Calling"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_4JgNeX4ur0v"
},
"source": [
"In this Notebook we showcase an alternative to dynamic routes where an `openai_function_schema` is specified or the Route, and in doing so, OpenAIs \"Function Calling\" API is utilized. \n",
"\n",
"The usual Semantic Router procedure is used to determine which Route is most appropriate for the input utterance. Then, rather than using a regular LLM call with careful prompt engineering to allow the LLM to extract function arguments from the input, OpenAIs Function Calling is leveraged to try and obtain function arguments more reliably. \n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bbmw8CO4ur0v"
},
"source": [
"## Installing the Library"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "dLElfRhgur0v"
},
"outputs": [],
"source": [
"!pip install -qU semantic-router==0.0.34"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BixZd6Eour0w"
},
"source": [
"## Initializing Routes and RouteLayer"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PxnW9qBvur0x"
},
"source": [
"Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "kc9Ty6Lgur0x"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"c:\\Users\\Siraj\\Documents\\Personal\\Work\\Aurelio\\Virtual Environments\\semantic_router\\Lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"from semantic_router import Route\n",
"\n",
"politics = Route(\n",
" name=\"politics\",\n",
" utterances=[\n",
" \"isn't politics the best thing ever\",\n",
" \"why don't you tell me about your political opinions\",\n",
" \"don't you just love the president\" \"don't you just hate the president\",\n",
" \"they're going to destroy this country!\",\n",
" \"they will save the country!\",\n",
" ],\n",
")\n",
"chitchat = Route(\n",
" name=\"chitchat\",\n",
" utterances=[\n",
" \"how's the weather today?\",\n",
" \"how are things going?\",\n",
" \"lovely weather today\",\n",
" \"the weather is horrendous\",\n",
" \"let's go to the chippy\",\n",
" ],\n",
")\n",
"\n",
"routes = [politics, chitchat]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "voWyqmffur0x"
},
"source": [
"We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "BI9AiDspur0y",
"outputId": "27329a54-3f16-44a5-ac20-13a6b26afb97"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2024-04-28 22:54:05 INFO semantic_router.utils.logger local\u001b[0m\n"
]
}
],
"source": [
"import os\n",
"from getpass import getpass\n",
"from semantic_router import RouteLayer\n",
"from semantic_router.encoders import CohereEncoder, OpenAIEncoder\n",
"\n",
"# dashboard.cohere.ai\n",
"# os.environ[\"COHERE_API_KEY\"] = os.getenv(\"COHERE_API_KEY\") or getpass(\n",
"# \"Enter Cohere API Key: \"\n",
"# )\n",
"# platform.openai.com\n",
"os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\") or getpass(\n",
" \"Enter OpenAI API Key: \"\n",
")\n",
"\n",
"# encoder = CohereEncoder()\n",
"encoder = OpenAIEncoder()\n",
"\n",
"rl = RouteLayer(encoder=encoder, routes=routes)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GuLCeIS5ur0y"
},
"source": [
"We run the solely static routes layer:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "_rNREh7gur0y",
"outputId": "f3a1dc0b-d760-4efb-b634-d3547011dcb7"
},
"outputs": [
{
"data": {
"text/plain": [
"RouteChoice(name='chitchat', function_call=None, similarity_score=None)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"rl(\"how's the weather today?\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "McbLKO26ur0y"
},
"source": [
"## Creating a Dynamic Route"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ANAoEjxYur0y"
},
"source": [
"As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide a `function_schema`. The function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "5jaF1Xa5ur0y"
},
"outputs": [],
"source": [
"from datetime import datetime\n",
"from zoneinfo import ZoneInfo\n",
"\n",
"\n",
"def get_time(timezone: str) -> str:\n",
" \"\"\"Finds the current time in a specific timezone.\n",
"\n",
" :param timezone: The timezone to find the current time in, should\n",
" be a valid timezone from the IANA Time Zone Database like\n",
" \"America/New_York\" or \"Europe/London\". Do NOT put the place\n",
" name itself like \"rome\", or \"new york\", you must provide\n",
" the IANA format.\n",
" :type timezone: str\n",
" :return: The current time in the specified timezone.\"\"\"\n",
" now = datetime.now(ZoneInfo(timezone))\n",
" return now.strftime(\"%H:%M\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"id": "YyFKV8jMur0z",
"outputId": "29cf80f4-552c-47bb-fbf9-019f5dfdf00a"
},
"outputs": [
{
"data": {
"text/plain": [
"'14:54'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"get_time(\"America/New_York\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4qyaRuNXur0z"
},
"source": [
"To get the function schema we can use the `get_schema` function from the `function_call` module."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "tOjuhp5Xur0z",
"outputId": "ca88a3ea-d70a-4950-be9a-63fab699de3b"
},
"outputs": [
{
"data": {
"text/plain": [
"{'name': 'get_time',\n",
" 'description': 'Finds the current time in a specific timezone.\\n\\n:param timezone: The timezone to find the current time in, should\\n be a valid timezone from the IANA Time Zone Database like\\n \"America/New_York\" or \"Europe/London\". Do NOT put the place\\n name itself like \"rome\", or \"new york\", you must provide\\n the IANA format.\\n:type timezone: str\\n:return: The current time in the specified timezone.',\n",
" 'signature': '(timezone: str) -> str',\n",
" 'output': \"<class 'str'>\"}"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantic_router.utils.function_call import get_schema\n",
"\n",
"schema = get_schema(get_time)\n",
"schema"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'type': 'function',\n",
" 'function': {'name': 'get_time',\n",
" 'description': 'Finds the current time in a specific timezone.\\n\\n:param timezone: The timezone to find the current time in, should\\n be a valid timezone from the IANA Time Zone Database like\\n \"America/New_York\" or \"Europe/London\". Do NOT put the place\\n name itself like \"rome\", or \"new york\", you must provide\\n the IANA format.\\n:type timezone: str\\n:return: The current time in the specified timezone.',\n",
" 'parameters': {'type': 'object',\n",
" 'properties': {'timezone': {'type': 'string',\n",
" 'description': 'The timezone to find the current time in, should\\n be a valid timezone from the IANA Time Zone Database like\\n \"America/New_York\" or \"Europe/London\". Do NOT put the place\\n name itself like \"rome\", or \"new york\", you must provide\\n the IANA format.'}},\n",
" 'required': ['timezone']}}}"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantic_router.utils.function_call import get_schema_openai_func_calling\n",
"\n",
"openai_function_schema = get_schema_openai_func_calling(get_time)\n",
"openai_function_schema"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HcF7jGjAur0z"
},
"source": [
"We use this to define our dynamic route:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"id": "iesBG9P3ur0z"
},
"outputs": [],
"source": [
"time_route = Route(\n",
" name=\"get_time\",\n",
" utterances=[\n",
" \"what is the time in new york city?\",\n",
" \"what is the time in london?\",\n",
" \"I live in Rome, what time is it?\",\n",
" ],\n",
" # function_schema=schema,\n",
" openai_function_schema=openai_function_schema,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZiUs3ovpur0z"
},
"source": [
"Add the new route to our `layer`:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-0vY8PRXur0z",
"outputId": "db01e14c-eab3-4f93-f4c2-e30f508c8b5d"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2024-04-28 22:54:06 INFO semantic_router.utils.logger Adding `get_time` route\u001b[0m\n"
]
}
],
"source": [
"rl.add(time_route)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7yoE0IrNur0z"
},
"source": [
"Now we can ask our layer a time related question to trigger our new dynamic route."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 53
},
"id": "Wfb68M0-ur0z",
"outputId": "79923883-2a4d-4744-f8ce-e818cb5f14c3"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[33m2024-04-28 22:54:07 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"RouteChoice(name='get_time', function_call={'timezone': 'America/New_York'}, similarity_score=None)"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"out = rl(\"what is the time in new york city?\")\n",
"out"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'14:54'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"get_time(**out.function_call)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Qt0vkq2Xur00"
},
"source": [
"Our dynamic route provides both the route itself _and_ the input parameters required to use the route."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's just verify that the other route selection still work correctly, starting with chitchat."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"RouteChoice(name='chitchat', function_call=None, similarity_score=None)"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"out = rl(\"What's the weather like?\")\n",
"out"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, an utterance that shouldn't fall into any of the routes defined above."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"RouteChoice(name=None, function_call=None, similarity_score=None)"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"out = rl(\"Tell me how to bake cake?\")\n",
"out"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "J0oD1dxIur00"
},
"source": [
"---"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "decision-layer",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 0
}