{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "UxqB7_Ieur0s"
},
"source": [
"[](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EduhQaNAur0u"
},
"source": [
"# Dynamic Routes"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_4JgNeX4ur0v"
},
"source": [
"In semantic-router there are two types of routes that can be chosen. Both routes belong to the `Route` object, the only difference between them is that _static_ routes return a `Route.name` when chosen, whereas _dynamic_ routes use an LLM call to produce parameter input values.\n",
"\n",
"For example, a _static_ route will tell us if a query is talking about mathematics by returning the route name (which could be `\"math\"` for example). A _dynamic_ route does the same thing, but it also extracts key information from the input utterance to be used in a function associated with that route. \n",
"\n",
"For example we could provide a dynamic route with associated utterances: \n",
"\n",
"```\n",
"\"what is x to the power of y?\"\n",
"\"what is 9 to the power of 4?\"\n",
"\"calculate the result of base x and exponent y\"\n",
"\"calculate the result of base 10 and exponent 3\"\n",
"\"return x to the power of y\"\n",
"```\n",
"\n",
"and we could also provide the route with a schema outlining key features of the function:\n",
"\n",
"```\n",
"def power(base: float, exponent: float) -> float:\n",
" \"\"\"Raise base to the power of exponent.\n",
"\n",
" Args:\n",
" base (float): The base number.\n",
" exponent (float): The exponent to which the base is raised.\n",
"\n",
" Returns:\n",
" float: The result of base raised to the power of exponent.\n",
" \"\"\"\n",
" return base ** exponent\n",
"```\n",
"\n",
"Then, if the users input utterance is \"What is 2 to the power of 3?\", the route will be triggered, as the input utterance is semantically similar to the route utterances. Furthermore, the route utilizes an LLM to identify that `base=2` and `expoenent=3`. These values are returned in such a way that they can be used in the above `power` function. That is, the dynamic router automates the process of calling relevant functions from natural language inputs. \n",
"\n",
"***⚠️ Note: We have a fully local version of dynamic routes available at [docs/05-local-execution.ipynb](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb). The local 05 version tends to outperform the OpenAI version we demo in this notebook, so we'd recommend trying [05](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb)!***"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bbmw8CO4ur0v"
},
"source": [
"## Installing the Library"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "dLElfRhgur0v"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: tzdata in c:\\users\\siraj\\documents\\personal\\work\\aurelio\\virtual environments\\semantic_router_3\\lib\\site-packages (2024.1)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"[notice] A new release of pip is available: 23.1.2 -> 24.0\n",
"[notice] To update, run: python.exe -m pip install --upgrade pip\n",
"\n",
"[notice] A new release of pip is available: 23.1.2 -> 24.0\n",
"[notice] To update, run: python.exe -m pip install --upgrade pip\n"
]
}
],
"source": [
"!pip install tzdata\n",
"!pip install -qU semantic-router"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BixZd6Eour0w"
},
"source": [
"## Initializing Routes and RouteLayer"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PxnW9qBvur0x"
},
"source": [
"Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "kc9Ty6Lgur0x"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"c:\\Users\\Siraj\\Documents\\Personal\\Work\\Aurelio\\Virtual Environments\\semantic_router_3\\Lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"from semantic_router import Route\n",
"\n",
"politics = Route(\n",
" name=\"politics\",\n",
" utterances=[\n",
" \"isn't politics the best thing ever\",\n",
" \"why don't you tell me about your political opinions\",\n",
" \"don't you just love the president\" \"don't you just hate the president\",\n",
" \"they're going to destroy this country!\",\n",
" \"they will save the country!\",\n",
" ],\n",
")\n",
"chitchat = Route(\n",
" name=\"chitchat\",\n",
" utterances=[\n",
" \"how's the weather today?\",\n",
" \"how are things going?\",\n",
" \"lovely weather today\",\n",
" \"the weather is horrendous\",\n",
" \"let's go to the chippy\",\n",
" ],\n",
")\n",
"\n",
"routes = [politics, chitchat]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "voWyqmffur0x"
},
"source": [
"We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "BI9AiDspur0y",
"outputId": "27329a54-3f16-44a5-ac20-13a6b26afb97"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger local\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 1 length: 34\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 1 trunc length: 34\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 2 length: 51\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 2 trunc length: 51\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 3 length: 66\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 3 trunc length: 66\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 4 length: 38\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 4 trunc length: 38\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 5 length: 27\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 5 trunc length: 27\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 6 length: 24\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 6 trunc length: 24\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 7 length: 21\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 7 trunc length: 21\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 8 length: 20\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 8 trunc length: 20\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 9 length: 25\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 9 trunc length: 25\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 10 length: 22\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:54 INFO semantic_router.utils.logger Document 10 trunc length: 22\u001b[0m\n"
]
}
],
"source": [
"import os\n",
"from getpass import getpass\n",
"from semantic_router import RouteLayer\n",
"from semantic_router.encoders import CohereEncoder, OpenAIEncoder\n",
"\n",
"# dashboard.cohere.ai\n",
"# os.environ[\"COHERE_API_KEY\"] = os.getenv(\"COHERE_API_KEY\") or getpass(\n",
"# \"Enter Cohere API Key: \"\n",
"# )\n",
"# platform.openai.com\n",
"os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\") or getpass(\n",
" \"Enter OpenAI API Key: \"\n",
")\n",
"\n",
"# encoder = CohereEncoder()\n",
"encoder = OpenAIEncoder()\n",
"\n",
"rl = RouteLayer(encoder=encoder, routes=routes)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GuLCeIS5ur0y"
},
"source": [
"We run the solely static routes layer:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "_rNREh7gur0y",
"outputId": "f3a1dc0b-d760-4efb-b634-d3547011dcb7"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2024-05-01 01:25:55 INFO semantic_router.utils.logger Document 1 length: 24\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:55 INFO semantic_router.utils.logger Document 1 trunc length: 24\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"RouteChoice(name='chitchat', function_call=None, similarity_score=None)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"rl(\"how's the weather today?\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "McbLKO26ur0y"
},
"source": [
"## Creating a Dynamic Route"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ANAoEjxYur0y"
},
"source": [
"As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide the `function_schemas` as a list. Each function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "5jaF1Xa5ur0y"
},
"outputs": [],
"source": [
"from datetime import datetime\n",
"from zoneinfo import ZoneInfo\n",
"\n",
"\n",
"def get_time(timezone: str) -> str:\n",
" \"\"\"Finds the current time in a specific timezone.\n",
"\n",
" :param timezone: The timezone to find the current time in, should\n",
" be a valid timezone from the IANA Time Zone Database like\n",
" \"America/New_York\" or \"Europe/London\". Do NOT put the place\n",
" name itself like \"rome\", or \"new york\", you must provide\n",
" the IANA format.\n",
" :type timezone: str\n",
" :return: The current time in the specified timezone.\"\"\"\n",
" now = datetime.now(ZoneInfo(timezone))\n",
" return now.strftime(\"%H:%M\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"id": "YyFKV8jMur0z",
"outputId": "29cf80f4-552c-47bb-fbf9-019f5dfdf00a"
},
"outputs": [
{
"data": {
"text/plain": [
"'17:25'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"get_time(\"America/New_York\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4qyaRuNXur0z"
},
"source": [
"To get the function schema we can use the `get_schema` function from the `function_call` module."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "tOjuhp5Xur0z",
"outputId": "ca88a3ea-d70a-4950-be9a-63fab699de3b"
},
"outputs": [
{
"data": {
"text/plain": [
"[{'type': 'function',\n",
" 'function': {'name': 'get_time',\n",
" 'description': 'Finds the current time in a specific timezone.\\n\\n:param timezone: The timezone to find the current time in, should\\n be a valid timezone from the IANA Time Zone Database like\\n \"America/New_York\" or \"Europe/London\". Do NOT put the place\\n name itself like \"rome\", or \"new york\", you must provide\\n the IANA format.\\n:type timezone: str\\n:return: The current time in the specified timezone.',\n",
" 'parameters': {'type': 'object',\n",
" 'properties': {'timezone': {'type': 'string',\n",
" 'description': 'The timezone to find the current time in, should\\n be a valid timezone from the IANA Time Zone Database like\\n \"America/New_York\" or \"Europe/London\". Do NOT put the place\\n name itself like \"rome\", or \"new york\", you must provide\\n the IANA format.'}},\n",
" 'required': ['timezone']}}}]"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantic_router.utils.function_call import get_schemas_openai\n",
"\n",
"schema = get_schemas_openai([get_time])\n",
"schema"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HcF7jGjAur0z"
},
"source": [
"We use this to define our dynamic route:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"id": "iesBG9P3ur0z"
},
"outputs": [],
"source": [
"time_route = Route(\n",
" name=\"get_time\",\n",
" utterances=[\n",
" \"what is the time in new york city?\",\n",
" \"what is the time in london?\",\n",
" \"I live in Rome, what time is it?\",\n",
" ],\n",
" function_schemas=schema,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"time_route.llm"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZiUs3ovpur0z"
},
"source": [
"Add the new route to our `layer`:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-0vY8PRXur0z",
"outputId": "db01e14c-eab3-4f93-f4c2-e30f508c8b5d"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2024-05-01 01:25:56 INFO semantic_router.utils.logger Adding `get_time` route\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:56 INFO semantic_router.utils.logger Document 1 length: 34\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:56 INFO semantic_router.utils.logger Document 1 trunc length: 34\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:56 INFO semantic_router.utils.logger Document 2 length: 27\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:56 INFO semantic_router.utils.logger Document 2 trunc length: 27\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:56 INFO semantic_router.utils.logger Document 3 length: 32\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:56 INFO semantic_router.utils.logger Document 3 trunc length: 32\u001b[0m\n"
]
}
],
"source": [
"rl.add(time_route)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"time_route.llm"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7yoE0IrNur0z"
},
"source": [
"Now we can ask our layer a time related question to trigger our new dynamic route."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 53
},
"id": "Wfb68M0-ur0z",
"outputId": "79923883-2a4d-4744-f8ce-e818cb5f14c3"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2024-05-01 01:25:56 INFO semantic_router.utils.logger Document 1 length: 34\u001b[0m\n",
"\u001b[32m2024-05-01 01:25:56 INFO semantic_router.utils.logger Document 1 trunc length: 34\u001b[0m\n",
"\u001b[33m2024-05-01 01:25:57 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"RouteChoice(name='get_time', function_call=[{'function_name': 'get_time', 'arguments': '{\"timezone\":\"America/New_York\"}'}], similarity_score=None)"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"out = rl(\"what is the time in new york city?\")\n",
"out\n"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[{'function_name': 'get_time', 'arguments': '{\"timezone\":\"America/New_York\"}'}]\n"
]
}
],
"source": [
"print(out.function_call)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"17:25\n"
]
}
],
"source": [
"import json\n",
"\n",
"for call in out.function_call:\n",
" if call['function_name'] == 'get_time':\n",
" args = json.loads(call['arguments'])\n",
" result = get_time(**args)\n",
"print(result)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Qt0vkq2Xur00"
},
"source": [
"Our dynamic route provides both the route itself _and_ the input parameters required to use the route."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "J0oD1dxIur00"
},
"source": [
"---"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "decision-layer",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 0
}