Skip to content
Snippets Groups Projects
Unverified Commit 45b2079d authored by James Briggs's avatar James Briggs
Browse files

added LLM to llm classes, update version and docs

parent 548bd403
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 73 additions and 68 deletions
README.md
+ 12
− 10
View file @ 45b2079d
......@@ -12,7 +12,7 @@
<img alt="Github License" src="https://img.shields.io/badge/License-MIT-yellow.svg" />
</p>
Semantic Router is a superfast decision layer for your LLMs and agents. Rather than waiting for slow LLM generations to make tool-use decisions, we use the magic of semantic vector space to make those decisions — _routing_ our requests using _semantic_ meaning.
Semantic Router is a superfast decision-making layer for your LLMs and agents. Rather than waiting for slow LLM generations to make tool-use decisions, we use the magic of semantic vector space to make those decisions — _routing_ our requests using _semantic_ meaning.
## Quickstart
......@@ -22,7 +22,9 @@ To get started with _semantic-router_ we install it like so:
pip install -qU semantic-router
```
We begin by defining a set of `Decision` objects. These are the decision paths that the semantic router can decide to use, let's try two simple decisions for now — one for talk on _politics_ and another for _chitchat_:
❗️ _If wanting to use local embeddings you can use `FastEmbedEncoder` (`pip install -qU semantic-router[fastembed]`). To use the `HybridRouteLayer` you must `pip install -qU semantic-router[hybrid]`._
We begin by defining a set of `Route` objects. These are the decision paths that the semantic router can decide to use, let's try two simple routes for now — one for talk on _politics_ and another for _chitchat_:
```python
from semantic_router import Route
......@@ -56,7 +58,7 @@ chitchat = Route(
routes = [politics, chitchat]
```
We have our decisions ready, now we initialize an embedding / encoder model. We currently support a `CohereEncoder` and `OpenAIEncoder` — more encoders will be added soon. To initialize them we do:
We have our routes ready, now we initialize an embedding / encoder model. We currently support a `CohereEncoder` and `OpenAIEncoder` — more encoders will be added soon. To initialize them we do:
```python
import os
......@@ -71,18 +73,18 @@ os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>"
encoder = OpenAIEncoder()
```
With our `decisions` and `encoder` defined we now create a `DecisionLayer`. The decision layer handles our semantic decision making.
With our `routes` and `encoder` defined we now create a `RouteLayer`. The route layer handles our semantic decision making.
```python
from semantic_router.layer import RouteLayer
dl = RouteLayer(encoder=encoder, routes=routes)
rl = RouteLayer(encoder=encoder, routes=routes)
```
We can now use our decision layer to make super fast decisions based on user queries. Let's try with two queries that should trigger our decisions:
We can now use our route layer to make super fast decisions based on user queries. Let's try with two queries that should trigger our route decisions:
```python
dl("don't you love politics?").name
rl("don't you love politics?").name
```
```
......@@ -92,7 +94,7 @@ dl("don't you love politics?").name
Correct decision, let's try another:
```python
dl("how's the weather today?").name
rl("how's the weather today?").name
```
```
......@@ -102,14 +104,14 @@ dl("how's the weather today?").name
We get both decisions correct! Now lets try sending an unrelated query:
```python
dl("I'm interested in learning about llama 2").name
rl("I'm interested in learning about llama 2").name
```
```
[Out]:
```
In this case, no decision could be made as we had no matches — so our decision layer returned `None`!
In this case, no decision could be made as we had no matches — so our route layer returned `None`!
## 📚 [Resources](https://github.com/aurelio-labs/semantic-router/tree/main/docs)
......
docs/02-dynamic-routes.ipynb
+ 42
− 38
View file @ 45b2079d
%% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb)
%% Cell type:markdown id: tags:
# Dynamic Routes
%% Cell type:markdown id: tags:
In semantic-router there are two types of routes that can be chosen. Both routes belong to the `Route` object, the only difference between them is that _static_ routes return a `Route.name` when chosen, whereas _dynamic_ routes use an LLM call to produce parameter input values.
For example, a _static_ route will tell us if a query is talking about mathematics by returning the route name (which could be `"math"` for example). A _dynamic_ route can generate additional values, so it may decide a query is talking about maths, but it can also generate Python code that we can later execute to answer the user's query, this output may look like `"math", "import math; output = math.sqrt(64)`.
%% Cell type:markdown id: tags:
## Installing the Library
%% Cell type:code id: tags:
``` python
!pip install -qU semantic-router==0.0.14
!pip install -qU semantic-router==0.0.15
```
%% Cell type:markdown id: tags:
_**⚠️ If using Google Colab, install the prerequisites and then restart the notebook before continuing**_
%% Cell type:markdown id: tags:
## Initializing Routes and RouteLayer
%% Cell type:markdown id: tags:
Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes.
%% Cell type:code id: tags:
``` python
from semantic_router import Route
politics = Route(
name="politics",
utterances=[
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"don't you just love the president" "don't you just hate the president",
"they're going to destroy this country!",
"they will save the country!",
],
)
chitchat = Route(
name="chitchat",
utterances=[
"how's the weather today?",
"how are things going?",
"lovely weather today",
"the weather is horrendous",
"let's go to the chippy",
],
)
routes = [politics, chitchat]
```
%% Output
%% Cell type:markdown id: tags:
/Users/jamesbriggs/opt/anaconda3/envs/decision-layer/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`.
%% Cell type:code id: tags:
``` python
import os
from getpass import getpass
from semantic_router import RouteLayer
from semantic_router.encoders import CohereEncoder, OpenAIEncoder
# dashboard.cohere.ai
os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass(
"Enter Cohere API Key: "
# os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass(
# "Enter Cohere API Key: "
# )
# platform.openai.com
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass(
"Enter OpenAI API Key: "
)
rl = RouteLayer(routes=routes)
# encoder = CohereEncoder()
encoder = OpenAIEncoder()
rl = RouteLayer(encoder=encoder, routes=routes)
```
%% Output
2023-12-28 19:19:39 INFO semantic_router.utils.logger Initializing RouteLayer
2024-01-07 15:23:12 INFO semantic_router.utils.logger Initializing RouteLayer
%% Cell type:markdown id: tags:
We run the solely static routes layer:
%% Cell type:code id: tags:
``` python
rl("how's the weather today?")
```
%% Output
RouteChoice(name='chitchat', function_call=None)
%% Cell type:markdown id: tags:
## Creating a Dynamic Route
%% Cell type:markdown id: tags:
As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide a `function_schema`. The function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly.
%% Cell type:code id: tags:
``` python
from datetime import datetime
from zoneinfo import ZoneInfo
def get_time(timezone: str) -> str:
"""Finds the current time in a specific timezone.
:param timezone: The timezone to find the current time in, should
be a valid timezone from the IANA Time Zone Database like
"America/New_York" or "Europe/London".
:type timezone: str
:return: The current time in the specified timezone."""
now = datetime.now(ZoneInfo(timezone))
return now.strftime("%H:%M")
```
%% Cell type:code id: tags:
``` python
get_time("America/New_York")
```
%% Output
'13:19'
'09:23'
%% Cell type:markdown id: tags:
To get the function schema we can use the `get_schema` function from the `function_call` module.
%% Cell type:code id: tags:
``` python
from semantic_router.utils.function_call import get_schema
schema = get_schema(get_time)
schema
```
%% Output
{'name': 'get_time',
'description': 'Finds the current time in a specific timezone.\n\n:param timezone: The timezone to find the current time in, should\n be a valid timezone from the IANA Time Zone Database like\n "America/New_York" or "Europe/London".\n:type timezone: str\n:return: The current time in the specified timezone.',
'signature': '(timezone: str) -> str',
'output': "<class 'str'>"}
%% Cell type:markdown id: tags:
We use this to define our dynamic route:
%% Cell type:code id: tags:
``` python
time_route = Route(
name="get_time",
utterances=[
"what is the time in new york city?",
"what is the time in london?",
"I live in Rome, what time is it?",
],
function_schema=schema,
)
```
%% Cell type:markdown id: tags:
Add the new route to our `layer`:
%% Cell type:code id: tags:
``` python
rl.add(time_route)
```
%% Output
Adding route `get_time`
Adding route to categories
Adding route to index
2024-01-07 15:23:16 INFO semantic_router.utils.logger Adding `get_time` route
%% Cell type:markdown id: tags:
Now we can ask our layer a time related question to trigger our new dynamic route.
%% Cell type:code id: tags:
``` python
# https://openrouter.ai/keys
os.environ["OPENROUTER_API_KEY"] = os.getenv("OPENROUTER_API_KEY") or getpass(
"Enter OpenRouter API Key: "
# https://platform.openai.com/
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass(
"Enter OpenAI API Key: "
)
rl("what is the time in new york city?")
```
%% Output
2023-12-28 19:21:58 INFO semantic_router.utils.logger Extracting function input...
2024-01-07 15:23:17 WARNING semantic_router.utils.logger No LLM provided for dynamic route, will use OpenAI LLM default. Ensure API key is set in OPENAI_API_KEY environment variable.
2024-01-07 15:23:17 INFO semantic_router.utils.logger Extracting function input...
RouteChoice(name='get_time', function_call={'timezone': 'America/New_York'})
RouteChoice(name='get_time', function_call={'timezone': 'new york city'})
%% Cell type:markdown id: tags:
---
......
pyproject.toml
+ 1
− 1
View file @ 45b2079d
[tool.poetry]
name = "semantic-router"
version = "0.0.14"
version = "0.0.15"
description = "Super fast semantic router for AI decision making"
authors = [
"James Briggs <james@aurelio.ai>",
......
semantic_router/layer.py
+ 8
− 8
View file @ 45b2079d
......@@ -10,7 +10,7 @@ from semantic_router.encoders import (
OpenAIEncoder,
FastEmbedEncoder,
)
from semantic_router.llms import BaseLLM
from semantic_router.llms import BaseLLM, OpenAILLM
from semantic_router.linear import similarity_matrix, top_scores
from semantic_router.route import Route
from semantic_router.schema import Encoder, EncoderType, RouteChoice
......@@ -193,9 +193,13 @@ class RouteLayer:
route = [route for route in self.routes if route.name == top_class][0]
if route.function_schema and not isinstance(route.llm, BaseLLM):
if not self.llm:
raise ValueError(
"LLM is required for dynamic routes. Please ensure the 'llm' is set."
logger.warning(
"No LLM provided for dynamic route, will use OpenAI LLM "
"default. Ensure API key is set in OPENAI_API_KEY environment "
"variable."
)
self.llm = OpenAILLM()
route.llm = self.llm
else:
route.llm = self.llm
return route(text)
......@@ -228,24 +232,20 @@ class RouteLayer:
return cls(encoder=encoder, routes=config.routes)
def add(self, route: Route):
print(f"Adding route `{route.name}`")
logger.info(f"Adding `{route.name}` route")
# create embeddings
embeds = self.encoder(route.utterances)
# create route array
if self.categories is None:
print("Initializing categories array")
self.categories = np.array([route.name] * len(embeds))
else:
print("Adding route to categories")
str_arr = np.array([route.name] * len(embeds))
self.categories = np.concatenate([self.categories, str_arr])
# create utterance array (the index)
if self.index is None:
print("Initializing index array")
self.index = np.array(embeds)
else:
print("Adding route to index")
embed_arr = np.array(embeds)
self.index = np.concatenate([self.index, embed_arr])
# add route to routes list
......
semantic_router/llms/__init__.py
+ 4
− 4
View file @ 45b2079d
from semantic_router.llms.base import BaseLLM
from semantic_router.llms.openai import OpenAI
from semantic_router.llms.openrouter import OpenRouter
from semantic_router.llms.cohere import Cohere
from semantic_router.llms.openai import OpenAILLM
from semantic_router.llms.openrouter import OpenRouterLLM
from semantic_router.llms.cohere import CohereLLM
__all__ = ["BaseLLM", "OpenAI", "OpenRouter", "Cohere"]
__all__ = ["BaseLLM", "OpenAILLM", "OpenRouterLLM", "CohereLLM"]
semantic_router/llms/cohere.py
+ 1
− 1
View file @ 45b2079d
......@@ -4,7 +4,7 @@ from semantic_router.llms import BaseLLM
from semantic_router.schema import Message
class Cohere(BaseLLM):
class CohereLLM(BaseLLM):
client: cohere.Client | None = None
def __init__(
......
semantic_router/llms/openai.py
+ 1
− 1
View file @ 45b2079d
......@@ -5,7 +5,7 @@ from semantic_router.llms import BaseLLM
from semantic_router.schema import Message
class OpenAI(BaseLLM):
class OpenAILLM(BaseLLM):
client: openai.OpenAI | None
temperature: float | None
max_tokens: int | None
......
semantic_router/llms/openrouter.py
+ 1
− 1
View file @ 45b2079d
......@@ -5,7 +5,7 @@ from semantic_router.llms import BaseLLM
from semantic_router.schema import Message
class OpenRouter(BaseLLM):
class OpenRouterLLM(BaseLLM):
client: openai.OpenAI | None
base_url: str | None
temperature: float | None
......
semantic_router/route.py
+ 3
− 4
View file @ 45b2079d
......@@ -5,12 +5,10 @@ from typing import Any, Callable, Union
from pydantic import BaseModel
from semantic_router.llms import BaseLLM
from semantic_router.schema import RouteChoice
from semantic_router.schema import Message, RouteChoice
from semantic_router.utils import function_call
from semantic_router.utils.logger import logger
from semantic_router.schema import Message
def is_valid(route_config: str) -> bool:
try:
......@@ -51,7 +49,8 @@ class Route(BaseModel):
if self.function_schema:
if not self.llm:
raise ValueError(
"LLM is required for dynamic routes. Please ensure the 'llm' is set."
"LLM is required for dynamic routes. Please ensure the `llm` "
"attribute is set."
)
# if a function schema is provided we generate the inputs
extracted_inputs = function_call.extract_function_inputs(
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

:feet::heart: Strive to be the person your dogs believe you are :heart::feet: