"[](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/encoders/huggingface.ipynb) [](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/encoders/huggingface.ipynb)"
"[](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/encoders/google.ipynb) [](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/encoders/google.ipynb)"
]
]
},
},
{
{
...
@@ -53,18 +53,9 @@
...
@@ -53,18 +53,9 @@
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [],
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/Stephen.Witkowski/miniforge3/envs/semantic-router/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"source": [
"from semantic_router import Route\n",
"from semantic_router import Route\n",
"\n",
"\n",
...
@@ -90,7 +81,7 @@
...
@@ -90,7 +81,7 @@
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
...
@@ -117,7 +108,7 @@
...
@@ -117,7 +108,7 @@
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
...
@@ -138,17 +129,9 @@
...
@@ -138,17 +129,9 @@
},
},
{
{
"cell_type": "code",
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"metadata": {},
"outputs": [
"outputs": [],
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2024-04-01 09:25:34 INFO semantic_router.utils.logger local\u001b[0m\n"
"rl(\"I'm interested in learning about llama 2\")"
"rl(\"I'm interested in learning about llama 2\")"
]
]
...
...
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
[](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/encoders/huggingface.ipynb) [](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/encoders/huggingface.ipynb)
[](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/encoders/google.ipynb) [](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/encoders/google.ipynb)
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
# Using GoogleEncoder
# Using GoogleEncoder
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
Google's [Pathways Language Model](https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html)(PaLM) is a dense decoder-only model that is trained on a large corpus of text data. The hidden states of the model can be used as embeddings for text data, and Google has released versions of those layers for public use. This notebook demonstrates how to use the GoogleEncoder with the Semantic Router.
Google's [Pathways Language Model](https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html)(PaLM) is a dense decoder-only model that is trained on a large corpus of text data. The hidden states of the model can be used as embeddings for text data, and Google has released versions of those layers for public use. This notebook demonstrates how to use the GoogleEncoder with the Semantic Router.
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Getting Started
## Getting Started
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
We start by installing semantic-router. Support for the new `GoogleEncoder` class was added in `semantic-router==0.0.X`.
We start by installing semantic-router. Support for the new `GoogleEncoder` class was added in `semantic-router==0.0.X`.
%% Cell type:code id: tags:
%% Cell type:code id: tags:
``` python
``` python
!pipinstall-qUsemantic-router==0.0.X
!pipinstall-qUsemantic-router==0.0.X
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
We start by defining a dictionary mapping routes to example phrases that should trigger those routes.
We start by defining a dictionary mapping routes to example phrases that should trigger those routes.
%% Cell type:code id: tags:
%% Cell type:code id: tags:
``` python
``` python
fromsemantic_routerimportRoute
fromsemantic_routerimportRoute
politics=Route(
politics=Route(
name="politics",
name="politics",
utterances=[
utterances=[
"isn't politics the best thing ever",
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"why don't you tell me about your political opinions",
"don't you just love the president",
"don't you just love the president",
"don't you just hate the president",
"don't you just hate the president",
"they're going to destroy this country!",
"they're going to destroy this country!",
"they will save the country!",
"they will save the country!",
],
],
)
)
```
```
%% Output
/Users/Stephen.Witkowski/miniforge3/envs/semantic-router/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
Let's define another for good measure:
Let's define another for good measure:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
``` python
``` python
chitchat=Route(
chitchat=Route(
name="chitchat",
name="chitchat",
utterances=[
utterances=[
"how's the weather today?",
"how's the weather today?",
"how are things going?",
"how are things going?",
"lovely weather today",
"lovely weather today",
"the weather is horrendous",
"the weather is horrendous",
"let's go to the chippy",
"let's go to the chippy",
],
],
)
)
routes=[politics,chitchat]
routes=[politics,chitchat]
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
Now we initialize our embedding model. To do this with GoogleEncoder, you'll need to have an active Google Cloud Platform account and a project with the Embeddings API enabled. You can find more information on how to set up a development project in the [Google Cloud documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-text-embeddings).
Now we initialize our embedding model. To do this with GoogleEncoder, you'll need to have an active Google Cloud Platform account and a project with the Embeddings API enabled. You can find more information on how to set up a development project in the [Google Cloud documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-text-embeddings).
Now we define the `RouteLayer`. When called, the route layer will consume text (a query) and output the category (`Route`) it belongs to — to initialize a `RouteLayer` we need our `encoder` model and a list of `routes`.
Now we define the `RouteLayer`. When called, the route layer will consume text (a query) and output the category (`Route`) it belongs to — to initialize a `RouteLayer` we need our `encoder` model and a list of `routes`.
%% Cell type:code id: tags:
%% Cell type:code id: tags:
``` python
``` python
fromsemantic_router.layerimportRouteLayer
fromsemantic_router.layerimportRouteLayer
rl=RouteLayer(encoder=encoder,routes=routes)
rl=RouteLayer(encoder=encoder,routes=routes)
```
```
%% Output
[32m2024-04-01 09:25:34 INFO semantic_router.utils.logger local[0m