Skip to content
Snippets Groups Projects
Commit 47a5b83a authored by Stephen Witkowski's avatar Stephen Witkowski
Browse files

Updating colab link in google.ipynb

parent c651bd29
No related branches found
No related tags found
No related merge requests found
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/encoders/huggingface.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/encoders/huggingface.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/encoders/google.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/encoders/google.ipynb)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Using GoogleEncoder # Using GoogleEncoder
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Google's [Pathways Language Model](https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html) (PaLM) is a dense decoder-only model that is trained on a large corpus of text data. The hidden states of the model can be used as embeddings for text data, and Google has released versions of those layers for public use. This notebook demonstrates how to use the GoogleEncoder with the Semantic Router. Google's [Pathways Language Model](https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html) (PaLM) is a dense decoder-only model that is trained on a large corpus of text data. The hidden states of the model can be used as embeddings for text data, and Google has released versions of those layers for public use. This notebook demonstrates how to use the GoogleEncoder with the Semantic Router.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Getting Started ## Getting Started
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We start by installing semantic-router. Support for the new `GoogleEncoder` class was added in `semantic-router==0.0.X`. We start by installing semantic-router. Support for the new `GoogleEncoder` class was added in `semantic-router==0.0.X`.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
!pip install -qU semantic-router==0.0.X !pip install -qU semantic-router==0.0.X
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We start by defining a dictionary mapping routes to example phrases that should trigger those routes. We start by defining a dictionary mapping routes to example phrases that should trigger those routes.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router import Route from semantic_router import Route
politics = Route( politics = Route(
name="politics", name="politics",
utterances=[ utterances=[
"isn't politics the best thing ever", "isn't politics the best thing ever",
"why don't you tell me about your political opinions", "why don't you tell me about your political opinions",
"don't you just love the president", "don't you just love the president",
"don't you just hate the president", "don't you just hate the president",
"they're going to destroy this country!", "they're going to destroy this country!",
"they will save the country!", "they will save the country!",
], ],
) )
``` ```
%% Output
/Users/Stephen.Witkowski/miniforge3/envs/semantic-router/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Let's define another for good measure: Let's define another for good measure:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
chitchat = Route( chitchat = Route(
name="chitchat", name="chitchat",
utterances=[ utterances=[
"how's the weather today?", "how's the weather today?",
"how are things going?", "how are things going?",
"lovely weather today", "lovely weather today",
"the weather is horrendous", "the weather is horrendous",
"let's go to the chippy", "let's go to the chippy",
], ],
) )
routes = [politics, chitchat] routes = [politics, chitchat]
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now we initialize our embedding model. To do this with GoogleEncoder, you'll need to have an active Google Cloud Platform account and a project with the Embeddings API enabled. You can find more information on how to set up a development project in the [Google Cloud documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-text-embeddings). Now we initialize our embedding model. To do this with GoogleEncoder, you'll need to have an active Google Cloud Platform account and a project with the Embeddings API enabled. You can find more information on how to set up a development project in the [Google Cloud documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-text-embeddings).
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router.encoders import GoogleEncoder from semantic_router.encoders import GoogleEncoder
PROJECT_ID = "your-project-id" PROJECT_ID = "your-project-id"
LOCATION = "us-central1" LOCATION = "us-central1"
encoder = GoogleEncoder(project_id=PROJECT_ID, location=LOCATION) encoder = GoogleEncoder(project_id=PROJECT_ID, location=LOCATION)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now we define the `RouteLayer`. When called, the route layer will consume text (a query) and output the category (`Route`) it belongs to — to initialize a `RouteLayer` we need our `encoder` model and a list of `routes`. Now we define the `RouteLayer`. When called, the route layer will consume text (a query) and output the category (`Route`) it belongs to — to initialize a `RouteLayer` we need our `encoder` model and a list of `routes`.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from semantic_router.layer import RouteLayer from semantic_router.layer import RouteLayer
rl = RouteLayer(encoder=encoder, routes=routes) rl = RouteLayer(encoder=encoder, routes=routes)
``` ```
%% Output
2024-04-01 09:25:34 INFO semantic_router.utils.logger local
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now we can test it: Now we can test it:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl("don't you love politics?") rl("don't you love politics?")
``` ```
%% Output
chitchat 0.6414884458605304
politics 0.7184715572295559
politics 0.7439275034824486
politics 0.7782497192765216
politics 0.896539779971592
RouteChoice(name='politics', function_call=None, similarity_score=None)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl("how's the weather today?") rl("how's the weather today?")
``` ```
%% Output
politics 0.6707593528326423
chitchat 0.7714977687548606
chitchat 0.8433716231360381
chitchat 0.8563028552685018
chitchat 1.0000000000000004
RouteChoice(name='chitchat', function_call=None, similarity_score=None)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Both are classified accurately, what if we send a query that is unrelated to our existing `Route` objects? Both are classified accurately, what if we send a query that is unrelated to our existing `Route` objects?
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
rl("I'm interested in learning about llama 2") rl("I'm interested in learning about llama 2")
``` ```
%% Output
politics 0.5136569490418237
politics 0.5147603969017434
politics 0.5481785565189041
chitchat 0.5495495505326361
chitchat 0.5587930037610617
RouteChoice(name=None, function_call=None, similarity_score=None)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In this case, we return `None` because no matches were identified. In this case, we return `None` because no matches were identified.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment