Skip to content
Snippets Groups Projects
Unverified Commit 68ea7ec6 authored by Thuc Pham's avatar Thuc Pham Committed by GitHub
Browse files

chore: use agent workflow for examples (#1726)

parent 2d11ffba
No related branches found
No related tags found
No related merge requests found
Showing
with 261 additions and 271 deletions
......@@ -137,12 +137,12 @@ const reader = new SimpleDirectoryReader();
const documents = await reader.loadData(currentDir);
const index = await VectorStoreIndex.fromDocuments(documents);
const agent = agent({
const myAgent = agent({
llm: openai({ model: "gpt-4o" }),
tools: [index.queryTool()],
});
await agent.run('...');`}
await myAgent.run('...');`}
lang="ts"
/>
</Feature>
......
......@@ -7,7 +7,7 @@ import CodeSource from "!raw-loader!../../../../../../../examples/mistral";
By default LlamaIndex.TS uses OpenAI's LLMs and embedding models, but we support [lots of other LLMs](../modules/llms) including models from Mistral (Mistral, Mixtral), Anthropic (Claude) and Google (Gemini).
If you don't want to use an API at all you can [run a local model](../../examples/local_llm).
If you don't want to use an API at all you can [run a local model](./local_llm).
This example runs you through the process of setting up a Mistral model:
......
......@@ -106,21 +106,38 @@ Some modules uses `Web Stream` API like `ReadableStream` and `WritableStream`, y
}
```
```ts twoslash
import { OpenAIAgent } from '@llamaindex/openai'
```typescript
import { agent, tool } from 'llamaindex'
import { openai } from "@llamaindex/openai";
const agent = new OpenAIAgent({
tools: []
})
Settings.llm = openai({
model: "gpt-4o-mini",
});
const response = await agent.chat({
message: 'Hello, how are you?',
stream: true
})
for await (const _ of response) {
//^?
// ...
const addTool = tool({
name: "add",
description: "Adds two numbers",
parameters: z.object({x: z.number(), y: z.number()}),
execute: ({ x, y }) => x + y,
});
const myAgent = agent({
tools: [addTool],
});
// Chat with the agent
const context = myAgent.run("Hello, how are you?");
for await (const event of context) {
if (event instanceof AgentStream) {
for (const chunk of event.data.delta) {
process.stdout.write(chunk); // stream response
}
} else {
console.log(event); // other events
}
}
```
## Run TypeScript Script in Node.js
......
......@@ -25,15 +25,21 @@ npx tsx example.ts
First we'll need to pull in our dependencies. These are:
- The OpenAI class to use the OpenAI LLM
- FunctionTool to provide tools to our agent
- OpenAIAgent to create the agent itself
- tool to provide tools to our agent
- agent to create the single agent
- Settings to define some global settings for the library
- Dotenv to load our API key from the .env file
- Zod to define the schema for our tool
```javascript
import { FunctionTool, Settings } from "llamaindex";
import { OpenAI, OpenAIAgent } from "@llamaindex/openai";
import "dotenv/config";
import {
agent,
AgentStream,
tool,
openai,
Settings,
} from "llamaindex";
import { z } from "zod";
```
......@@ -42,25 +48,12 @@ import { z } from "zod";
We need to tell our OpenAI class where its API key is, and which of OpenAI's models to use. We'll be using `gpt-4o`, which is capable while still being pretty cheap. This is a global setting, so anywhere an LLM is needed will use the same model.
```javascript
Settings.llm = new OpenAI({
Settings.llm = openai({
apiKey: process.env.OPENAI_API_KEY,
model: "gpt-4o",
});
```
### Turn on logging
We want to see what our agent is up to, so we're going to hook into some events that the library generates and print them out. There are several events possible, but we'll specifically tune in to `llm-tool-call` (when a tool is called) and `llm-tool-result` (when it responds).
```javascript
Settings.callbackManager.on("llm-tool-call", (event) => {
console.log(event.detail);
});
Settings.callbackManager.on("llm-tool-result", (event) => {
console.log(event.detail);
});
```
### Create a function
We're going to create a very simple function that adds two numbers together. This will be the tool we ask our agent to use.
......@@ -75,7 +68,7 @@ Note that we're passing in an object with two named parameters, `a` and `b`. Thi
### Turn the function into a tool for the agent
This is the most complicated part of creating an agent. We need to define a `FunctionTool`. We have to pass in:
This is the most complicated part of creating an agent. We need to define a `tool`. We have to pass in:
- The function itself (`sumNumbers`)
- A name for the function, which the LLM will use to call it
......@@ -84,7 +77,7 @@ This is the most complicated part of creating an agent. We need to define a `Fun
- You can see [more examples of function schemas](https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models).
```javascript
const tool = FunctionTool.from(sumNumbers, {
const addTool = tool({
name: "sumNumbers",
description: "Use this function to sum two numbers",
parameters: z.object({
......@@ -95,13 +88,14 @@ const tool = FunctionTool.from(sumNumbers, {
description: "Second number to sum",
}),
}),
execute: sumNumbers,
});
```
We then wrap up the tools into an array. We could provide lots of tools this way, but for this example we're just using the one.
```javascript
const tools = [tool];
const tools = [addTool];
```
### Create the agent
......@@ -109,7 +103,7 @@ const tools = [tool];
With your LLM already set up and your tools defined, creating an agent is simple:
```javascript
const agent = new OpenAIAgent({ tools });
const myAgent = agent({ tools });
```
### Ask the agent a question
......@@ -117,61 +111,109 @@ const agent = new OpenAIAgent({ tools });
We can use the `chat` interface to ask our agent a question, and it will use the tools we've defined to find an answer.
```javascript
let response = await agent.chat({
message: "Add 101 and 303",
});
console.log(response);
const context = myAgent.run("Sum 101 and 303");
const result = await context;
console.log(result.data);
```
Let's see what running this looks like using `npx tsx agent.ts`
You will see the following output:
**_Output_**
```
{ result: 'The sum of 101 and 303 is 404.' }
```
To stream the response, you can use the `AgentStream` event which provides chunks of the response as they become available. This allows you to display the response incrementally rather than waiting for the full response:
```javascript
{
toolCall: {
id: 'call_ze6A8C3mOUBG4zmXO8Z4CPB5',
name: 'sumNumbers',
input: { a: 101, b: 303 }
},
toolResult: {
tool: FunctionTool { _fn: [Function: sumNumbers], _metadata: [Object] },
input: { a: 101, b: 303 },
output: '404',
isError: false
const context = myAgent.run("Add 101 and 303");
for await (const event of context) {
if (event instanceof AgentStream) {
process.stdout.write(event.data.delta);
}
}
```
**_Streaming Output_**
```
The sum of 101 and 303 is 404.
```
### Logging workflow events
To log the workflow events, you can check the event type and log the event data.
```javascript
{
response: {
raw: {
id: 'chatcmpl-9KwauZku3QOvH78MNvxJs81mDvQYK',
object: 'chat.completion',
created: 1714778824,
model: 'gpt-4-turbo-2024-04-09',
choices: [Array],
usage: [Object],
system_fingerprint: 'fp_ea6eb70039'
},
message: {
content: 'The sum of 101 and 303 is 404.',
role: 'assistant',
options: {}
const context = myAgent.run("Sum 202 and 404");
for await (const event of context) {
if (event instanceof AgentStream) {
// Stream the response
for (const chunk of event.data.delta) {
process.stdout.write(chunk);
}
} else {
// Log other events
console.log("\nWorkflow event:", JSON.stringify(event, null, 2));
}
}
```
Let's see what running this looks like using `npx tsx agent.ts`
**_Output_**
```
Workflow event: {
"data": {
"userInput": "Sum 202 and 404"
},
sources: [Getter]
"displayName": "StartEvent"
}
Workflow event: {
"data": {
"input": [
{
"role": "user",
"content": "Sum 202 and 404"
}
],
"currentAgentName": "Agent"
},
"displayName": "AgentInput"
}
Workflow event: {
"data": {
"input": [
{
"role": "system",
"content": "You are a helpful assistant. Use the provided tools to answer questions."
},
{
"role": "user",
"content": "Sum 202 and 404"
}
],
"currentAgentName": "Agent"
},
"displayName": "AgentSetup"
}
....
```
We're seeing two pieces of output here. The first is our callback firing when the tool is called. You can see in `toolResult` that the LLM has correctly passed `101` and `303` to our `sumNumbers` function, which adds them up and returns `404`.
We're seeing several workflow events being logged:
The second piece of output is the response from the LLM itself, where the `message.content` key is giving us the answer.
1. `AgentToolCall` - Shows the agent preparing to call our tool with the numbers 202 and 404
2. `AgentToolCallResult` - Shows the result of calling the tool, which returned "606"
3. `AgentInput` - Shows the original user input
4. `AgentOutput` - Shows the agent's response
Great! We've built an agent with tool use! Next you can:
Great! We've built an agent that can understand requests and use tools to fulfill them. Next you can:
- [See the full code](https://github.com/run-llama/ts-agents/blob/main/1_agent/agent.ts)
- [See the full code](https://github.com/run-llama/LlamaIndexTS/blob/main/examples/agentworkflow/blog-writer.ts)
- [Switch to a local LLM](3_local_model)
- Move on to [add Retrieval-Augmented Generation to your agent](4_agentic_rag)
......@@ -23,70 +23,27 @@ The first time you run it will also automatically download and install the model
There are two changes you need to make to the code we already wrote in `1_agent` to get Mixtral 8x7b to work. First, you need to switch to that model. Replace the call to `Settings.llm` with this:
```javascript
Settings.llm = new Ollama({
Settings.llm = ollama({
model: "mixtral:8x7b",
});
```
### Swap to a ReActAgent
### Run local agent
In our original code we used a specific OpenAIAgent, so we'll need to switch to a more generic agent pattern, the ReAct pattern. This is simple: change the `const agent` line in your code to read
You can also create local agent by importing `agent` from `llamaindex`.
```javascript
const agent = new ReActAgent({ tools });
```
(You will also need to bring in `Ollama` and `ReActAgent` in your imports)
### Run your totally local agent
Because your embeddings were already local, your agent can now run entirely locally without making any API calls.
```bash
node agent.mjs
```
import { agent } from "llamaindex";
Note that your model will probably run a lot slower than OpenAI, so be prepared to wait a while!
**_Output_**
const workflow = agent({
tools: [getWeatherTool],
});
```javascript
{
response: {
message: {
role: 'assistant',
content: ' Thought: I need to use a tool to add the numbers 101 and 303.\n' +
'Action: sumNumbers\n' +
'Action Input: {"a": 101, "b": 303}\n' +
'\n' +
'Observation: 404\n' +
'\n' +
'Thought: I can answer without using any more tools.\n' +
'Answer: The sum of 101 and 303 is 404.'
},
raw: {
model: 'mixtral:8x7b',
created_at: '2024-05-09T00:24:30.339473Z',
message: [Object],
done: true,
total_duration: 64678371209,
load_duration: 57394551334,
prompt_eval_count: 475,
prompt_eval_duration: 4163981000,
eval_count: 94,
eval_duration: 3116692000
}
},
sources: [Getter]
}
const workflowContext = workflow.run(
"What's the weather like in San Francisco?",
);
```
Tada! You can see all of this in the folder `1a_mixtral`.
### Extending to other examples
You can use a ReActAgent instead of an OpenAIAgent in any of the further examples below, but keep in mind that GPT-4 is a lot more capable than Mixtral 8x7b, so you may see more errors or failures in reasoning if you are using an entirely local setup.
### Next steps
Now you've got a local agent, you can [add Retrieval-Augmented Generation to your agent](4_agentic_rag).
......@@ -37,7 +37,7 @@ import { Tab, Tabs } from "fumadocs-ui/components/tabs";
We'll be bringing in `SimpleDirectoryReader`, `HuggingFaceEmbedding`, `VectorStoreIndex`, and `QueryEngineTool`, `OpenAIContextAwareAgent` from LlamaIndex.TS, as well as the dependencies we previously used.
```javascript
import { FunctionTool, QueryEngineTool, Settings, VectorStoreIndex } from "llamaindex";
import { QueryEngineTool, Settings, VectorStoreIndex } from "llamaindex";
import { OpenAI, OpenAIAgent } from "@llamaindex/openai";
import { HuggingFaceEmbedding } from "@llamaindex/huggingface";
import { SimpleDirectoryReader } from "@llamaindex/readers/directory";
......@@ -115,10 +115,8 @@ The total budget for the City and County of San Francisco for the fiscal year 20
If you prefer more flexibility and don't mind additional complexity, you can create a `QueryEngineTool`. This approach allows you to define the query logic, providing a more tailored way to interact with the data, but note that it introduces a delay due to the extra tool call.
```javascript
const queryEngine = await index.asQueryEngine({ retriever });
const tools = [
new QueryEngineTool({
queryEngine: queryEngine,
index.queryTool({
metadata: {
name: "san_francisco_budget_tool",
description: `This tool can answer detailed questions about the individual components of the budget of San Francisco in 2023-2024.`,
......@@ -127,11 +125,9 @@ const tools = [
];
// Create an agent using the tools array
const agent = new OpenAIAgent({ tools });
const myAgent = agent({ tools });
let toolResponse = await agent.chat({
message: "What's the budget of San Francisco in 2023-2024?",
});
let toolResponse = await myAgent.run("What's the budget of San Francisco in 2023-2024?");
console.log(toolResponse);
```
......
......@@ -7,14 +7,13 @@ In [our third iteration of the agent](https://github.com/run-llama/ts-agents/blo
```javascript
// define the query engine as a tool
const tools = [
new QueryEngineTool({
queryEngine: queryEngine,
index.queryTool({
metadata: {
name: "san_francisco_budget_tool",
description: `This tool can answer detailed questions about the individual components of the budget of San Francisco in 2023-2024.`,
},
}),
FunctionTool.from(sumNumbers, {
tool({
name: "sumNumbers",
description: "Use this function to sum two numbers",
parameters: z.object({
......@@ -25,14 +24,15 @@ const tools = [
description: "Second number to sum",
}),
}),
execute: ({ a, b }) => `${a + b}`,
}),
];
```
You can also use JSON Schema to define the tool parameters as an alternative to Zod.
You can also use JSON Schema to define the tool parameters as an alternative to Zod.
```javascript
FunctionTool.from(sumNumbers, {
tool(sumNumbers, {
name: "sumNumbers",
description: "Use this function to sum two numbers",
parameters: {
......@@ -56,22 +56,13 @@ FunctionTool.from(sumNumbers, {
These tool descriptions are identical to the ones we previously defined. Now let's ask it 3 questions in a row:
```javascript
let response = await agent.chat({
message:
"What's the budget of San Francisco for community health in 2023-24?",
});
let response = await agent.run("What's the budget of San Francisco for community health in 2023-24?");
console.log(response);
let response2 = await agent.chat({
message:
"What's the budget of San Francisco for public protection in 2023-24?",
});
let response2 = await agent.run("What's the budget of San Francisco for public protection in 2023-24?");
console.log(response2);
let response3 = await agent.chat({
message:
"What's the combined budget of San Francisco for community health and public protection in 2023-24?",
});
let response3 = await agent.run("What's the combined budget of San Francisco for community health and public protection in 2023-24?");
console.log(response3);
```
......
......@@ -29,6 +29,8 @@ Note: calling the `bind` method will return a new `FunctionTool` instance, witho
Example to pass a `userToken` as additional argument:
```ts
import { agent, tool } from "llamaindex";
// first arg is LLM input, second is bound arg
const queryKnowledgeBase = async ({ question }, { userToken }) => {
const response = await fetch(`https://knowledge-base.com?token=${userToken}&query=${question}`);
......@@ -36,7 +38,7 @@ const queryKnowledgeBase = async ({ question }, { userToken }) => {
};
// define tool as usual
const kbTool = FunctionTool.from(queryKnowledgeBase, {
const kbTool = tool(queryKnowledgeBase, {
name: 'queryKnowledgeBase',
description: 'Query knowledge base',
parameters: z.object({
......@@ -48,7 +50,7 @@ const kbTool = FunctionTool.from(queryKnowledgeBase, {
// create an agent
const additionalArg = { userToken: 'abcd1234' };
const kbAgent = new LLMAgent({
const workflow = agent({
tools: [kbTool.bind(additionalArg)],
// llm, systemPrompt etc
})
......
import { FunctionTool } from "llamaindex";
import { tool } from "llamaindex";
import { z } from "zod";
export const getCurrentIDTool = FunctionTool.from(
() => {
export const getCurrentIDTool = tool({
name: "get_user_id",
description: "Get a random user id",
parameters: z.object({}),
execute: () => {
console.log("Getting user id...");
return crypto.randomUUID();
},
{
name: "get_user_id",
description: "Get a random user id",
},
);
});
export const getUserInfoTool = FunctionTool.from(
({ userId }: { userId: string }) => {
console.log("Getting user info...", userId);
return `Name: Alex; Address: 1234 Main St, CA; User ID: ${userId}`;
},
{
name: "get_user_info",
description: "Get user info",
parameters: z.object({
userId: z.string().describe("The user id"),
}),
},
);
export const getUserInfoTool = tool({
name: "get_user_info",
description: "Get user info",
parameters: z.object({
userId: z.string().describe("The user id"),
}),
execute: ({ userId }) =>
`Name: Alex; Address: 1234 Main St, CA; User ID: ${userId}`,
});
export const getWeatherTool = FunctionTool.from(
({ address }: { address: string }) => {
console.log("Getting weather...", address);
return `${address} is in a sunny location!`;
},
{
name: "get_weather",
description: "Get the current weather for a location",
parameters: z.object({
address: z.string().describe("The address"),
}),
},
);
export const getWeatherTool = tool({
name: "get_weather",
description: "Get the current weather for a location",
parameters: z.object({
address: z.string().describe("The address"),
}),
execute: ({ address }) => `${address} is in a sunny location!`,
});
import { ollama } from "@llamaindex/ollama";
import { agent } from "llamaindex";
import { getWeatherTool } from "../agent/utils/tools";
async function main() {
const myAgent = agent({
tools: [getWeatherTool],
verbose: false,
llm: ollama({ model: "granite3.2:2b" }),
});
const sfResult = await myAgent.run(
"What's the weather like in San Francisco?",
);
// The weather in San Francisco, CA is currently sunny.
console.log(`${JSON.stringify(sfResult, null, 2)}`);
// Reuse the context from the previous run
const caResult = await myAgent.run("Compare it with California?");
// Both San Francisco and California are currently experiencing sunny weather.
console.log(`${JSON.stringify(caResult, null, 2)}`);
}
main().catch(console.error);
import { Anthropic, AnthropicAgent } from "@llamaindex/anthropic";
import { FunctionTool, Settings } from "llamaindex";
import { anthropic } from "@llamaindex/anthropic";
import { agent, tool } from "llamaindex";
import { z } from "zod";
import { WikipediaTool } from "../wiki";
Settings.callbackManager.on("llm-tool-call", (event) => {
console.log("llm-tool-call", event.detail.toolCall);
});
const anthropic = new Anthropic({
apiKey: process.env.ANTHROPIC_API_KEY,
model: "claude-3-7-sonnet",
});
const agent = new AnthropicAgent({
llm: anthropic,
const workflow = agent({
tools: [
FunctionTool.from(
(query) => {
return `The weather in ${query.location} is sunny`;
},
{
name: "weather",
description: "Get the weather",
parameters: z.object({
location: z.string().describe("The location to get the weather for"),
}),
},
),
tool({
name: "weather",
description: "Get the weather",
parameters: z.object({
location: z.string().describe("The location to get the weather for"),
}),
execute: ({ location }) => `The weather in ${location} is sunny`,
}),
new WikipediaTool(),
],
llm: anthropic({
apiKey: process.env.ANTHROPIC_API_KEY,
model: "claude-3-7-sonnet",
}),
});
async function main() {
const { message } = await agent.chat({
message:
"What is the weather in New York? What's the history of New York from Wikipedia in 3 sentences?",
});
console.log(message.content);
const result = await workflow.run(
"What is the weather in New York? What's the history of New York from Wikipedia in 3 sentences?",
);
console.log(result.data);
}
void main();
import { Gemini, GEMINI_MODEL } from "@llamaindex/google";
import { FunctionTool, LLMAgent, Settings } from "llamaindex";
import { gemini, GEMINI_MODEL } from "@llamaindex/google";
import { agent, tool } from "llamaindex";
import { z } from "zod";
Settings.callbackManager.on("llm-tool-call", (event) => {
console.log(event.detail);
const sumNumbers = tool({
name: "sumNumbers",
description: "Use this function to sum two numbers",
parameters: z.object({
a: z.number().describe("The first number"),
b: z.number().describe("The second number"),
}),
execute: ({ a, b }) => `${a + b}`,
});
Settings.callbackManager.on("llm-tool-result", (event) => {
console.log(event.detail);
const divideNumbers = tool({
name: "divideNumbers",
description: "Use this function to divide two numbers",
parameters: z.object({
a: z.number().describe("The dividend a to divide"),
b: z.number().describe("The divisor b to divide by"),
}),
execute: ({ a, b }) => `${a / b}`,
});
const sumNumbers = FunctionTool.from(
({ a, b }: { a: number; b: number }) => `${a + b}`,
{
name: "sumNumbers",
description: "Use this function to sum two numbers",
parameters: z.object({
a: z.number().describe("The first number"),
b: z.number().describe("The second number"),
}),
},
);
const divideNumbers = FunctionTool.from(
({ a, b }: { a: number; b: number }) => `${a / b}`,
{
name: "divideNumbers",
description: "Use this function to divide two numbers",
parameters: z.object({
a: z.number().describe("The dividend a to divide"),
b: z.number().describe("The divisor b to divide by"),
}),
},
);
const subtractNumbers = FunctionTool.from(
({ a, b }: { a: number; b: number }) => `${a - b}`,
{
name: "subtractNumbers",
description: "Use this function to subtract two numbers",
parameters: z.object({
a: z.number().describe("The number to subtract from"),
b: z.number().describe("The number to subtract"),
}),
},
);
const subtractNumbers = tool({
name: "subtractNumbers",
description: "Use this function to subtract two numbers",
parameters: z.object({
a: z.number().describe("The number to subtract from"),
b: z.number().describe("The number to subtract"),
}),
execute: ({ a, b }) => `${a - b}`,
});
async function main() {
const gemini = new Gemini({
model: GEMINI_MODEL.GEMINI_PRO,
});
const agent = new LLMAgent({
llm: gemini,
const myAgent = agent({
tools: [sumNumbers, divideNumbers, subtractNumbers],
llm: gemini({ model: GEMINI_MODEL.GEMINI_PRO }),
});
const response = await agent.chat({
message: "How much is 5 + 5? then divide by 2 then subtract 1",
});
console.log(response.message);
const result = await myAgent.run(
"How much is 5 + 5? then divide by 2 then subtract 1",
);
console.log(result.data);
}
void main().then(() => {
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment