Skip to content
Snippets Groups Projects
Unverified Commit df058910 authored by Logan's avatar Logan Committed by GitHub
Browse files

LLM Docs Nits (#17999)

parent ca631d81
No related branches found
No related tags found
No related merge requests found
Source diff could not be displayed: it is too large. Options to address this: view the blob.
......@@ -21,6 +21,9 @@ export OPENAI_API_KEY=XXXXX
set OPENAI_API_KEY=XXXXX
```
!!! tip
If you are using an OpenAI-Compatible API, you can use the `OpenAILike` LLM class. You can find more information in the [OpenAILike LLM](https://docs.llamaindex.ai/en/stable/api_reference/llms/openai_like/#llama_index.llms.openai_like.OpenAILike) integration.
## Basic Agent Example
Let's start with a simple example using an agent that can perform basic multiplication by calling a tool. Create a file called `starter.py`:
......
......@@ -47,6 +47,8 @@ messages = [
chat_response = llm.chat(messages)
```
`stream_chat` and `astream_chat` are also available.
## Specifying models
Many LLM integrations provide more than one model. You can specify a model by passing the `model` parameter to the LLM constructor:
......@@ -57,6 +59,60 @@ response = llm.complete("Who is Laurie Voss?")
print(response)
```
## Multi-Modal LLMs
Some LLMs support multi-modal chat messages. This means that you can pass in a mix of text and other modalities (images, audio, video, etc.) and the LLM will handle it.
Currently, LlamaIndex supports text, images, and audio inside ChatMessages using content blocks.
```python
from llama_index.core.llms import ChatMessage, TextBlock, ImageBlock
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-4o")
messages = [
ChatMessage(
role="user",
blocks=[
ImageBlock(path="image.png"),
TextBlock(text="Describe the image in a few sentences."),
],
)
]
resp = llm.chat(messages)
print(resp.message.content)
```
## Tool Calling
Some LLMs (OpenAI, Anthropic, Gemini, Ollama, etc.) support tool calling directly over API calls -- this means tools and functions can be called without specific prompts and parsing mechanisms.
```python
from llama_index.core.tools import FunctionTool
from llama_index.llms.openai import OpenAI
def generate_song(name: str, artist: str) -> Song:
"""Generates a song with provided name and artist."""
return {"name": name, "artist": artist}
tool = FunctionTool.from_defaults(fn=generate_song)
llm = OpenAI(model="gpt-4o")
response = llm.predict_and_call(
[tool],
"Pick a random song for me",
)
print(str(response))
```
For more details on even more advanced tool calling, check out the in-depth guide using [OpenAI](../../examples/llm/openai.ipynb). The same approaches work for any LLM that supports tools/functions (e.g. Anthropic, Gemini, Ollama, etc.).
You can learn more about tools and agents in the [tools guide](../../understanding/agent/tools.md).
## Available LLMs
We support integrations with OpenAI, Anthropic, Mistral, DeepSeek, Hugging Face, and dozens more. Check out our [module guide to LLMs](../../module_guides/models/llms.md) for a full list, including how to run a local model.
......
......@@ -27,13 +27,50 @@ class OpenAILike(OpenAI):
OpenAILike is a thin wrapper around the OpenAI model that makes it compatible with
3rd party tools that provide an openai-compatible api.
Currently, llama_index prevents using custom models with their OpenAI class
because they need to be able to infer some metadata from the model name.
NOTE: You still need to set the OPENAI_BASE_API and OPENAI_API_KEY environment
variables or the api_key and api_base constructor arguments.
OPENAI_API_KEY/api_key can normally be set to anything in this case,
but will depend on the tool you're using.
Args:
model (str):
The model to use for the api.
api_base (str):
The base url to use for the api.
Defaults to "https://api.openai.com/v1".
is_chat_model (bool):
Whether the model uses the chat or completion endpoint.
Defaults to False.
is_function_calling_model (bool):
Whether the model supports OpenAI function calling/tools over the API.
Defaults to False.
api_key (str):
The api key to use for the api.
Set this to some random string if your API does not require an api key.
context_window (int):
The context window to use for the api. Set this to your model's context window for the best experience.
Defaults to 3900.
max_tokens (int):
The max number of tokens to generate.
Defaults to None.
temperature (float):
The temperature to use for the api.
Default is 0.1.
additional_kwargs (dict):
Specify additional parameters to the request body.
max_retries (int):
How many times to retry the API call if it fails.
Defaults to 3.
timeout (float):
How long to wait, in seconds, for an API call before failing.
Defaults to 60.0.
reuse_client (bool):
Reuse the OpenAI client between requests.
Defaults to True.
default_headers (dict):
Override the default headers for API requests.
Defaults to None.
http_client (httpx.Client):
Pass in your own httpx.Client instance.
Defaults to None.
async_http_client (httpx.AsyncClient):
Pass in your own httpx.AsyncClient instance.
Defaults to None.
Examples:
`pip install llama-index-llms-openai-like`
......
......@@ -27,7 +27,7 @@ exclude = ["**/BUILD"]
license = "MIT"
name = "llama-index-llms-openai-like"
readme = "README.md"
version = "0.3.3"
version = "0.3.4"
[tool.poetry.dependencies]
python = ">=3.9,<4.0"
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment