Skip to content
Snippets Groups Projects
Unverified Commit 439e1b67 authored by Logan's avatar Logan Committed by GitHub
Browse files

update MyMagicAI (#11263) 

* update

* address comments
parent 9a13deea
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
docs/examples/llm/mymagic.ipynb
+ 19
− 15
View file @ 439e1b67
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# MyMagic AI LLM # MyMagic AI LLM
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Introduction ## Introduction
This notebook demonstrates how to use MyMagicAI for batch inference on massive data stored in cloud buckets. The only enpoints implemented are `complete` and `acomplete` which can work on many use cases including Completion, Summariation and Extraction. This notebook demonstrates how to use MyMagicAI for batch inference on massive data stored in cloud buckets. The only enpoints implemented are `complete` and `acomplete` which can work on many use cases including Completion, Summariation and Extraction.
To use this notebook, you need an API key (Personal Access Token) from MyMagicAI and data stored in cloud buckets. To use this notebook, you need an API key (Personal Access Token) from MyMagicAI and data stored in cloud buckets.
Sign up by clicking Get Started at [MyMagicAI's website](https://mymagic.ai/) to get your API key. Sign up by clicking Get Started at [MyMagicAI's website](https://mymagic.ai/) to get your API key.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Setup ## Setup
To set up your bucket and grant MyMagic API a secure access to your cloud storage, please visit [MyMagic docs](https://docs.mymagic.ai/) for reference. To set up your bucket and grant MyMagic API a secure access to your cloud storage, please visit [MyMagic docs](https://docs.mymagic.ai/) for reference.
If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙. If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%pip install llama-index-llms-mymagic %pip install llama-index-llms-mymagic
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
!pip install llama-index !pip install llama-index
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from llama_index.llms.mymagic import MyMagicAI from llama_index.llms.mymagic import MyMagicAI
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
llm = MyMagicAI( llm = MyMagicAI(
api_key="your_api_key", api_key="your-api-key",
storage_provider="your_storage_provider", # s3, gcs storage_provider="s3", # s3, gcs
bucket_name="your_bucket_name", bucket_name="your-bucket-name",
session="your_session", # files should be located in this folder on which batch inference will be run session="your-session-name", # files should be located in this folder on which batch inference will be run
system_prompt="Answer the question succinctly", role_arn="your-role-arn",
system_prompt="your-system-prompt",
) )
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
resp = llm.complete( resp = llm.complete(
question="Summarize the document!", question="your-question",
model="mistral7b", model="chhoose-model", # currently we support mistral7b, llama7b, mixtral8x7b,codellama70b, llama70b, more to come...
max_tokens=10, # currently we support mistral7b, llama7b, mixtral8x7b,codellama70b, llama70b, more to come... max_tokens=5, # number of tokens to generate, default is 10
) )
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# The response indicated that the final output is stored in your bucket or raises an exception if the job failed # The response indicated that the final output is stored in your bucket or raises an exception if the job failed
print(resp) print(resp)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Asynchronous Requests by using `acomplete` endpoint ## Asynchronous Requests by using `acomplete` endpoint
For asynchronous operations, use the following approach. For asynchronous operations, use the following approach.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import asyncio import asyncio
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
async def main(): async def main():
allm = MyMagicAI( allm = MyMagicAI(
api_key="your_api_key", api_key="your-api-key",
storage_provider="your_storage_provider", storage_provider="s3", # s3, gcs
bucket_name="your_bucket_name", bucket_name="your-bucket-name",
session="your_session_name", session="your-session-name", # files should be located in this folder on which batch inference will be run
system_prompt="your_system_prompt", role_arn="your-role-arn",
system_prompt="your-system-prompt",
) )
response = await allm.acomplete( response = await allm.acomplete(
question="your_question", model="mistral7b", max_tokens=10 question="your-question",
model="chhoose-model", # currently we support mistral7b, llama7b, mixtral8x7b,codellama70b, llama70b, more to come...
max_tokens=5, # number of tokens to generate, default is 10
) )
print("Async completion response:", response) print("Async completion response:", response)
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
asyncio.run(main()) await main()
``` ```
......
llama-index-integrations/llms/llama-index-llms-mymagic/llama_index/llms/mymagic/base.py
+ 19
− 6
View file @ 439e1b67
...@@ -23,6 +23,7 @@ class MyMagicAI(LLM): ...@@ -23,6 +23,7 @@ class MyMagicAI(LLM):
max_tokens: int = Field( max_tokens: int = Field(
default=10, description="The maximum number of tokens to generate." default=10, description="The maximum number of tokens to generate."
) )
question = Field(default="", description="The user question.")
storage_provider: str = Field( storage_provider: str = Field(
default="gcs", description="The storage provider to use." default="gcs", description="The storage provider to use."
) )
...@@ -105,11 +106,17 @@ class MyMagicAI(LLM): ...@@ -105,11 +106,17 @@ class MyMagicAI(LLM):
return resp.json() return resp.json()
async def acomplete( async def acomplete(
self, question: str, model: str, max_tokens: int, poll_interval: float = 1.0 self,
question: str,
model: Optional[str] = None,
max_tokens: Optional[int] = None,
poll_interval: float = 1.0,
) -> CompletionResponse: ) -> CompletionResponse:
self.question_data["question"] = question self.question_data["question"] = question
self.question_data["model"] = model self.model = self.question_data["model"] = model or self.model
self.question_data["max_tokens"] = max_tokens self.max_tokens = self.question_data["max_tokens"] = (
max_tokens or self.max_tokens
)
task_response = await self._submit_question(self.question_data) task_response = await self._submit_question(self.question_data)
task_id = task_response.get("task_id") task_id = task_response.get("task_id")
...@@ -120,11 +127,17 @@ class MyMagicAI(LLM): ...@@ -120,11 +127,17 @@ class MyMagicAI(LLM):
await asyncio.sleep(poll_interval) await asyncio.sleep(poll_interval)
def complete( def complete(
self, question: str, model: str, max_tokens: int, poll_interval: float = 1.0 self,
question: str,
model: Optional[str] = None,
max_tokens: Optional[int] = None,
poll_interval: float = 1.0,
) -> CompletionResponse: ) -> CompletionResponse:
self.question_data["question"] = question self.question_data["question"] = question
self.question_data["model"] = model self.model = self.question_data["model"] = model or self.model
self.question_data["max_tokens"] = max_tokens self.max_tokens = self.question_data["max_tokens"] = (
max_tokens or self.max_tokens
)
task_response = self._submit_question_sync(self.question_data) task_response = self._submit_question_sync(self.question_data)
task_id = task_response.get("task_id") task_id = task_response.get("task_id")
......
llama-index-integrations/llms/llama-index-llms-mymagic/pyproject.toml
+ 1
− 1
View file @ 439e1b67
...@@ -26,7 +26,7 @@ description = "llama-index llms mymagic integration" ...@@ -26,7 +26,7 @@ description = "llama-index llms mymagic integration"
license = "MIT" license = "MIT"
name = "llama-index-llms-mymagic" name = "llama-index-llms-mymagic"
readme = "README.md" readme = "README.md"
version = "0.1.0" version = "0.1.1"
[tool.poetry.dependencies] [tool.poetry.dependencies]
python = ">=3.8.1,<3.12" python = ">=3.8.1,<3.12"
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

:feet::heart: Strive to be the person your dogs believe you are :heart::feet: