-
- Downloads
Adds abstractions for `LlamaDatasets` and downloader (#9165)
* init * add rag dataset * reshape Dataset * use reference context and answers * add RagDatasetGenerator * add serialization * add prediction abstratction and predict method * separate predictions from examples * add download dataset from github logic * remove train test splits * add notebooks * add async for predictions * clewan up base * add model name attr * mark QRD and DatasetGenerator as deprecated * update batch_runner to load asycio_module from async_utils * add kwargs is_dataset flag in download_utils * use pydantic BaseModel instead of dataclass * use pydantic BaseModel for CreatedBy * update nb * fix sync * update nb * point to run-llama llama_datasets repo * update nbs * remove unit test * lint * relax download utils return type * lint * removed data file * add cli command * point to run-llama * make entry to CHANGELOG; add open in colab button
Showing
- CHANGELOG.md 8 additions, 0 deletionsCHANGELOG.md
- docs/examples/llama_datasets/downloading_llama_datasets.ipynb 641 additions, 0 deletions.../examples/llama_datasets/downloading_llama_datasets.ipynb
- docs/examples/llama_datasets/labelled-rag-datasets.ipynb 954 additions, 0 deletionsdocs/examples/llama_datasets/labelled-rag-datasets.ipynb
- docs/examples/llama_datasets/uploading_llama_dataset.ipynb 344 additions, 0 deletionsdocs/examples/llama_datasets/uploading_llama_dataset.ipynb
- llama_index/async_utils.py 11 additions, 0 deletionsllama_index/async_utils.py
- llama_index/command_line/command_line.py 59 additions, 0 deletionsllama_index/command_line/command_line.py
- llama_index/download/download_utils.py 75 additions, 42 deletionsllama_index/download/download_utils.py
- llama_index/evaluation/batch_runner.py 1 addition, 1 deletionllama_index/evaluation/batch_runner.py
- llama_index/evaluation/dataset_generation.py 10 additions, 0 deletionsllama_index/evaluation/dataset_generation.py
- llama_index/evaluation/eval_utils.py 1 addition, 11 deletionsllama_index/evaluation/eval_utils.py
- llama_index/llama_dataset/__init__.py 31 additions, 0 deletionsllama_index/llama_dataset/__init__.py
- llama_index/llama_dataset/base.py 212 additions, 0 deletionsllama_index/llama_dataset/base.py
- llama_index/llama_dataset/download.py 48 additions, 0 deletionsllama_index/llama_dataset/download.py
- llama_index/llama_dataset/generator.py 248 additions, 0 deletionsllama_index/llama_dataset/generator.py
- llama_index/llama_dataset/rag.py 138 additions, 0 deletionsllama_index/llama_dataset/rag.py
Loading
Please register or sign in to comment