-
- Downloads
[Feature] AnythingLLM use locally hosted Llama.cpp and GGUF files for inferencing (#413)
* Implement use of native embedder (all-Mini-L6-v2) stop showing prisma queries during dev * Add native embedder as an available embedder selection * wrap model loader in try/catch * print progress on download * add built-in LLM support (expiermental) * Update to progress output for embedder * move embedder selection options to component * saftey checks for modelfile * update ref * Hide selection when on hosted subdomain * update documentation hide localLlama when on hosted * saftey checks for storage of models * update dockerfile to pre-build Llama.cpp bindings * update lockfile * add langchain doc comment * remove extraneous --no-metal option * Show data handling for private LLM * persist model in memory for N+1 chats * update import update dev comment on token model size * update primary README * chore: more readme updates and remove screenshots - too much to maintain, just use the app! * remove screeshot link
Showing
- README.md 16 additions, 12 deletionsREADME.md
- docker/Dockerfile 6 additions, 1 deletiondocker/Dockerfile
- frontend/src/components/LLMSelection/NativeLLMOptions/index.jsx 84 additions, 0 deletions...nd/src/components/LLMSelection/NativeLLMOptions/index.jsx
- frontend/src/components/PrivateRoute/index.jsx 4 additions, 6 deletionsfrontend/src/components/PrivateRoute/index.jsx
- frontend/src/pages/GeneralSettings/LLMPreference/index.jsx 15 additions, 0 deletionsfrontend/src/pages/GeneralSettings/LLMPreference/index.jsx
- frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/DataHandling/index.jsx 7 additions, 0 deletions...boardingFlow/OnboardingModal/Steps/DataHandling/index.jsx
- frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/LLMSelection/index.jsx 11 additions, 0 deletions...boardingFlow/OnboardingModal/Steps/LLMSelection/index.jsx
- images/screenshots/SCREENSHOTS.md 0 additions, 18 deletionsimages/screenshots/SCREENSHOTS.md
- images/screenshots/document.png 0 additions, 0 deletionsimages/screenshots/document.png
- images/screenshots/home.png 0 additions, 0 deletionsimages/screenshots/home.png
- images/screenshots/llm_selection.png 0 additions, 0 deletionsimages/screenshots/llm_selection.png
- images/screenshots/uploading_doc.gif 0 additions, 0 deletionsimages/screenshots/uploading_doc.gif
- images/screenshots/vector_databases.png 0 additions, 0 deletionsimages/screenshots/vector_databases.png
- server/models/systemSettings.js 7 additions, 2 deletionsserver/models/systemSettings.js
- server/package.json 3 additions, 2 deletionsserver/package.json
- server/storage/models/README.md 24 additions, 4 deletionsserver/storage/models/README.md
- server/utils/AiProviders/native/index.js 196 additions, 0 deletionsserver/utils/AiProviders/native/index.js
- server/utils/chats/stream.js 30 additions, 0 deletionsserver/utils/chats/stream.js
- server/utils/helpers/customModels.js 23 additions, 1 deletionserver/utils/helpers/customModels.js
- server/utils/helpers/index.js 3 additions, 0 deletionsserver/utils/helpers/index.js
Loading
Please register or sign in to comment