* feat: add support for voyage-3-large and voyage-code-3 embedding models

- Add voyage-3-large and voyage-code-3 to VoyageAiOptions dropdown
- Update getMaxEmbeddingLength to support 32k context for new models
- Update .env.example with new model options

* unset env example

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

resolves #2943

update example env files

* Reranker WIP

* add cacheing and singleton loading

* Add field to workspaces for vectorSearchMode
Add UI for lancedb to change mode
update all search endpoints to pass in reranker prop if provider can use it

* update hint text

* When reranking, swap score to rerank score

* update optchain

* make `similarityResponse` object arguments and not positional

* reuse client for qdrant

floor ms value

* Add support for Google Generative AI (Gemini) embedder

* Add missing example in docker
Fix UI key elements in options
Add Gemini to data handling section
Patch issues with chunk handling during embedding

* remove dupe in env

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* prevent non admin users from changing llm settings via custom-models endpoint

* permission middleware to JSDOC

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* add audio file validations

* patch sharp to support wavfile parsing

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* patch unauthorized viewing of other user's pfps

* inline return responses

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

* normalize paths on files uploaded to prevent arbitrary file writes

* force normalize path in string parse

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

fix typo in frontend for SearXNG agent setup

show scrollbar on rendered tables in chat

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

connect #2898

fix styles of ChangeWarningModal

* fix light mode table colors

* fix table styles from being overwritten in other sections that are not chat messages

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

fixed wrong metadata assignment

remove fine-tuning flow

* Add support for gemini authenticated models endpoint
add customModels entry
add un-authed fallback to default listing
separate models by expiermental status
resolves #2866

* add back improved logic for apiVersion decision making

* Enable default users to be able to add attachments, but not files to chats

* add writible fields to dev api new workspace endpoint

* lint

* implement validations for workspace model

* update swagger comments

* simplify validations for workspace on frontend and API

* cleanup validations

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

* WIP performance metric tracking

* fix: patch UI trying to .toFixed() null metric
Anthropic tracking migraiton
cleanup logs

* Apipie implmentation, not tested

* Cleanup Anthropic notes, Add support for AzureOpenAI tracking

* bedrock token metric tracking

* Cohere support

* feat: improve default stream handler to track for provider who are actually OpenAI compliant in usage reporting
add deepseek support

* feat: Add FireworksAI tracking reporting
fix: improve handler when usage:null is reported (why?)

* Add token reporting for GenericOpenAI

* token reporting for koboldcpp + lmstudio

* lint

* support Groq token tracking

* HF token tracking

* token tracking for togetherai

* LiteLLM token tracking

* linting + Mitral token tracking support

* XAI token metric reporting

* native provider runner

* LocalAI token tracking

* Novita token tracking

* OpenRouter token tracking

* Apipie stream metrics

* textwebgenui token tracking

* perplexity token reporting

* ollama token reporting

* lint

* put back comment

* Rip out LC ollama wrapper and use official library

* patch images with new ollama lib

* improve ollama offline message

* fix image handling in ollama llm provider

* lint

* NVIDIA NIM token tracking

* update openai compatbility responses

* UI/UX show/hide metrics on click for user preference

* update bedrock client

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>

* wip remove all docs clear vector db on embedder/vector db change

* purge all cached docs and remove docs from workspaces on vectordb/embedder change

* lint

* remove unneeded console log

* remove reset vector stores endpoint and move to server side updateENV with postUpdate check

* reset embed module

* remove unused import

* simplify deletion process
rescoped document deletion to be more general for speed, everything needs to be reset anyway
fixed issue where unembedded docs not in any workspaces, but cached, were not removed

* add back missing readme file
update warning text modals

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* added attachments to genericopenai prompt

* add devnote

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

deprecate old endpoint which users workspace ID vs slug
resolves #2838

add thread name to workspaces get endpoint

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

* fix apipie streaming/sort by chat models

* lint

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>