* implement thread forking feature

* rename thread based on forked message

* refactor bulk message create for thread fork + bump prisma version

* revert prisma version bump

* add todo to bulkCreate function in workspace chats

* cast user input to expected type to prevent prisma injection

* refactor: update order of ops for thread fork

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

* WIP replace langchain pdfloader with pdfjs and add more context to each page

* remove extras from pdfjs and just replace langchain library

* remove unneeded dep

* fix console log in docs

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

implement custom confluence loader to extract code blocks properly from documents

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

patch website depth data connector to work for other links that are not root url

Max. number of context snippets increased from 12 to 200. 
TODO: May remove restraint entirely

* Patch Embed styles with prefixing

* forgot files

Add type defs to helpers

fix query mode always responding with refusal message on develop api openai compatible endpoints

resolves #1777

WIP on migration to @lancedb/lancedb NodeJS SDK

patch parseChunkHeader bug

remove all references to users_can_delete_workspaces option

* add ollama automatic url detection in llm and embedder prefrences

* implement auto detection for lmstudio llm and embedder/improve performance of checking common urls

* fix modal not clearing

* fix lmstudio url check

* improve ux for ollama llm provider option

* improve ux for lm studio llm provider option

* improve ux for ollama embedder option

* improve ux for lmstudio embedder option

* ux improvement lmstudio embedder options

* refactor implementation to hook and use native timeout
Swap to promise.any for resolving of available endpoints

* implement useProviderEndpointAutoDiscovery hook for lmstudio and ollama provider options

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* fix: sanitize filename before writing

Fixes: https://github.com/Mintplex-Labs/anything-llm/issues/1737

* fixup

* fixup

remove azure url validation

* implement generic openai embedding provider

* linting

* comment & description update for generic openai embedding provider

* fix privacy for generic

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* wip bg workers for live document sync

* Add ability to re-embed specific documents across many workspaces via background queue
bgworkser is gated behind expieremental system setting flag that needs to be explictly enabled
UI for watching/unwatching docments that are embedded.
TODO: UI to easily manage all bg tasks and see run results
TODO: UI to enable this feature and background endpoints to manage it

* create frontend views and paths
Move elements to correct experimental scope

* update migration to delete runs on removal of watched document

* Add watch support to YouTube transcripts (#1716)

* Add watch support to YouTube transcripts
refactor how sync is done for supported types

* Watch specific files in Confluence space (#1718)

Add failure-prune check for runs

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* dual build
update copy of alert modals

* update job interval

* Add support for live-sync of Github files

* update copy for document sync feature

* hide Experimental features from UI

* update docs links

* [FEAT] Implement new settings menu for experimental features (#1735)

* implement new settings menu for experimental features

* remove unused context save bar

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* dont run job on boot

* unset workflow changes

* Add persistent encryption service
Relay key to collector so persistent encryption can be used
Encrypt any private data in chunkSources used for replay during resync jobs

* update jsDOC

* Linting and organization

* update modal copy for feature

---------

Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>

auto write ENV on changes
Dont double-track envs that are already tracked

* WIP workspace threads developer api endpoints

* fix swagger docs

* fix broken endpoints and test on single/multi user modes

* support for chatting with workspace thread by userId

* update swagger json

* log query refusals to workspace chats but hide in ui

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* ru language added

* uncheck bad changes
Use chatGPT to create new dictionary for RU translation
turn off debug mode in produ for i18n
patch issue with null comparison in verifyTranslation.js

---------

Co-authored-by: UUSR <konstantik@gmail.com>

resolves #1367

add support for claude sonnet 3.5 model

* Init support of i18n and English and mandarin

* Update common.js (#1320)

* add General Appearance and Chat setting zh translate (#1414)

* add config zh translate (#1461)

* patch some translation pages

* Update locality fixes

* update: complete login page Mandarin translation. (#1709)

update: complete Mandarin translation.

* complete translation

* update github to run validator

* bump to test workflow failure

* bump to fix tests

* update workflow

* refactor lang selector support

* add Spanish and French

* add dictionaries

---------

Co-authored-by: GetOffer.help <13744916+getofferhelp@users.noreply.github.com>
Co-authored-by: AIR <129256286+KochabStar@users.noreply.github.com>
Co-authored-by: Ezio T <ezio5600@gmail.com>