* set message limit per user

* remove old limit user messages + unused admin page

* fix daily message validation

* refactor message limit input
refactor canSendChat on user to a method on user model

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* initial commit for chrome extension

* wip browser extension backend

* wip frontend browser extension settings

* fix typo for browserExtension route

* implement verification codes + frontend panel for browser extension keys

* reorganize + state management for all connection states

* implement embed to workspace

* add send page to anythingllm extension option + refactor

* refactor connection string auth + update context menus + organize background.js into models

* popup extension from main app and save if successful

* fix hebrew translation misspelling

* fetch custom logo inside chrome extension

* delete api keys on disconnect of extension

* use correct apiUrl constant in frontend + remove unneeded comments

* remove upload-link endpoint and send inner text html to raw text collector endpoint

* update readme

* fix readme link

* fix readme typo

* update readme

* handle deletion of browser keys with key id and DELETE endpoint

* move event string to constant

* remove tablename and writable fields from BrowserExtensionApiKey backend model

* add border-none to all buttons and inputs for desktop compatibility

* patch prisma injections

* update delete endpoints to delete keys by id

* remove unused prop

* add button to attempt browser extension connection + remove max active keys

* wip multi user mode support

* multi user mode support

* clean up backend + show created by in frotend browser extension page

* show multi user warning message on key creation + hide context menus when no workspaces

* show browser extension options to managers

* small backend changes and refactors

* extension cleanup

* rename submodule

* extension updates & docs

* dev docker build

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>

* Refactor api endpoint chat handler to its own function
remove legacy `chatWithWorkspace` and cleanup `index.js`

* Add `sessionId` in dev API to partition chats logically statelessly

remove all references to users_can_delete_workspaces option

* wip bg workers for live document sync

* Add ability to re-embed specific documents across many workspaces via background queue
bgworkser is gated behind expieremental system setting flag that needs to be explictly enabled
UI for watching/unwatching docments that are embedded.
TODO: UI to easily manage all bg tasks and see run results
TODO: UI to enable this feature and background endpoints to manage it

* create frontend views and paths
Move elements to correct experimental scope

* update migration to delete runs on removal of watched document

* Add watch support to YouTube transcripts (#1716)

* Add watch support to YouTube transcripts
refactor how sync is done for supported types

* Watch specific files in Confluence space (#1718)

Add failure-prune check for runs

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* dual build
update copy of alert modals

* update job interval

* Add support for live-sync of Github files

* update copy for document sync feature

* hide Experimental features from UI

* update docs links

* [FEAT] Implement new settings menu for experimental features (#1735)

* implement new settings menu for experimental features

* remove unused context save bar

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* dont run job on boot

* unset workflow changes

* Add persistent encryption service
Relay key to collector so persistent encryption can be used
Encrypt any private data in chunkSources used for replay during resync jobs

* update jsDOC

* Linting and organization

* update modal copy for feature

---------

Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>

* WIP slash presets

* WIP slash command customization CRUD + validations complete

* backend slash command support

* fix permission setting on new slash commands
rework form submit and pattern on frontend

* Add field updates for hooks,
required=true to field
add user<>command constraint to keep them unique
enforce uniquness via teritary uid field on table for multi and non-multi user

* reset migration

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* Customize refusal response for `query` mode

* remove border for desktop

* WIP new login screen UI

* update prisma schema/create new models for pw recovery

* WIP password recovery backend

* WIP reset password flow

* WIP pw reset flow

* password reset logic complete & functional UI

* WIP login screen redesign for single and multi user

* create placeholder modal to display recovery codes

* implement UI for recovery code modals/download recovery codes

* multiuser desktop password reset UI/functionality complete

* support single user mode for pw reset

* mobile styles for all password reset/login flows complete

* lint

* remove single user password recovery

* create PasswordRecovery util file to make more readable

* do not drop-replace users table in migration

* review pr

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

V1 of agent support via built-in `@agent` that can be invoked alongside normal workspace RAG chat.

* Enable per-workspace provider/model combination

* cleanup

* remove resetWorkspaceChatModels and wipeWorkspaceModelPreference to prevent workspace from resetting model

* add space

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>

* WIP workspace pfp, CRUD functions complete

* implement fetching workspace pfp in UserIcon component

* update UI for workspace settings pfp

* minor css refactor

* WIP fixes to workspace pfp

* create responseCache for workspace pfp blob to improve performance

* fix cache not clearing when removing workspace pfp and remove unneeded util

* load workspace image once, dont reload

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* Enable ability to do full-text query on documents
Show alert modal on first pin for client
Add ability to use pins in stream/chat/embed

* typo and copy update

* simplify spread of context and sources

Remove useless icons in prompt bar
Add chatMode column to workspaces that defaults to chat
Add UI for toggle of chat mode with hint
Update UI for workspace settings to match designs

* WIP RLHF works on historical messages

* refactor Actions component

* completed RLHF up and down votes for chats

* add defaults for HistoricalMessage params

* refactor RLHF implmenation
remove forwardRef on history items to prevent rerenders

* remove dup id

* Add rating to CSV output

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* Implement workspace thread that is compatible with legacy versions

* last touches

* comment on chat qty enforcement

* WIP event logging - new table for events and new settings view for viewing

* WIP add logging

* UI for log rows

* rename files to Logging to prevent getting gitignore

* add metadata for all logging events and colored badges in logs page

* remove unneeded comment

* cleanup namespace for logging

* clean up backend calls

* update logging to show to => from settings changes

* add logging for invitations, created, deleted, and accepted

* add logging for user created, updated, suspended, or removed

* add logging for workspace deleted

* add logging for chat logs exported

* add logging for API keys, LLM, embedder, vector db, embed chat, and reset button

* modify event logs

* update to event log types

* simplify rendering of event badges

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* create custom suggested chat messages per workspace

* update how suggestedChats are passed to chat window

* update mobile styles

* update edit change handler

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* WIP embedded app

* WIP got response from backend in embedded app

* WIP streaming prints to embedded app

* implemented streaming and tailwind min for styling into embedded app

* WIP embedded app history functional

* load params from script tag into embedded app

* rough in modularization of embed chat
cleanup dev process for easier dev support
move all chat to components
todo: build process
todo: backend support

* remove eslint config

* Implement models and cleanup embed chat endpoints
Improve build process for embed
prod minification and bundle size awareness
WIP

* forgot files

* rename to embed folder

* introduce chat modal styles

* add middleware validations on embed chat

* auto open param and default greeting

* reset chat history

* Admin embed config page

* Admin Embed Chats mgmt page

* update embed

* nonpriv

* more style support
reopen if chat was last opened

* update comments

* remove unused imports

* allow change of workspace for embedconfig

* update failure to lookup message

* update reset script

* update instructions

* Add more styling options
Add sponsor text at bottom
Support dynamic container height
Loading animations

* publish new embed script

* Add back syntax highlighting and keep bundle small via dynamic script build

* add hint

* update readme

* update copy model for snippet with link to styles

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* create configurable topN per workspace

* Update TopN UI text
Fix fallbacks for all providers
Add SQLite CHECK to TOPN value

* merge with master
Update zilliz provider for variable TopN

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* WIP model selection per workspace (migrations and openai saves properly

* revert OpenAiOption

* add support for models per workspace for anthropic, localAi, ollama, openAi, and togetherAi

* remove unneeded comments

* update logic for when LLMProvider is reset, reset Ai provider files with master

* remove frontend/api reset of workspace chat and move logic to updateENV
add postUpdate callbacks to envs

* set preferred model for chat on class instantiation

* remove extra param

* linting

* remove unused var

* refactor chat model selection on workspace

* linting

* add fallback for base path to localai models

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* WIP onboarding v2

* Welcome screen for onboarding complete

* fix home page and WIP create skeleton for llm preference search/options

* render llms as options

* add search functionality to llm preference & add survey step

* fix openai settings undefined & create custom logo onboarding page

* add user setup UI

* add data handling & privacy onboarding screen

* add create workspace onboarding screen

* fix survey width in onboarding

* create vector database connection onboarding page

* add workspace image & all skeleton ui complete

* fix navigation buttons and ui tweaks to fit on screen

* WIP LLMPreference

* LLM Preference screen fully functional

* create components for vector db options and fix styling of azure options

* remove unneeded comment

* vector db connection onboarding screen complete

* minor ui tweak to searchbar

* user setup page fully working

* create workspace onboarding page fully working

* useNavigate for navigation between pages

* mobile layout, cleanup old files, survey functionality implemented

* fix default logo appearing when should be blank & password setup bug fix

* Modify flow of onboarding
todo: embedding set up

* Add embedder setup screen & insert into flow

* update embedding back button
auto-dismiss toasts on each step

* move page defs under imports
fix bg color on mobile styling

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>

* fix sizing of onboarding modals & lint

* fix extra scrolling on mobile onboarding flow

* added message to use desktop for onboarding

* linting

* add arrow to scroll to bottom (debounced) and fix chat scrolling to always scroll to very bottom on message history change

* fix for empty chat

* change mobile alert copy

* WIP adding PFP upload support

* WIP pfp for users

* edit account menu complete with change username/password and upload profile picture

* add pfp context to update all instances of usePfp hook on update

* linting

* add context for logo change to immediately update logo

* fix div with bullet points to use list-disc instead

* fix: small changes

* update multer file storage locations

* fix: use STORAGE_DIR for filepathing

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* settings for similarity score threshold and prisma schema updated

* prisma schema migration for adding similarityScore setting

* WIP

* Min score default change

* added similarityThreshold checking for all vectordb providers

* linting

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>

* WIP on continuous prompt window summary

* wip

* Move chat out of VDB
simplify chat interface
normalize LLM model interface
have compression abstraction
Cleanup compressor
TODO: Anthropic stuff

* Implement compression for Anythropic
Fix lancedb sources

* cleanup vectorDBs and check that lance, chroma, and pinecone are returning valid metadata sources

* Resolve Weaviate citation sources not working with schema

* comment cleanup

* WIP converted all sqlite models into prisma calls

* modify db setup and fix ApiKey model calls in admin.js

* renaming function params to be consistent

* converted adminEndpoints to utilize prisma orm

* converted chatEndpoints to utilize prisma orm

* converted inviteEndpoints to utilize prisma orm

* converted systemEndpoints to utilize prisma orm

* converted workspaceEndpoints to utilize prisma orm

* converting sql queries to prisma calls

* fixed default param bug for orderBy and limit

* fixed typo for workspace chats

* fixed order of deletion to account for sql relations

* fix invite CRUD and workspace management CRUD

* fixed CRUD for api keys

* created prisma setup scripts/docs for understanding how to use prisma

* prisma dependency change

* removing unneeded console.logs

* removing unneeded sql escape function

* linting and creating migration script

* migration from depreciated sqlite script update

* removing unneeded migrations in prisma folder

* create backup of old sqlite db and use transactions to ensure all operations complete successfully

* adding migrations to gitignore

* updated PRISMA.md docs for info on how to use sqlite migration script

* comment changes

* adding back migrations folder to repo

* Reviewing SQL and prisma integraiton on fresh repo

* update inline key replacement

* ensure migration script executes and maps foreign_keys regardless of db ordering

* run migration endpoint

* support new prisma backend

* bump version

* change migration call

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>