* OCR PDFs as fallback in spawn thread

* wip

* build our own worker fanout and wrapper

* norm pkgs

* Add image OCR support

* OCR PDFs as fallback in spawn thread

* wip

* build our own worker fanout and wrapper

* norm pkgs

* bump dev

* Add `querySelectorAll` capability to web-scraping block

* patches and fallbacks

* fix styles of text in web scraping block

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>

* chore: rename `Gitlab` to `GitLab`

Signed-off-by: Adam Setch <adam.setch@outlook.com>

* undo code changes - breaks refs

---------

Signed-off-by: Adam Setch <adam.setch@outlook.com>
Co-authored-by: timothycarambat <rambat1010@gmail.com>

resolves #3201

* chore: rename Github to GitHub

Signed-off-by: Adam Setch <adam.setch@outlook.com>

* chore: rename Github to GitHub

Signed-off-by: Adam Setch <adam.setch@outlook.com>

* Undo some code changes for references

---------

Signed-off-by: Adam Setch <adam.setch@outlook.com>
Co-authored-by: timothycarambat <rambat1010@gmail.com>

* wip agent builder

* refactor structure for agent builder

* improve ui for add block menu and sidebar

* lint

* node ui improvement

* handle deleting variable in all nodes

* add headers and body to apiCall node

* lint

* Agent flow builder backend (#3078)

* wip agent builder backend

* save/load agent tasks

* lint

* refactor agent task to use uuids instead of names

* placeholder for run task

* update frontend sidebar + seperate backend to agent-tasks utils

* lint

* add deleting of agent tasks

* create AgentTasks class + wip load agent tasks into aibitat

* lint

* inject + call agent tasks

* wip call agent tasks

* add llm instruction + fix api calling blocks

* add ui + backend for editing/toggling agent tasks

* lint

* add back middlewares

* disable run task + add navigate to home on logo click

* implement normalizePath to prevent path traversal

* wip make api calling more consistent

* lint

* rename all references from task to flow

* patch load flow bug when on editing page

* remove unneeded files/comments

* lint

* fix delete endpoint + rename load flows

* add move block to ui + fix api-call backend + add telemetry

* lint

* add web scraping block

* only allow admin for agent builder

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* Move AgentFlowManager flows to static
simplify UI states
Handle LLM prompt flow when provided non-string

* delete/edit menu for agent flow panel + update flow icon

* lint

* fix open builder button hidden bug

* add tooltips to move up/down block buttons

* add tooltip to delete block

* truncate block description to fit on blocklist component

* light mode agent builder sidebar

* light mode api call block

* fix light mode styles for agent builder blocks

* agent flow fetch in UI

* sync delete flow

* agent flow ui/ux improvements

* remove unused AgentSidebar component

* comment out /run

* UI changes and updates for flow builder

* format flow panel info

* update link handling

* ui tweaks to header menu

* remove unused import

* update doc links
update block icons

* bump readme

* Patch code block header oddity
resolves #3117

* bump dev image

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>

resolves #3117

* Add handling to resoning models for Generic OpenAI connector
resolves #3177

* linting

fix: fix UserMenu rendered twice on Main page

* wip onboarding translations

* add normalized EN reference for languages

* add zh/zh-tw translations

* Fix translations

resolves #3160

Add in-text citations as well for PPLX token streaming
handle timeouts for stream/buffer hanging

* Enable `num_ctx` to match defined chunk length in ollama embedder

* remove console

connect #3023
Note: depends on user naming the deployment correctly.

* Add tokenizer improvments via Singleton class
linting

* dev build

* Estimation fallback when string exceeds a fixed byte size

* Add notice to tiktoken on backend

* wip agent ui animation

* WIP agent ui revision

* linting

* simplify css

* memoize agent responses

* patch hook memo issue

* dev build

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>

* Add ability to disable default agent skills

* debug build