-
- Downloads
[FEAT] Website depth scraping data connector (#1191)
* WIP website depth scraping, (sort of works)
* website depth data connector stable + add maxLinks option
* linting + loading small ui tweak
* refactor website depth data connector for stability, speed, & readability
* patch: remove console log
Guard clause on URL validitiy check
reasonable overrides
---------
Co-authored-by:
Timothy Carambat <rambat1010@gmail.com>
Showing
- collector/extensions/index.js 20 additions, 0 deletionscollector/extensions/index.js
- collector/utils/extensions/WebsiteDepth/index.js 153 additions, 0 deletionscollector/utils/extensions/WebsiteDepth/index.js
- frontend/src/components/DataConnectorOption/media/index.js 2 additions, 0 deletionsfrontend/src/components/DataConnectorOption/media/index.js
- frontend/src/components/DataConnectorOption/media/link.svg 1 addition, 0 deletionsfrontend/src/components/DataConnectorOption/media/link.svg
- frontend/src/components/Modals/MangeWorkspace/DataConnectors/Connectors/WebsiteDepth/index.jsx 134 additions, 0 deletions...orkspace/DataConnectors/Connectors/WebsiteDepth/index.jsx
- frontend/src/components/Modals/MangeWorkspace/DataConnectors/index.jsx 7 additions, 0 deletions...components/Modals/MangeWorkspace/DataConnectors/index.jsx
- frontend/src/models/dataConnector.js 18 additions, 0 deletionsfrontend/src/models/dataConnector.js
- server/endpoints/extensions/index.js 21 additions, 0 deletionsserver/endpoints/extensions/index.js
Loading
Please register or sign in to comment