Skip to content

GitLab

Explore

Sign in

Primary navigation

Project

LlamaIndexTS
- Activity
- Members
- Labels
- Issues
- Issue boards
- Milestones
- Iterations
- Wiki
- Environments
- Terraform modules
- Incidents
- Service Desk

Snippets Groups Projects

ae49ff4e

Unverified Commit ae49ff4e authored 5 months ago by Alex Yang Committed by GitHub 5 months ago

Downloads
- Patches
- Plain Diff

feat: use `gpt-tokenizer` (#1352)

parent a75af835

Branches

Tags

No related merge requests found

Changes 9

Expand all Hide whitespace changes

Inline Side-by-side

Showing

.changeset/friendly-chicken-switch.md 6 additions, 0 deletions

.changeset/friendly-chicken-switch.md
packages/env/package.json 3 additions, 3 deletions

packages/env/package.json
packages/env/src/tokenizers/node.ts 7 additions, 10 deletions

packages/env/src/tokenizers/node.ts
packages/env/tests/tokenizer.test.ts 11 additions, 0 deletions

packages/env/tests/tokenizer.test.ts
packages/llamaindex/e2e/examples/waku-query-engine/package.json 9 additions, 10 deletions

...es/llamaindex/e2e/examples/waku-query-engine/package.json
packages/llamaindex/e2e/examples/waku-query-engine/vite.config.ts 0 additions, 8 deletions

.../llamaindex/e2e/examples/waku-query-engine/vite.config.ts
packages/llamaindex/package.json 1 addition, 1 deletion

packages/llamaindex/package.json
packages/llamaindex/src/next.ts 0 additions, 6 deletions

packages/llamaindex/src/next.ts
pnpm-lock.yaml 282 additions, 235 deletions

pnpm-lock.yaml

with 319 additions and 273 deletions

Loading

0% Loading or .

You are about to add 0 people to the discussion. Proceed with caution.

Finish editing this message first!

Please register or sign in to comment

Strive to be the person your dogs believe you are