Merge branch 'main' of github.com:THUDM/SwissArmyTransformer into main
No related branches found
No related tags found
Showing
- tokenization/__init__.py 36 additions, 0 deletionstokenization/__init__.py
- tokenization/cogview/__init__.py 16 additions, 0 deletionstokenization/cogview/__init__.py
- tokenization/cogview/sp_tokenizer.py 150 additions, 0 deletionstokenization/cogview/sp_tokenizer.py
- tokenization/cogview/templates.py 77 additions, 0 deletionstokenization/cogview/templates.py
- tokenization/cogview/unified_tokenizer.py 193 additions, 0 deletionstokenization/cogview/unified_tokenizer.py
- tokenization/cogview/vqvae_tokenizer.py 86 additions, 0 deletionstokenization/cogview/vqvae_tokenizer.py
Loading
Please register or sign in to comment
