curated-tokenizers
Lightweight piece tokenization library
- Category
- Python packages
pip install curated-tokenizers Official docs outline
curated-tokenizers.readthedocs.ioExtra Credit builds an outline of curated-tokenizers's docs the first time your work uses it. Until then, start at the official documentation.
More in Python packages
All 9,553-
pyghmi Python General Hardware Management Initiative (IPMI and others)
-
query-cache-common Common code for Query Cache shared across client and server
-
trec-car-tools Support tools for TREC CAR participants. Also see trec-car.cs.unh.edu
-
vk-api Python модуль для создания скриптов для социальной сети Вконтакте (vk.ru API wrapper)
-
django-user-agents A django package that allows easy identification of visitors' browser, operating system and device information (mobile phone, tablet or has
-
clevercsv A Python package for handling messy CSV files
-
vcver provide package versions with version control data.
-
fairscale FairScale: A PyTorch library for large-scale and high-performance training.
-
google-python-cloud-debugger Python Cloud Debugger
-
python-logstash Python logging handler for Logstash.
-
optuna-integration Integration libraries of Optuna.
-
django-annoying This is a django application that tries to eliminate annoying things in the Django framework.