tiktoken_ruby

An unofficial Ruby wrapper for Tiktoken, a BPE tokenizer written by and used by OpenAI. It can be used to count the number of tokens in text

gem install tiktoken_ruby

Official docs outline

Extra Credit builds an outline of tiktoken_ruby's docs the first time your work uses it. Until then, start at the official documentation.

isolator Detect non-atomic interactions within DB transactions
bulk_insert Faster inserts! Insert N records in a single statement.
go-duration A parser and formatter for Golang's time.Duration
allure-cucumber Cucumber adapter to generate rich allure test reports
logstash-codec-netflow This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemna
redmine_extensions Redmine Extensions provide many extended functionalities for Redmine project.
aws-sdk-taxsettings Official AWS Ruby gem for Tax Settings. This gem is part of the AWS SDK for Ruby.
buff-shell_out A mixin for issuing shell commands and collecting the output
buff-ruby_engine A mixin for querying the platform running Ruby
logstash-filter-drop This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemna
buff-config A simple configuration class
logstash-output-sns This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemna