Character
class CharacterHeuristicTokenCountEstimator constructor(val charsPerToken: Int = DEFAULT_CHARS_PER_TOKEN) : TokenCountEstimator<String>
Estimates token count by dividing character length by a configurable characters-per-token ratio. The default ratio of 4 approximates tokenization for English text across most LLM tokenizers. Callers working with non-Latin scripts or code may supply a different ratio.