Tokenizer Uncommon

Definition, synonyms and related words

Definitions

Noun

A system that parses an input stream into its component tokens.

See Also

tokenize input parse tokenizers system stream tokeniser tokenization

Related Terms

tokenize input parse tokenizers system stream tokeniser tokenization token

Rhyming Words

izer nzer tazer sizer pizer gozer dozer razer mazer lazer gazer hazer tizer glozer keezer momzer keizer pfizer goozer seizer

Compare