Quex is a tool to generate lexical analyzers. A lexical analyzer is a program that transforms a stream of characters into a stream of 'atomic chunks of meaning'.
As a result of the lexical analysis process the list of 'atomic chunks of meaning', so called 'tokens', prepare the interpretation on some higher level.
Each token consists at least of a type identifier. Additionally, some parameters about the matching 'lexeme' might be stored in the token.
Comments