qscript.compiler.tokengen

For reading a script into tokens

Members

stringToTokens Token[] stringToTokens(string[] s): returns Token[] with type identified based on string[] input
toTokens TokenList toTokens(string[] script, CompileError[] errors): Takes script, and separates into tokens (using separateTokens), identifies token types, retuns the Tokens with Token.Type in an array