qscript.compiler.tokengen

For reading a script into tokens

Members

Functions

toTokens
TokenList toTokens(string[] script, CompileError[] errors)

Takes script, and separates into tokens (using separateTokens), identifies token types, retuns the Tokens with Token.Type in an array

tokenBracketPos
uinteger tokenBracketPos(Token[] tokens, uinteger index)

Returns index of closing/openinig bracket of the provided bracket

Structs

Token
struct Token

Each token is stored as a Token with the type and the actual token

TokenList
struct TokenList

To store Tokens with Types where the line number of each token is required

Meta