Token

Object
- Token

```
public class Token
extends Object
```
A linked list of tokens. Each token has three fields - a token identifier, which is a byte value that can be looked up in the array returned by SyntaxDocument.getColors() to get a color value, a length value which is the length of the token in the text, and a pointer to the next token in the list.

Field Summary

Fields
Modifier and Type	Field and Description
`static byte`	`COMMENT1` This can be used to mark a comment.
`static byte`	`COMMENT2` This can be used to mark a comment.
`static byte`	`END` The token type, that along with a length of 0 marks the end of the token list.
`static byte`	`FUNCTION1` Functions
`static byte`	`FUNCTION2` Methods (functions inside a class)
`static byte`	`FUNCTION3` Loop/function-like blocks (for, while, etc.)
`static byte`	`FUNCTION4` Built-in Processing functions (setup, draw, mouseDragged).
`byte`	`id` The id of this token.
`static byte`	`ID_COUNT` The total number of defined token ids.
`static byte`	`INTERNAL_FIRST` The first id that can be used for internal state in a token marker.
`static byte`	`INTERNAL_LAST` The last id that can be used for internal state in a token marker.
`static byte`	`INVALID` Invalid token id.
`static byte`	`KEYWORD1` Keywords (void, int, boolean, etc.)
`static byte`	`KEYWORD2` Fields [variables within a class]
`static byte`	`KEYWORD3` Loop/function-like blocks (for, while, etc.)
`static byte`	`KEYWORD4` Processing variables (width, height, focused, etc.)
`static byte`	`KEYWORD5` Datatypes (int, boolean, etc.)
`static byte`	`KEYWORD6` Keywords which can be followed by parenthesis
`static byte`	`LABEL` Label token id.
`int`	`length` The length of this token.
`static byte`	`LITERAL1` Strings in quotes
`static byte`	`LITERAL2` Constants (QUARTER_PI, CORNERS, etc.)
`Token`	`next` The next token in the linked list.
`static byte`	`NULL` Normal text token id.
`static byte`	`OPERATOR` Operator token id.

Constructor Summary

Constructors
Constructor and Description

Token(int length, byte id)
Creates a new token.

Constructors
Constructor and Description
`Token(int length, byte id)` Creates a new token.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type Method and Description

String toString()
Returns a string representation of this token.
- Methods inherited from class Object
  equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`String`	`toString()` Returns a string representation of this token.

- Field Detail
  - NULL
```
public static final byte NULL
```
    Normal text token id. This should be used to mark normal text.
    
    See Also:
    
    Constant Field Values
  - COMMENT1
```
public static final byte COMMENT1
```
    This can be used to mark a comment.
    
    See Also:
    
    Constant Field Values
  - COMMENT2
```
public static final byte COMMENT2
```
    This can be used to mark a comment.
    
    See Also:
    
    Constant Field Values
  - LITERAL1
```
public static final byte LITERAL1
```
    Strings in quotes
    
    See Also:
    
    Constant Field Values
  - LITERAL2
```
public static final byte LITERAL2
```
    Constants (QUARTER_PI, CORNERS, etc.)
    
    See Also:
    
    Constant Field Values
  - LABEL
```
public static final byte LABEL
```
    Label token id. This can be used to mark labels (eg, C mode uses this to mark ...: sequences)
    
    See Also:
    
    Constant Field Values
  - KEYWORD1
```
public static final byte KEYWORD1
```
    Keywords (void, int, boolean, etc.)
    
    See Also:
    
    Constant Field Values
  - KEYWORD2
```
public static final byte KEYWORD2
```
    Fields [variables within a class]
    
    See Also:
    
    Constant Field Values
  - KEYWORD3
```
public static final byte KEYWORD3
```
    Loop/function-like blocks (for, while, etc.)
    
    See Also:
    
    Constant Field Values
  - KEYWORD4
```
public static final byte KEYWORD4
```
    Processing variables (width, height, focused, etc.)
    
    See Also:
    
    Constant Field Values
  - KEYWORD5
```
public static final byte KEYWORD5
```
    Datatypes (int, boolean, etc.)
    
    See Also:
    
    Constant Field Values
  - KEYWORD6
```
public static final byte KEYWORD6
```
    Keywords which can be followed by parenthesis
    
    See Also:
    
    Constant Field Values
  - FUNCTION1
```
public static final byte FUNCTION1
```
    Functions
    
    See Also:
    
    Constant Field Values
  - FUNCTION2
```
public static final byte FUNCTION2
```
    Methods (functions inside a class)
    
    See Also:
    
    Constant Field Values
  - FUNCTION3
```
public static final byte FUNCTION3
```
    Loop/function-like blocks (for, while, etc.)
    
    See Also:
    
    Constant Field Values
  - FUNCTION4
```
public static final byte FUNCTION4
```
    Built-in Processing functions (setup, draw, mouseDragged).
    
    See Also:
    
    Constant Field Values
  - OPERATOR
```
public static final byte OPERATOR
```
    Operator token id. This can be used to mark an operator. (eg, SQL mode marks +, -, etc with this token type)
    
    See Also:
    
    Constant Field Values
  - INVALID
```
public static final byte INVALID
```
    Invalid token id. This can be used to mark invalid or incomplete tokens, so the user can easily spot syntax errors.
    
    See Also:
    
    Constant Field Values
  - ID_COUNT
```
public static final byte ID_COUNT
```
    The total number of defined token ids.
    
    See Also:
    
    Constant Field Values
  - INTERNAL_FIRST
```
public static final byte INTERNAL_FIRST
```
    The first id that can be used for internal state in a token marker.
    
    See Also:
    
    Constant Field Values
  - INTERNAL_LAST
```
public static final byte INTERNAL_LAST
```
    The last id that can be used for internal state in a token marker.
    
    See Also:
    
    Constant Field Values
  - END
```
public static final byte END
```
    The token type, that along with a length of 0 marks the end of the token list.
    
    See Also:
    
    Constant Field Values
  - length
```
public int length
```
    The length of this token.
  - id
```
public byte id
```
    The id of this token.
  - next
```
public Token next
```
    The next token in the linked list.
- Constructor Detail
  - Token
```
public Token(int length,
             byte id)
```
    Creates a new token.
    
    Parameters:
    
    length - The length of the token
    
    id - The id of the token
- Method Detail
  - toString
```
public String toString()
```
    Returns a string representation of this token.
    
    Overrides:
    
    toString in class Object

Class Token

Field Summary

Constructor Summary

Method Summary

Methods inherited from class Object

Field Detail

NULL

COMMENT1

COMMENT2

LITERAL1

LITERAL2

LABEL

KEYWORD1

KEYWORD2

KEYWORD3

KEYWORD4

KEYWORD5

KEYWORD6

FUNCTION1

FUNCTION2

FUNCTION3

FUNCTION4

OPERATOR

INVALID

ID_COUNT

INTERNAL_FIRST

INTERNAL_LAST

END

length

id

next

Constructor Detail

Token

Method Detail

toString