Lexer

java.lang.Object
- compilation.analysis.lexical.Lexer

```
public final class Lexer
extends java.lang.Object
```
The Lexer class is responsible for performing lexical analysis on a string of text, identifying each character one by one.
It creates Tokens that represent each individual character, ultimately producing a List of Token objects ordered sequentially. During this stage any characters that are unrecognised by source language are identified and reported as an error.
The contents of the String contained within SourceInput are parsed character by character, with each one being examined individually in order to gather information about its general characteristics (e.g. syntax, value, type, and position in the text). A Token object is then generated to store this data, with a new instance being created to represent every character. These are incrementally and sequentially added to a List, which is the single output of this class.
Any compilation errors that occur during this stage are passed to the ErrorHandler in the form of LexicalError objects.

Since:

1.0

- Constructor Summary
  
  Constructors
  Constructor Description
  
  Lexer(SourceInput sourceInput, ErrorHandler errorHandler)
  Constructs a Lexer object initialised with contents of sourceInput.
- Method Summary
  
  All Methods Instance Methods Concrete Methods
  Modifier and Type Method Description
  
  java.util.List<Token> getTokens()
  Returns a List of Token objects generated from sourceInput.
  - Methods inherited from class java.lang.Object
    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - Lexer
```
public Lexer(SourceInput sourceInput,
             ErrorHandler errorHandler)
```
    Constructs a Lexer object initialised with contents of sourceInput.
    The Lexer always begins at position 0 in the text, and adds any errors to the errorHandler.
    
    Parameters:
    
    sourceInput - The source code input to be lexed
    
    errorHandler - The ErrorHandler to store any errors that occur
- Method Detail
  - getTokens
```
public java.util.List<Token> getTokens()
```
    Returns a List of Token objects generated from sourceInput.
    The contents of sourceInput are lexed into Token objects, and added sequentially into a List.
    
    Returns:
    
    A List of Token objects