org.fife.rsta.ac.java.rjc.lexer
Class Scanner

java.lang.Object
  org.fife.rsta.ac.java.rjc.lexer.Scanner

public class Scanner
extends Object

A scanner that allows the user to "push back" tokens. This scanner allows arbitrary lookahead.

Version:

1.0

Author:

Robert Futrell

Constructor Summary

Scanner()
Constructor.

Scanner(List tokens)
Constructor.

Scanner(Reader r)
Constructor.

Method Summary

void	clearResetPosition()
Offset	createOffset(int offs) Returns an offset into the source being parsed.
void	decreaseTypeArgumentsLevel() Decreases the depth in which we're in TypeArguments or TypeParameters.
void	eatParenPairs() Eats through (possibly nested) paren pairs, e.g.:
void	eatThroughNext(int tokenType) Eats all tokens up to (and including) the next token of the specified type.
void	eatThroughNextSkippingBlocks(int tokenType) Eats all tokens up to (and including) the next token of the specified type.
void	eatUntilNext(int type1, int type2)
void	eatUntilNext(int type1, int type2, int type3)
int	getColumn() Returns the current column into the current line.
String	getLastDocComment() Returns the last documentation comment parsed.
int	getLine() Returns the current line into the document.
Token	getMostRecentToken() Returns the most recently-lexed token.
int	getOffset() Returns the current offset into the document.
int	getTypeArgumentsLevel() Returns the current TypeArgument/TypeParameter level.
void	increaseTypeArgumentsLevel() Increases the depth in which we're in TypeArguments or TypeParameters.
void	markResetPosition()
void	resetToLastMarkedPosition()
void	setDocument(Document doc) Sets the Swing Document whose content is being parsed.
int	skipBracketPairs() Skips all bracket pairs ('[' followed by ']') in the stream.
Token	yylex() Returns the next token from the input stream.
Token	yylexNonNull(int type1, int type2, int type3, String error) Returns the next token from the input stream, or throws an exception if the end of stream is reached or if the token is not of three given types.
Token	yylexNonNull(int type1, int type2, String error) Returns the next token from the input stream, or throws an exception if the end of stream is reached or if the token is not of two given types.
Token	yylexNonNull(int type, String error) Returns the next token from the input stream, or throws an exception if the end of stream is reached or if the token is not of a given type.
Token	yylexNonNull(String error) Returns the next token from the input stream, or throws an exception if the end of stream is reached.
Token	yyPeek() Returns the next token, but does not take it off of the stream.
Token	yyPeek(int depth) Returns the depth-th token, but does not anything off of the stream.
int	yyPeekCheckType() Peeks at and returns the type of the next token on the stream.
int	yyPeekCheckType(int index) Peeks at and returns the type of the specified token on the stream.
Token	yyPeekNonNull(int type1, int type2, int type3, String error) Returns the next token, but does not take it off of the stream.
Token	yyPeekNonNull(int type1, int type2, String error) Returns the next token, but does not take it off of the stream.
Token	yyPeekNonNull(int type, String error) Returns the next token, but does not take it off of the stream.
Token	yyPeekNonNull(String error) Returns the next token, but does not take it off of the stream.
void	yyPushback(Token t) Pushes a token back onto the stream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Scanner

public Scanner()

Constructor. This scanner will return no tokens unless some are pushed onto it via yyPushback(Token).

Scanner

public Scanner(List tokens)

Constructor. This scanner will only return those tokens pushed onto it.

Parameters:

tokens - Tokens to return.

Scanner

public Scanner(Reader r)

Constructor.

Parameters:

r - The stream to read from.

Method Detail

decreaseTypeArgumentsLevel

public void decreaseTypeArgumentsLevel()

Decreases the depth in which we're in TypeArguments or TypeParameters.

See Also:

increaseTypeArgumentsLevel(), getTypeArgumentsLevel()

createOffset

public Offset createOffset(int offs)

Returns an offset into the source being parsed. This offset will be tracked if we are parsing code from a Swing JTextComponent.

Parameters:

offs - The offset.

Returns:

An object representing the offset.

See Also:

setDocument(Document)

getColumn

public int getColumn()

Returns the current column into the current line.

Returns:

The current column.

See Also:

getLine()

getLastDocComment

public String getLastDocComment()

Returns the last documentation comment parsed. The "last documentation comment" is cleared when this method returns.

Returns:

The last documentation comment parsed, or null if there was none.

getLine

public int getLine()

Returns the current line into the document.

Returns:

The current line.

See Also:

getColumn()

getMostRecentToken

public Token getMostRecentToken()

Returns the most recently-lexed token.

Returns:

The token, or null if EOS was reached.

getOffset

public int getOffset()

Returns the current offset into the document.

Returns:

The offset.

eatParenPairs

public void eatParenPairs()
                   throws IOException

Eats through (possibly nested) paren pairs, e.g.:

(int i=0; i<getFoo(getParam()); i++)

. Blocks nested inside the paren pairs are also skipped.

Throws:

IOException - If an IO error occurs.

InternalError - If the next token is not a '('.

eatThroughNext

public void eatThroughNext(int tokenType)
                    throws IOException

Eats all tokens up to (and including) the next token of the specified type. This is useful, for example, to eat until the next semicolon.

Parameters:

tokenType - The type of token to eat through.

Throws:

IOException - If an IO error occurs.

eatThroughNextSkippingBlocks

public void eatThroughNextSkippingBlocks(int tokenType)
                                  throws IOException

Eats all tokens up to (and including) the next token of the specified type. This is useful, for example, to eat until the next semicolon.

Parameters:

tokenType - The type of token to eat through.

Throws:

IOException - If an IO error occurs.

eatUntilNext

public void eatUntilNext(int type1,
                         int type2)
                  throws IOException

Throws:

IOException

eatUntilNext

public void eatUntilNext(int type1,
                         int type2,
                         int type3)
                  throws IOException

Throws:

IOException

getTypeArgumentsLevel

public int getTypeArgumentsLevel()

Returns the current TypeArgument/TypeParameter level.

Returns:

The current level.

See Also:

increaseTypeArgumentsLevel(), decreaseTypeArgumentsLevel()

increaseTypeArgumentsLevel

public void increaseTypeArgumentsLevel()

Increases the depth in which we're in TypeArguments or TypeParameters.

See Also:

decreaseTypeArgumentsLevel(), getTypeArgumentsLevel()

markResetPosition

public void markResetPosition()

resetToLastMarkedPosition

public void resetToLastMarkedPosition()

clearResetPosition

public void clearResetPosition()

setDocument

public void setDocument(Document doc)

Sets the Swing Document whose content is being parsed. This method should be called if we are parsing code inside a JTextComponent, as it will help our parsed code to track changes when the document is modified. If we are parsing source from a flat file, this method shouldn't be called.

Parameters:

doc - The document being parsed.

skipBracketPairs

public int skipBracketPairs()
                     throws IOException

Skips all bracket pairs ('[' followed by ']') in the stream.

Returns:

The number of bracket pairs skipped.

Throws:

IOException - If an IO error occurs.

yylex

public Token yylex()
            throws IOException

Returns the next token from the input stream.

Returns:

The next token.

Throws:

IOException - If an IO error occurs.

yylexNonNull

public Token yylexNonNull(String error)
                   throws IOException

Returns the next token from the input stream, or throws an exception if the end of stream is reached.

Parameters:

error - The error description for the exception if the end of stream is reached.

Returns:

The token.

Throws:

IOException - If an IO error occurs or the end of stream is reached.

yylexNonNull

public Token yylexNonNull(int type,
                          String error)
                   throws IOException

Returns the next token from the input stream, or throws an exception if the end of stream is reached or if the token is not of a given type.

Parameters:

type - The type the token must be.

error - The error description for the exception if the end of stream is reached, or if the token is of an unexpected type.

Returns:

The token.

Throws:

IOException - If an IO error occurs or the end of stream is reached, or if the token is of the wrong type.

yylexNonNull

public Token yylexNonNull(int type1,
                          int type2,
                          String error)
                   throws IOException

Returns the next token from the input stream, or throws an exception if the end of stream is reached or if the token is not of two given types.

Parameters:

type1 - One type the token can be.

type2 - Another type the token can be, or -1 if we should only check against type1.

error - The error description for the exception if the end of stream is reached, or if the token is of an unexpected type.

Returns:

The token.

Throws:

IOException - If an IO error occurs or the end of stream is reached, or if the token is of a wrong type.

yylexNonNull

public Token yylexNonNull(int type1,
                          int type2,
                          int type3,
                          String error)
                   throws IOException

Returns the next token from the input stream, or throws an exception if the end of stream is reached or if the token is not of three given types.

Parameters:

type1 - One type the token can be.

type2 - Another type the token can be, or -1 if we should only check against type1.

type3 - Another type the token can be, or -1 if we should only check against type1 and type2.

error - The error description for the exception if the end of stream is reached, or if the token is of an unexpected type.

Returns:

The token.

Throws:

IOException - If an IO error occurs or the end of stream is reached, or if the token is of a wrong type.

yyPeek

public Token yyPeek()
             throws IOException

Returns the next token, but does not take it off of the stream. This is useful for lookahead.

Returns:

The next token.

Throws:

IOException - If an IO error occurs.

yyPeek

public Token yyPeek(int depth)
             throws IOException

Returns the depth-th token, but does not anything off of the stream. This is useful for lookahead.

Parameters:

depth - The token to peek at, from 1 forward.

Returns:

The token, or null if that token index is past the end of the stream.

Throws:

IOException - If an IO error occurs.

yyPeekCheckType

public int yyPeekCheckType()
                    throws IOException

Peeks at and returns the type of the next token on the stream.

Returns:

The type of the next token, or -1 if the end of stream has been reached.

Throws:

IOException - If an IO error occurs.

yyPeekCheckType

public int yyPeekCheckType(int index)
                    throws IOException

Peeks at and returns the type of the specified token on the stream.

Parameters:

index - The index of the token to retrieve.

Returns:

The type of the token, or -1 if the end of stream was reached first.

Throws:

IOException - If an IO error occurs.

yyPeekNonNull

public Token yyPeekNonNull(String error)
                    throws IOException

Returns the next token, but does not take it off of the stream. This is useful for lookahead.

Returns:

The next token.

Throws:

IOException - If an IO error occurs.

yyPeekNonNull

public Token yyPeekNonNull(int type,
                           String error)
                    throws IOException

Returns the next token, but does not take it off of the stream. This is useful for lookahead.

Parameters:

type - The type the token must be.

Returns:

The next token.

Throws:

IOException - If an IO error occurs, or if EOS is reached, or if the token is not of the specified type.

yyPeekNonNull

public Token yyPeekNonNull(int type1,
                           int type2,
                           String error)
                    throws IOException

Returns the next token, but does not take it off of the stream. This is useful for lookahead.

Parameters:

type - The type the token must be.

Returns:

The next token.

Throws:

IOException - If an IO error occurs, or if EOS is reached, or if the token is not of the specified type.

yyPeekNonNull

public Token yyPeekNonNull(int type1,
                           int type2,
                           int type3,
                           String error)
                    throws IOException

Returns the next token, but does not take it off of the stream. This is useful for lookahead.

Parameters:

type - The type the token must be.

Returns:

The next token.

Throws:

IOException - If an IO error occurs, or if EOS is reached, or if the token is not of the specified type.

yyPushback

public void yyPushback(Token t)

Pushes a token back onto the stream.

Parameters:

t - The token.