This module implements a base object of a lexer with efficient buffer handling. Only at line endings checks are necessary if the buffer needs refilling.
TBaseLexer = object of TObject
bufpos*: int
buf*: cstring
bufLen*: int
input: PStream
lineNumber*: int
sentinel: int
lineStart: int
fileOpened: bool
-
the base lexer. Inherit your lexer from this object.
EndOfFile = '\0'
-
end of file marker
NewLines = {'\x0D', '\x0A'}-
proc close(L: var TBaseLexer) {.raises: [E_Base], tags: [].}-
closes the base lexer. This closes L's associated stream too.
proc handleCR(L: var TBaseLexer; pos: int): int {.raises: [E_Base],
tags: [FReadIO].}-
Call this if you scanned over 'c' in the buffer; it returns the the position to continue the scanning from. pos must be the position of the 'c'.
proc handleLF(L: var TBaseLexer; pos: int): int {.raises: [E_Base],
tags: [FReadIO].}-
Call this if you scanned over 'L' in the buffer; it returns the the position to continue the scanning from. pos must be the position of the 'L'.
proc open(L: var TBaseLexer; input: PStream; bufLen: int = 8192) {.
raises: [E_Base], tags: [FReadIO].}-
inits the TBaseLexer with a stream to read from
proc getColNumber(L: TBaseLexer; pos: int): int {.raises: [], tags: [].}-
retrieves the current column.
proc getCurrentLine(L: TBaseLexer; marker: bool = true): string {.raises: [],
tags: [].}-
retrieves the current line.