Lexical Structure

This page outlines the tokens recognized by the Branchline lexer. Tokens are grouped into operators & punctuators, literals, and keywords.

Operators & Punctuators

Token(s)	Lexeme(s)	Example
LEFT_PAREN / RIGHT_PAREN	( )	call(arg)
LEFT_BRACE / RIGHT_BRACE	{ }	{ x: 1 }
LEFT_BRACKET / RIGHT_BRACKET	[ ]	items[0]
COMMA	,	a, b
DOT	.	obj.field
SEMICOLON	;	let x = 1;
COLON	:	name: Type
QUESTION	?	value?
COALESCE	??	a ?? b
PLUS	+	a + b
MINUS	-	a - b
STAR	*	a * b
SLASH	/	a / b
PERCENT	%	a % b
CONCAT	++	xs ++ ys
LT / LE	< <=	a <= b
GT / GE	> >=	a > b
ASSIGN	=	x = 1
EQ / NEQ	== !=	a != b
BANG	!	!flag
AND	&&	a && b
OR	||	a || b
ARROW	->	x -> x * 2
PIPE	|	src | step
DOLLAR	$	$name

Literals

Token	Example	Description
IDENTIFIER	userName	Name for variables or functions.
STRING	"hello"	Text enclosed in double quotes.
NUMBER	42	Integer or floating-point number.

Keywords

Reserved words. Each links to a page with usage examples.

Statement Keywords

Keyword	Description	Usage
SOURCE	Declare a data source.	example
OUTPUT	Specify pipeline output.	example
USING	Reference an adapter or module.	example
TRANSFORM	Define a transformation step.	example
STREAM	Declare a stream block.	example
BUFFER	Declare a buffer block.	example
FOR / EACH	Start a loop over items.	example
IF / THEN / ELSE	Conditional branching.	example
ENUM	Define an enumeration.	example
FOREACH	Loop shortcut.	example
INPUT	Reference pipeline input.	example
PARALLEL / ONBLOCK	Run blocks concurrently.	example
ABORT	Abort execution.	example
THROW	Throw an error.	example
TRY / CATCH	Error handling block.	example
RETRY / TIMES / BACKOFF	Retry logic modifiers.	example
SHARED / SINGLE / MANY	Resource qualifiers.	example
FUNC	Declare a function.	example
TYPE	Declare a type.	example
RETURN	Return from function.	example
MODIFY	Modify an existing value.	example
WHERE	Filter clause.	example
SET / APPEND / TO	Assignment operations.	example
INIT	Initial value for a variable.	example

Expression Keywords

Keyword	Description	Usage
AS	Alias or cast.	example
LET / IN	Local binding expression.	example
AWAIT	Await asynchronous result.	example
SUSPEND	Suspend execution.	example
CALL	Invoke a host function.	example
TRUE / FALSE	Boolean literals.	example
NULL	Null literal.	example
UNION	Union type expression.	example

Special Token

EOF marks the end of the input.

Summary

Branchline uses a small set of tokens. Keywords are written in upper case and include SOURCE, OUTPUT, TRANSFORM, SHARED, FUNC, TYPE, LET, IF, FOR, TRY, CALL, AWAIT, and SUSPEND, among others as seen throughout the grammar【F:language/src/test/kotlin/v2/ebnf.txt†L22-L90】.

Names are expressed using identifiers: sourceDecl ::= SOURCE IDENTIFIER adapterSpec? ; and funcDecl ::= FUNC IDENTIFIER "(" paramList? ")" funcBody show identifiers applied to declarations【F:language/src/test/kotlin/v2/ebnf.txt†L22-L63】.

Literals support numbers, strings, booleans, and null via the rule literal ::= NUMBER | STRING | TRUE | FALSE | NULL【F:language/src/test/kotlin/v2/ebnf.txt†L112-L116】.

Punctuation such as parentheses, braces, brackets, commas, and semicolons structure programs and appear in the production rules.