Lexical Structure

This page outlines the tokens recognized by the Branchline lexer. Tokens are grouped into operators & punctuators, literals, and keywords.

Operators & Punctuators

Token(s)	Lexeme(s)	Example
LEFT_PAREN / RIGHT_PAREN	( )	call(arg)
LEFT_BRACE / RIGHT_BRACE	{ }	{ x: 1 }
LEFT_BRACKET / RIGHT_BRACKET	[ ]	items[0]
COMMA	,	a, b
DOT	.	obj.field
SEMICOLON	;	let x = 1;
COLON	:	name: Type
QUESTION	?	value?
COALESCE	??	a ?? b
PLUS	+	a + b
MINUS	-	a - b
STAR	*	a * b
SLASH	/	a / b
SLASH_SLASH	//	a // b
PERCENT	%	a % b
CONCAT	++	xs ++ ys
LT / LE	< <=	a <= b
GT / GE	> >=	a > b
ASSIGN	=	x = 1
EQ / NEQ	== !=	a != b
BANG	!	!flag
AND	&&	a && b
OR	||	a || b
ARROW	->	x -> x * 2
PIPE	|	src | step
DOLLAR	$	$name

Literals

Token	Example	Description
IDENTIFIER	userName	Name for variables or functions (backticks can escape keywords).
STRING	"hello"	Text enclosed in double quotes.
NUMBER	42	Integer or floating-point number.

Keywords

Keywords are case-insensitive. A small set is hard-reserved; the rest are contextual and can be used as identifiers where a name is expected.

Hard-reserved: IF, THEN, ELSE, CASE, WHEN, FOR, EACH, IN, WHERE, TRY, CATCH, AWAIT, SUSPEND, TRUE, FALSE, NULL. Use backticks to escape any keyword when you need it as a name (for example, `if`).

Backtick identifiers accept any characters except a backtick or newline.

Line comments use // only when they appear at the start of a line or after a statement delimiter (for example, after a semicolon). Use that form if you want comments that would otherwise conflict with the // integer division operator.

Statement Keywords

Keyword	Description	Usage
OUTPUT	Specify pipeline output.	example
USING	Reference an adapter or module.	example
TRANSFORM	Define a transformation step.	example
OPTIONS	Configure per-transform settings.	example
BUFFER	Buffer mode value for OPTIONS.	example
FOR / EACH	Start a loop over items.	example
IF / THEN / ELSE	Conditional branching.	example
ENUM	Define an enumeration.	example
FOREACH	Loop shortcut.	example
INPUT	Reference pipeline input.	example
ABORT	Abort execution.	example
THROW	Throw an error.	example
TRY / CATCH	Error handling expression/statement.	example
RETRY / TIMES / BACKOFF	Retry logic modifiers.	example
CALL	Invoke a host function.	example
SHARED / SINGLE / MANY	Resource qualifiers.	example
FUNC	Declare a function.	example
TYPE	Declare a type.	example
RETURN	Return from function.	example
MODIFY	Modify an existing value.	example
WHERE	Filter clause.	example
SET / APPEND / TO	Assignment operations.	example
INIT	Initial value for a variable.	example

Expression Keywords

Keyword	Description	Usage
AS	Alias or cast.	example
LET / IN	Local binding expression.	example
AWAIT	Await asynchronous result.	example
SUSPEND	Suspend execution.	example
TRUE / FALSE	Boolean literals.	example
NULL	Null literal.	example
UNION	Union type expression.	example

Special Token

EOF marks the end of the input.

Summary

Branchline uses a small set of tokens. Keywords are written in upper case and include OUTPUT, TRANSFORM, OPTIONS, BUFFER, SHARED, FUNC, TYPE, LET, IF, FOR, TRY, CALL, AWAIT, and SUSPEND, among others as seen throughout the grammar.

Names are expressed using identifiers: funcDecl ::= FUNC IDENTIFIER "(" paramList? ")" funcBody shows identifiers applied to declarations.

Literals support numbers, strings, booleans, and null via the rule literal ::= NUMBER | STRING | TRUE | FALSE | NULL.

Punctuation such as parentheses, braces, brackets, commas, and semicolons structure programs and appear in the production rules.