Module lexer

Source

Structsยง

SpannedToken

Functionsยง

eat_identifier ๐Ÿ”’
Eat an identifier, that is, a string matching โ€˜[a-zA-Z][a-zA-Z\d_]*โ€™, if any.
ensure_no_bidi_override ๐Ÿ”’
Ensure that string contains no Unicode Bidirectional Override code points.
is_bidi_override ๐Ÿ”’
Checks if a char is a Unicode Bidirectional Override code point