pub(crate) fn tokenize_iter(
input: &str,
lo: u32,
) -> impl '_ + Iterator<Item = Result<SpannedToken>>Expand description
Yields spanned tokens from the given source code text.
The lo byte position determines where spans will start.
pub(crate) fn tokenize_iter(
input: &str,
lo: u32,
) -> impl '_ + Iterator<Item = Result<SpannedToken>>Yields spanned tokens from the given source code text.
The lo byte position determines where spans will start.