Parser generators: the unglamorous tool that beats LLMs at structured parsing

Your phone gets a message:

Rs 500 debited from Ac XX1234 on 14-Apr-26. UPI Ref 123456789012. Available Bal: Rs 2,500.30

You need to extract the amount, account, date, reference, and balance - from millions of these, across dozens of bank formats, in real time.

My first instinct was regex. It worked for one bank. Then another bank formatted the date differently. Then a third abbreviated “Available Bal” as “Avl Bal”. The regex count climbed. The maintenance burden grew faster.

Why regex breaks down

Regex works when your format is truly fixed. One sender, one template, no drift. The moment you have ten banks each formatting amounts, dates, and balances slightly differently, you're writing a separate pattern for each. Then edge cases start appearing - trimmed values, encoding differences, senders who change their templates without notice. You patch one pattern and break another.

The deeper problem: regex handles patterns or structure, not both. You can match an amount. You can't easily express "an amount, followed by an account reference, followed by a date - and label each one." That requires something that understands sequence and hierarchy, not just character matching.

My second instinct: throw an LLM at it. It handles variation well. But at scale, I was capped at 30 requests per second in batch mode - and paying per token for what is, structurally, a deterministic problem.

Why LLMs are the wrong fit here

LLMs handle variation beautifully. Give one a novel SMS format it's never seen and it'll still extract the right fields. That flexibility is real.

But SMS parsing isn't a flexible problem. The formats are constrained - finite senders, known templates, predictable structure. You don't need a model that can reason about language; you need one that can reliably tokenise a known pattern at volume. And LLMs are expensive at that job: slow, probabilistic, and opaque. When a field comes back wrong, you can't easily explain why or guarantee it won't happen again.

Probabilistic outputs are the right tradeoff for open-ended text. For structured formats, they're a liability.

There’s a third option most engineers skip straight past: a parser generator. It gave me 1,200 RPS, exact outputs, and a system I could debug line by line.

How parsers work

Parsing converts a flat string into a structured tree by working in three stages.

Stage 1 - Tokenisation (lexing). The raw string is broken into named chunks called tokens. Given Rs 500 debited from Ac XX1234, the lexer doesn’t scan character by character - it classifies: Rs is a currency marker, 500 is an amount, XX1234 is an account reference. You define what each token looks like; the lexer applies those definitions to produce a token stream. Not every token needs a specific rule - words and numbers that don’t need special handling are still valid tokens, classified by their type (word, integer, decimal). This is how filler text like debited, from, and Ac gets handled: no special rule, just words in a known position.

Stage 2 - Parsing. The token stream is matched against rules that describe valid sequences. A rule like alert: currency amount word+ account_ref either matches or it doesn’t. word+ handles filler like debited from Ac without listing each word explicitly.

You can label the tokens you care about: alert: currency amt=amount word+ acct=account_ref. When the rule matches, you get a parse tree. No ambiguity, no backtracking.

Stage 3 - Semantic analysis. The parse tree is just structure - it doesn’t know what things mean. This is where your code takes over. You walk the tree with simple visitor classes, extract fields, apply types, run business logic. Labels from the parser rule make this straightforward: tree.amt and tree.acct are right there, no positional indexing needed. The parser hands you a clean skeleton; you decide what to do with it.

The output for our SMS looks like:

{
  "type": "debit",
  "amount": 500.00,
  "account": "XX1234",
  "date": "2026-04-14",
  "reference": "123456789012",
  "balance": 2500.30
}

Exact. Every time. And when a field is wrong, you know exactly which rule failed.

You don’t have to write the grammar yourself

The grammar file is the hard part. Writing token rules and parser rules by hand for a few SMS formats is manageable. Writing them for 40 banks, each with 3–5 template variants, is a maintenance nightmare - I hit this ceiling early. A rule change in one place breaks three others. New formats mean new rules, new regression tests, new bugs.

For a constrained, known dataset, there’s a better path: generate the grammar automatically.

The approach works in three steps.

Step 1 - Tokenise your samples with a standard lexical ruleset. Start with a base set of token rules that cover the primitives: amounts, dates, account references, UPI IDs, reference numbers, words, separators. Run your sample dataset through these rules. Each input string becomes a token sequence - CURRENCY AMOUNT word word word ACCOUNT word DATE - without any parser rules yet.

Step 2 - Find the recurrent sequences. Across a large enough sample set, certain token sequences will appear again and again. CURRENCY AMOUNT word+ ACCOUNT word+ DATE word+ REFERENCE might account for 60% of your debit alerts. These high-frequency sequences are your parser rules. You don’t invent them; you observe them.

Step 3 - Write the grammar file automatically. A small GrammarWriter class takes the most frequent labelled sequences and emits a valid .g4 (ANTLR) or .lark file. The output is a grammar you didn’t have to author. Coverage scales with your dataset, not with developer hours.

The one manual piece is labelling - mapping token sequences to the fields you care about (amount, account, reference, and so on). This can be done through annotation tooling, or with an LLM doing the first pass. Either way, it’s a one-time cost per rule type, not per format.

The result: a parser that handles dozens of bank formats, was never written by hand, and can be regenerated whenever the dataset grows.

What you actually get

The performance gap surprised me. The same extraction problem that a fine-tuned LLM handles at 30 RPS in batch mode runs at 1,200 RPS deterministically. No GPU, no network call, no queuing. The grammar runs in-process. The cost difference is just as stark: I was running a Gemma3 model on a g5.xlarge - around $300/month; the deterministic parser runs on half a vCPU and 256 MB of memory, closer to $4–5.

Beyond speed, you get explainability. Every output traces back to a rule. When a field comes back wrong, I fix the rule - not retrain a model, not prompt-engineer around it, not accept a 2% error rate as good enough.

And the semantic layer is yours. Because business logic lives in code - not baked into a model - you can integrate anything: look up a merchant name against a knowledge base, call a secondary model only when the parser isn’t confident, extend trimmed values using a known lookup table. The parser handles structure; you handle meaning.

The honest tradeoffs

This approach has real limits worth naming.

It only works for constrained formats. Free-form text - customer support messages, survey responses, anything that doesn’t follow a template - is not a fit. If the structure varies unpredictably, there’s no grammar to write.

There is upfront tooling investment. You need a sample dataset, a base lexical ruleset, and a pipeline to generate and label rules. That’s not trivial to build from scratch - though once it exists, adding new formats is cheap. If you’d rather not build it from scratch, reach out at shashibudati@gmail.com.

Coverage is a function of how many formats you’ve seen. A parser won’t generalise to a format it hasn’t been written for. An LLM might.

None of these are dealbreakers for the right problem. They’re just the shape of the tradeoff.

The right fit: if you have a finite set of formats, a need for exact outputs, and any volume at all - it’s worth reaching for a grammar before you reach for a model.