flex and bison

CHAPTER 1 In troducin g Flex an d Bison
(p1)

Flex and Bison are tools for building programs that handle structured input.

Lexical Analysis and Parsing
The earliest compilers back in the 1950s used utterly ad hoc techniques to analyze the syntax of the source code of programs they were compiling.

One of the key insights was to break the job into two parts: lexical analysis (also called lexing or scanning) and syntax analysis (or parsing).

Roughly speaking, scanning divides the input into meaningful chunks, called tokens, and parsing figures out how the tokens relate to each other

Regular Expressions and Scanning
Scanners generally work by looking for patterns of characters in the input. For example,
in a C program, an integer constant is a string of one or more digits, a variable name is a letter followed by zero or more letters or digits, and the various operators are single characters or pairs of characters. A straightforward way to describe these patterns is regular expressions, often shortened to regex or regexp.