antlr, C++, Demo
A Guide To Parsing: Algorithms And Terminology
LR Parsing - Coding Ninjas CodeStudio
The Lemon LALR(1) Parser Generator (sqlite.org)
Why you should not use (f)lex, yacc and bison bison:
- %define api.pure purity Language(s): C Request a pure (reentrant) parser program. yychar = yylex (&yylval);
- %define api.parser.class {name} Language(s): C++, Java, D. The name of the parser class. [Rust比D好在哪里?]
- bison simple.yy -o simple.cc; g++ -std=c++14 simple.cc -o simple
- ‘variant’ (C++) This is similar to union, but special storage techniques are used to allow any kind of C++ object to be used.
- %define api.value.type {string}和#define YYSTYPE string一样好使;string里放0x00没问题
void do_expr() { reuse_tk = yying = 1; assert(yyparse() == 0); yying = 0; int n = ec.size(); memcpy(code + code_end, ec.c_str(), n), code_end += n; } // #undef了NDEBUG
Write text parsers with yacc and lex | Practical parsing with Flex and Bison
https://www.gnu.org/software/bison/manual/
- https://www.gnu.org/software/bison/manual/bison.html.gz
- https://www.gnu.org/software/bison/manual/bison.pdf
试了下antlr。痛定思痛,幸好我下载了现成的,虽然不慎2.7.7和4.7.2都装了 :-)。先啰嗦一段Debian的包管理程序的例子,假设已sudo bash:
# apt-get install aptitude (如果没装aptitude的话) # aptitude search antlr antlr antlr3 antlr4 libantlr4-runtime-dev ... # apt-get install antlr4 libantlr4-runtime-dev # dpkg -l | grep antlr ii antlr 2.7.7+dfsg-10 all language tool for constructing recognizers, compilers etc ii antlr4 4.7.2-5 all ANTLR Parser Generator ii libantlr-java 2.7.7+dfsg-10 all language tool for constructing recognizers, compilers etc (java library) ii libantlr3-runtime-java 3.5.2-9 all Runtime library for ANTLR 3 ii libantlr4-runtime-dev:amd64 4.9+dfsg-1.1 amd64 ANTLR Parser Generator - C++ runtime support (development files) ii libantlr4-runtime-java 4.7.2-5 all Runtime library for ANTLR 4 ii libantlr4-runtime4.9:amd64 4.9+dfsg-1.1 amd64 ANTLR Parser Generator - C++ runtime support (shared library) # dpkg -L antlr4 /usr/bin/antlr4 ... # dpkg -L libantlr4-runtime-dev /usr/include/antlr4-runtime /usr/include/antlr4-runtime/ANTLRErrorListener.h ... /usr/lib/x86_64-linux-gnu/libantlr4-runtime.a ...
然后用这里的例子:
$ cat main.cpp #include <iostream> #include "antlr4-runtime.h" #include "ExpressionLexer.h" #include "ExpressionParser.h" int main(int argc, const char* argv[]) { // Provide the input text in a stream antlr4::ANTLRInputStream input("6*(2+3)"); // Create a lexer from the input ExpressionLexer lexer(&input); // Create a token stream from the lexer antlr4::CommonTokenStream tokens(&lexer); // Create a parser from the token stream ExpressionParser parser(&tokens); // Display the parse tree std::cout << parser.expr()->toStringTree() << std::endl; return 0; } $ cat Expression.g4 grammar Expression; // Parser rule expr : '-' expr | expr ( '*' | '/' ) expr | expr ( '+' | '-' ) expr | '(' expr ')' | INT | ID ; // Lexer rules INT : [0-9]+; ID : [a-z]+; WS : [ \t\r\n]+ -> skip; $ antlr4 -Dlanguage=Cpp Expression.g4 (不快) $ g++ -I/usr/include/antlr4-runtime *.cpp -lantlr4-runtime (相当慢) $ a.out (6*(2+3) (6 6) * ((2+3) ( (2+3 (2 2) + (3 3)) ))) [不输出30?!] $ wc -l *.cpp 475 total $ cat /usr/bin/antlr4 #!/bin/sh CLASSPATH=/usr/share/java/stringtemplate4.jar:/usr/share/java/antlr4.jar:/usr/share/java/antlr4-runtime.jar:/usr/share/java/antlr3-runtime.jar/:/usr/share/java/treelayout.jar
exec java -cp $CLASSPATH org.antlr.v4.Tool "$@"
我还是继续用bison和讨厌Java:
prog : /*empty*/ | func | func prog ; func : LEX_NAME '(' opt_params ')' '{' opt_decls opt_stats '}' ; opt_params : /*empty*/ | params ; opt_decls : /*empty*/ | decls ; params : parameter | parameter ',' params ; parameter : LEX_INT LEX_NAME opt_stats : /* empty */ | stats stats : /* empty */ | stat | stat ';' stats ; stat : expr ';' | NAME '=' expr ';' | NAME '(' '(' opt_params ')' ';' // func call : '{' stats '}' | if_statement | LEX_WHILE '(' expr ')' stats // How many fors? 8. 000, 001, ... 111 | LEX_FOR '(' ';' ';' ')' stats | LEX_FOR '(' expr ';' ';' ')' stats | LEX_FOR '(' ';' expr ';' ')' stats | LEX_FOR '(' ';' ';' expr ')' stats | LEX_FOR '(' expr ';' expr ';' ')' stats | LEX_FOR '(' ';' expr ';' expr ')' stats | LEX_FOR '(' expr ';' ';' expr ')' stats | LEX_FOR '(' expr ';' expr ';' expr ')' // 虽然可以opt_expr : /*empty*/ | expr ';' ; 不就8行嘛。 // 上面的语法是错的,正在改。想来个混合型,bison处理expr,上层用原来的自顶向下。 | LEX_RETURN ';' | LEX_RETURN expr ';' | LEX_BREAK ';' ;
讨厌antlr和Java的理由:
- 基本能看懂的C编译器,只有361行!
- 手撕正则表达式 字符界面画图?我记得Java刚出来时,有FTP客户端用awt还是啥自己画UI控件,界面很漂亮,只是速度慢。现在Java做的界面好像都很寒酸。
bison生成的.cpp更长,但编译起来很快。
$ cat expr.y %{ %} %% expr : ; $ bison -o expr.cpp expr.y $ wc -l expr.cpp 1281 expr.cpp