FAQ Database Discussion Community


write parsed tokens and YYTEXT of flex lexical analyser to a file

compiler-construction,flex-lexer,lexical-analysis
I need to write the token and the text parsed of that token in a file with flex analyser. Basically I want to store each parsed token in an output file. Someone has some idea?...

Call flex yy_push_state() from bison parser

bison,flex-lexer
Is it possible to call yy_push_state() from a bison generated parser? How can this be done? context: /* empty */ { $$ = NULL; yy_push_state(SOME_STATE); } ; rule: context operator STRING { create_expr($2, $3); } ; I would like to be able to call yy_push_state() from parser and also would...

Multiple definitions?

c++,parsing,bison,yacc,flex-lexer
I'm having trouble with compiling my flex and bison code. more specifically my parser.yy file. In this file I included MathCalc.h and BaseProg.h, which are classes I've made. The issue is when I instantiate the classes, it gives me a "multiple definition" error on compilation. Any help would be appreciated!...

Flex & Bison Segmenation fault

c,bison,flex-lexer
I to make a compiler and I use flex and bison for that. I'm implementing the boolean comparision. But I have an error when bison goes into my token COMP. I have declared like that : ("==")|(">=")|("<=")|("!=")|(">")|("<") { sscanf(yytext,"%s",yylval.svalcmp); return COMP; } In bison my gramar is : ExpBool :...

multiple String literal in flex

regex,string,lex,flex-lexer,string-literals
I'm using flex to parse a whole buncha stuff, but I hit a roadbloack when I tried to detect two string literals on the same line. my regex: ["].*["] heres what I mean: "cats" < "dogs" is being recognized as one long string cats" < "dogs Why is flex only...

Lex/Flex :Regular expression for string literals in C/C++?

c++,c,regex,flex-lexer,lex
I look here ANSI C grammar . This page includes lot of regular expressions in Lex/Flex for ANSI C. Having problem in understanding regular expression for string literals. They have mentioned regular expression as \"(\\.|[^\\"])*\" As i can understand \" this is used for double quotes , \\ is for...

named variables ambiguous with other token identifiers

regex,yacc,flex-lexer,lex
Im using a lexer that needs to be able to identify the difference between a named variable and a keyword. to elaborate, in my .l file I have some definitions like "QUIT" {return QUIT;} "AND" {return AND;} "XOR" {return XOR;} and also I have the definition of a name(for a...

Troubles with Flex/Bison

grammar,bison,flex-lexer
I try set up grammar using flex/bison by next pattern: DATA: 1,2,3,4,5 PROGRAM: add,mult,div,read This input must be transfered into 4 add mult div read 5 1 2 3 4 5 Where: 4 is a number of commands after "PROGRAM" and 5 is a number of data after "DATA". At...

checking unfinished comments in flex

c,flex-lexer,lex,lexical-analysis,lexical-scanner
I am a new to flex. I have just written a sample code to detect multi line comments using a flex program. Now I want to improve the code. I want to detect unfinished and ill formed comments in the code. for example: a comment beginning with /* without an...

Implementing Wolfram language in flex, bison, c++

c++,bison,flex-lexer,wolfram-language
After seeing projects like mathics and symja, I am trying to implement an open-source parser for the Wolfram language using flex and bison in C++ . Invoking bison -d and flex++ don't raise any issues, but when I use g++, I get the following error message: parser.tab.cpp:1242:16: error: use of...

Checking wrong identifier patterns in flex

compiler-errors,flex-lexer,lex,lexical-analysis,lexical-scanner
I am just trying to learn flex and here is a sample code in flex to detect identifiers and digits. I want to improve the code by identifying wrong identifier and digit patterns (for example: 1var,12.2.2,5. etc). How I will detect it? which change do I have to make in...

Detecting ill formed strings and comments in flex

flex-lexer,lex,lexical-scanner
I am just learning flex and I have written a flex program to detect a given word is verb or not. I will take input from a text file.I want to improve the code. I want to detect if there is any ill formed or unfinished string in the code.Unfinished...

Can't compile flex & bison (Symbols not found x86_64)

osx,bison,osx-yosemite,flex-lexer
I am trying to compile a simple program on Flex & Bison on my Mac running Yosemite but get the following error: Undefined symbols for architecture x86_64: "_yyerror", referenced from: _yyparse in pr1-19c182.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use...

Debug assertion failed error in flex / bison program when trying to program “include” functionality

c,bison,yacc,flex-lexer,lex
I get a "debug assertion failed... Expression stream != NULL" error when running a flex / bison program. Here is the relevant code, at the top of the lex file: %x include_state %{ #define MAX_INCLUDE_DEPTH 10 YY_BUFFER_STATE include_stack[MAX_INCLUDE_DEPTH]; int include_stack_ptr = 0; %} ... and later in the lex file:...

jison grammar definition leads to wrong token recognition

parsing,yacc,flex-lexer,jison
I recently found the project jison and modified the calculator example from its website. (http://zaach.github.io/jison/demos/calc/) /* lexical grammar */ %lex %% "a" return 'TOKEN1' "b" return 'TOKEN2' <<EOF>> return 'EOF' . return 'INVALID' /lex %start letters %% /* language grammar */ letters : | letters letter ; letter : 'TOKEN1'...

bison unexpected identifier error[SOLVED]

grammar,bison,yacc,flex-lexer,lex
I'm getting an "unexpected identifier error", at 1-9, on the line string_op | string_lit { $$ = $1; } here is my grammar %union { int intval; double dubval; char* strval; obj object; } %token <intval> INTEGER %token <dubval> DOUBLE %token <strval> STRING_LITERAL %type <object> number factor value term constant...

Linking CUP and jflex

java,flex-lexer,jflex,cup
I am trying to link my parser.java and yylex.java using help from http://www2.cs.tum.edu/projects/cup/examples.php http://www.cs.princeton.edu/~appel/modern/java/CUP/manual.html http://jflex.de/manual.html But I am getting these errors. error: Yylex is not abstract and does not override abstract method next_token() in Scanner error: next_token() in Yylex cannot implement next_token() in Scanner How to resolve them ? My...

Why does this scanner not eat whitespaces?

regex,whitespace,bison,flex-lexer
These are my lexer-definitions, there are many lexer-definitions but this one is mine. I have several regexes trying to capture and ignore whitespace, from this sample. The error I get is that in line 1: 14 there is a $undefined Symbold to be found - that is of asci-value 32....

How to do proper error handling in BNFC? (C++, Flex, Bison)

c++,bison,flex-lexer,bnfc
I'm making a compiler in BNFC and it's got to a stage where it already compiles some stuff and the code works on my device. But before shipping it, I want my compiler to return proper error messages when the user tries to compile an invalid program. I found how...

Seg fault using flex, yy_scan_string(), and input() until end of input

c++,c,flex-lexer
The following flex code seems to create an executable that seg faults when the input string is an unterminated comment. Note that: this only occurs when the input buffer is a string (e.g. with yy_scan_string()) rather than a file (yyset_in()) the seg fault occurs when the flex code tries to...

Flex / Yacc program causing breakpoint on free instruction in VC++

c++,c,visual-c++,yacc,flex-lexer
I have Flex / Yacc program that is causing a breakpoint when it is run in the VC++ 2012 IDE. The breakpoint occurs on the instruction (in pre_lxr.l below): free(pre_fname); The project contains the lexer (.l file), the yacc file (.y) and an interface file (which sits between the parser...

How can I instruct the parser not to continue processing unterminated comments?

bison,yacc,flex-lexer
I'm working on improving error reporting on my compiler assignment. I'm handling unterminated comments in Flex using the following code: <INITIAL>"/*" {BEGIN(COMMENT);} <COMMENT>"*/" {BEGIN(INITIAL);} <COMMENT>([^*]|\n)+ {} <COMMENT><<EOF>> {yyerror("UNTERMINATED COMMENT"); BEGIN(INITIAL);} The issue is that the parser is printing its error message as well: $ ./comp tests/comments.cf ERROR: UNTERMINATED COMMENT: 27...

Storing the current line being analysed by flex

c++,parsing,bison,flex-lexer
In my parser generated by flex, I would like to be able to store each line in the file, so that when reporting errors, I can show the user the line that the error occurred on. I could of course do this using a vector and read in all lines...

Resetting the state of flex and/or bison

parsing,bison,flex-lexer
As part of a toy project I've been trying to make a small modification of someone else's parser based on flex/bison. I'm really not experienced with either. You can find the original parser here. I've been trying to put together a simple function that accepts a string and returns a...

flex -l longest pattern match strategy - not here?

compatibility,flex-lexer,lex
I have two lex rules and was wondering why I never matched the second rule. Instead rule 1 always fired upon the pattern 2005-05-09-11.23.04.790000 <data>[-]?[0-9]*[.][0-9]* { comma=0; printf("DEBUG: data 1 %s\n",yytext); strcat(data_line,yytext); } <data>[0-9]{4}[-][01][0-9][-][0-3][0-9][-][0-9]{2}[.][0-9]{2}[.][0-9]{2}[.][0-9]{6} { printf("DEBUG: data 2[%s]\n",yytext); /* 1996-07-15-hh.00.00*/ I thought, flex/lex would follow the longest string match rule?...

Where to free up memory allocated (for union) with _strdup in lex / yacc program?

c,bison,yacc,flex-lexer,lex
I have defined the following union structure in my yacc / bison file: %union { int num; double dbl; char ch; char *str; } In my lex / flex file I have the following match: [a-zA-Z][a-zA-Z0-9"_"]* { yylval->str = _strdup(yytext); return id; } My question, where do I put the...

(Gnu) make: multiple targets with special compiler switches

design-patterns,makefile,bison,flex-lexer
I have a problem getting a makefile to work. I have several parsers that compile XML schemas as follows: $(srcdir)/schema_1_parser.cpp: \ $(srcdir)/schema_1_parser.l \ $(srcdir)/schema_1_parser.tab.cpp $(LEX) -Pschema_1 -o$(srcdir)/schema_1_parser.cpp \ $(srcdir)/schema_1_parser.l $(srcdir)/schema_1_parser.tab.cpp $(srcdir)/schema_1_parser.tab.hpp: \ $(srcdir)/schema_1_parser.y $(YACC) -ldv -p schema_1 -o $(srcdir)/schema_1_parser.tab.cpp \ $(srcdir)/schema_1_parser.y There are multiple such rule pairs, using schema_2, schema_3,...

Flex (lexer) - matching unicode

ruby,unicode,cmake,flex-lexer
Is there a way to get flex to match unicode along the lines of ascSymbol !|#|$|%|&|⋆|+|.|/|<|=|>|?|@|\|^|-|~|: uniSymbol \p{Symbol}|\p{Other_Symbol}|\p{Punctuation} symbol ascSymbol|uniSymbol{-}[^|_"',;] I found http://lists.gnu.org/archive/html/help-flex/2005-01/msg00043.html via Flex(lexer) support for unicode but I'd want to be able to something in an automated way. For example, I'm using cmake and it is configured to...