FAQ Database Discussion Community


how to write lex file for input like “{\”a\“:1,\”b\“:2}”

json,parsing,racket,tokenize,lex
I want to implement a json parser, but having problem with parse object like "{\"a\":1,\"b\":2}", currently the parser output somthing like this '(json (object "{" (kvpair "\"a\":1,\"b\"" ":" (json (number "2"))) "}")) but what i actually want is '(json (object "{" (kvpair "\"a\"" ":" (json (number "1"))) "," (kvpair "\"b\""...

Lex/Flex :Regular expression for string literals in C/C++?

c++,c,regex,flex-lexer,lex
I look here ANSI C grammar . This page includes lot of regular expressions in Lex/Flex for ANSI C. Having problem in understanding regular expression for string literals. They have mentioned regular expression as \"(\\.|[^\\"])*\" As i can understand \" this is used for double quotes , \\ is for...

Checking Valid Arithmetic Expression in Lex (in C)

c,regex,validation,lex
I have to write code for checking if an arithmetic expression is valid or not , in lex. I am aware that I could do this very easily using yacc but doing only in lex is not so easy. I have written the code below, which for some reason doesn't...

Error while compiling lex file

lex
%{ #include <stdio.h> #include "y.tab.h" extern int yylval; %} %% [a-zA-Z] { yylval= *yytext[0]; return ID; } [0-9] { yylval= *yytext[0]; return NUM;} . return yytext[0]; \n return 0; %% i'm compiling this lex file along with the yacc file, when I hit the following command (cc lex.yy.c y.tab.h -ll)the...

“first use” error when change the code in lex file

yacc,lex
Given a .l file like this: %{ #include "y.tab.h" %} %% [ \t\n] "if" return IF_TOKEN ; "while" return ELSE_TOKEN ; . yyerror("Invalid Character"); %% int yywrap(void){ return 1; } and a .y file like this: %{ #include <stdio.h> void yyerror(char *); %} %token IF_TOKEN ELSE_TOKEN MINUS_TOKEN DIGIT_TOKEN %% program...

Including additional rules with flex

include,lex
I've been working on a small assembler which uses flex, however, the flex rule list is reasonably long. Ideally, I'd like to solve this by splitting the rules into several files which can be included into the primary lex file. My searching has turned up nothing of relevance which leads...

Removing comments using lex: why doesn't this work?

python,lex,ply
I'm writing a parser using Python/lex and trying to create an entry to remove C-style comments. My current (faulty) attempt is: def t_comment_ignore(t): r'(\/\*[^*]*\*\/)|(//[^\n]*)' pass This produced a quirk that baffled me. When I parse the string below: input = """ if // else mystery =/*=*/= true /* false */...

Lex match an angle bracket literally

regex,lex
I can't seem to get this lex regex working: %{ #include"y.tab.h" %} %option yylineno /* regular definitions */ angle_bracket_start "<" %% angle_bracket_start /*swallow it, do nothing!*/{} %% But when I test it with lex lex.l gcc lex.yy.c -lfl I got: $ ./a.out < < <--- If it prints out the...

lexical analysis stops after yy_scan_string() is finished

c,lex
I use flex to make a lexical analyzer. I want to analyse some define compiler statements which are in the form: #define identifier identifier_string. I keep a list of (identifier identifier_string) pair. So when I reach in the file a identifier that is #define list I need to switch the...

Lex and yacc program to find palindrome string

yacc,lex
Here are my lex and yacc file to recognise palindrome strings but it is giving "INVALID "for both valid as well as invalid string. Please help me to find the problem, I am new to lex and yacc. Thanx in advance LEX file %{ #include "y.tab.h" %} %% a return...

multiple String literal in flex

regex,string,lex,flex-lexer,string-literals
I'm using flex to parse a whole buncha stuff, but I hit a roadbloack when I tried to detect two string literals on the same line. my regex: ["].*["] heres what I mean: "cats" < "dogs" is being recognized as one long string cats" < "dogs Why is flex only...

checking unfinished comments in flex

c,flex-lexer,lex,lexical-analysis,lexical-scanner
I am a new to flex. I have just written a sample code to detect multi line comments using a flex program. Now I want to improve the code. I want to detect unfinished and ill formed comments in the code. for example: a comment beginning with /* without an...

Debug assertion failed error in flex / bison program when trying to program “include” functionality

c,bison,yacc,flex-lexer,lex
I get a "debug assertion failed... Expression stream != NULL" error when running a flex / bison program. Here is the relevant code, at the top of the lex file: %x include_state %{ #define MAX_INCLUDE_DEPTH 10 YY_BUFFER_STATE include_stack[MAX_INCLUDE_DEPTH]; int include_stack_ptr = 0; %} ... and later in the lex file:...

Lex priority label opcode priority

lex
I am using lex / yacc to write an assembler I have some opcodes for example ORA [Oo][Rr][Aa] AND [Aa][Nn][Dd] EOR [Ee][Oo][Rr] and rules {ORA} { yylval.iValue = ora; return OPCODE; } {AND} { yylval.iValue = and; return OPCODE; } {EOR} { yylval.iValue = eor; return OPCODE; } I also...

named variables ambiguous with other token identifiers

regex,yacc,flex-lexer,lex
Im using a lexer that needs to be able to identify the difference between a named variable and a keyword. to elaborate, in my .l file I have some definitions like "QUIT" {return QUIT;} "AND" {return AND;} "XOR" {return XOR;} and also I have the definition of a name(for a...

Lex & yacc parsing statements

yacc,lex
I am trying to build a very simple language using lex and yacc It can have only one int variable through assignment like this a = 1 It can print variable like this print a It has to print the value only if variable name matches else it has to...

Where to free up memory allocated (for union) with _strdup in lex / yacc program?

c,bison,yacc,flex-lexer,lex
I have defined the following union structure in my yacc / bison file: %union { int num; double dbl; char ch; char *str; } In my lex / flex file I have the following match: [a-zA-Z][a-zA-Z0-9"_"]* { yylval->str = _strdup(yytext); return id; } My question, where do I put the...

Checking wrong identifier patterns in flex

compiler-errors,flex-lexer,lex,lexical-analysis,lexical-scanner
I am just trying to learn flex and here is a sample code in flex to detect identifiers and digits. I want to improve the code by identifying wrong identifier and digit patterns (for example: 1var,12.2.2,5. etc). How I will detect it? which change do I have to make in...

Detecting ill formed strings and comments in flex

flex-lexer,lex,lexical-scanner
I am just learning flex and I have written a flex program to detect a given word is verb or not. I will take input from a text file.I want to improve the code. I want to detect if there is any ill formed or unfinished string in the code.Unfinished...

bison unexpected identifier error[SOLVED]

grammar,bison,yacc,flex-lexer,lex
I'm getting an "unexpected identifier error", at 1-9, on the line string_op | string_lit { $$ = $1; } here is my grammar %union { int intval; double dubval; char* strval; obj object; } %token <intval> INTEGER %token <dubval> DOUBLE %token <strval> STRING_LITERAL %type <object> number factor value term constant...

Wrong lex state when parsing multiple files

c,bison,yacc,lex
I am trying to parse two files with win flex and bison, but I am encountering a problem where lex is not in the state I am expecting. In the lex file: include[ \t]+\" { BEGIN(include_state); } <include_state>([^\\\"\n]|\\.)+ { yyin = fopen(yytext, "r"); if (!yyin) { printf("Error opening include file:...

Error while compiling Concurrent YACC program

c,multithreading,yacc,lex
I am trying to build implement a basic calculator using Concurrent YACC. I have tried the code by statically creating the threads. But whe I want to dynamically specify how many threads to be created, the parser seems to have a problem. Here are the contents of my code. aa.y...

How to print comments in lex?

comments,lex,lexical-analysis,lexical-scanner
So the title might be a little bit misleading, but I can't think of any better way to phrase it. Basically, I'm writing a lexical-scanner using cygwin/lex. A part of the code reads a token /* . It the goes into a predefined state C_COMMENT, and ends when C_COMMENT"/*". Below...

Regular expression for HTML tags

html,regex,lex,lexical-analysis
I am working on Lexical Analyzer. I have an HTML file. I want to convert every letter in the file expect whatever written within an HTML tag into CAPITAL letter. Example: <html> <body> StackOverFlow </body> </html> This will be converted to following. <html> <body> STACKOVERFLOW </body> </html> I just want...

flex -l longest pattern match strategy - not here?

compatibility,flex-lexer,lex
I have two lex rules and was wondering why I never matched the second rule. Instead rule 1 always fired upon the pattern 2005-05-09-11.23.04.790000 <data>[-]?[0-9]*[.][0-9]* { comma=0; printf("DEBUG: data 1 %s\n",yytext); strcat(data_line,yytext); } <data>[0-9]{4}[-][01][0-9][-][0-3][0-9][-][0-9]{2}[.][0-9]{2}[.][0-9]{2}[.][0-9]{6} { printf("DEBUG: data 2[%s]\n",yytext); /* 1996-07-15-hh.00.00*/ I thought, flex/lex would follow the longest string match rule?...