From bounce-debian-devel=sam=via.ecp.fr@lists.debian.org Tue Apr 15 18:34:14 2003 Return-Path: Delivered-To: sam@via.ecp.fr Received: from murphy.debian.org (murphy.debian.org [65.125.64.134]) by zen.via.ecp.fr (Postfix) with ESMTP id 1CEA03AD01 for ; Tue, 15 Apr 2003 18:34:14 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by murphy.debian.org (Postfix) with QMQP id 9245E1FEA8; Tue, 15 Apr 2003 11:25:56 -0500 (CDT) Old-Return-Path: Received: from glaurung.green-gryphon.com (host-12-107-230-171.dtccom.net [12.107.230.171]) by murphy.debian.org (Postfix) with ESMTP id 9B6341F90C for ; Tue, 15 Apr 2003 11:09:53 -0500 (CDT) Received: from glaurung.green-gryphon.com (srivasta@localhost [127.0.0.1]) by glaurung.green-gryphon.com (8.12.9/8.12.9/Debian-1) with ESMTP id h3FG0elD031160; Tue, 15 Apr 2003 11:00:40 -0500 Received: (from srivasta@localhost) by glaurung.green-gryphon.com (8.12.9/8.12.9/Debian-1) id h3FG0dDV031155; Tue, 15 Apr 2003 11:00:40 -0500 X-Authentication-Warning: glaurung.green-gryphon.com: srivasta set sender to srivasta@debian.org using -f X-Mailer: emacs 21.2.2 (via feedmail 8 I) To: Joey Hess Cc: debian-devel@lists.debian.org Subject: Re: new version breaks some uses of unput() From: Manoj Srivastava Organization: The Debian Project X-URL: http://www.debian.org/%7Esrivasta/ User-Agent: Gnus/5.090019 (Oort Gnus v0.19) Emacs/21.2 (gnu/linux) (i386-pc-linux-gnu) Mail-Copies-To: nobody X-Face: #q.#]5@vq!Jz+E0t_/;Y^gTjR\T^"B'fbeuVGiyKrvbfKJl!^e|e:iu(kJ6c|QYB57LP*|t &YlP~HF/=h:GA6o6W@I#deQL-%#.6]!z:6Cj0kd#4]>*D,|0djf'CVlXkI,>aV4\}?d_KEqsN{Nnt7 78"OsbQ["56/!nisvyB/uA5Q.{)gm6?q.j71ww.>b9b]-sG8zNt%KkIa>xWg&1VcjZk[hBQ>]j~`Wq Xl,y1a!(>6`UM{~'X[Y_,Bv+}=L\SS*mA8=s;!=O`ja|@PEzb&i0}Qp,`Z\:6:OmRi* Date: Tue, 15 Apr 2003 11:00:39 -0500 In-Reply-To: <20030415025606.GA9359@dragon.kitenet.net> (Joey Hess's message of "Mon, 14 Apr 2003 22:56:06 -0400") Message-ID: <87istfn42g.fsf@glaurung.green-gryphon.com> References: <20030415025606.GA9359@dragon.kitenet.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Status: No, hits=-19.0 required=4.0 tests=BAYES_01,IN_REP_TO,REFERENCES,SIGNATURE_SHORT_SPARSE, USER_AGENT_GNUS_UA,X_AUTH_WARNING autolearn=ham version=2.53 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.53 (1.174.2.15-2003-03-30-exp) Resent-Message-ID: Resent-From: debian-devel@lists.debian.org X-Mailing-List: archive/latest/140889 X-Loop: debian-devel@lists.debian.org List-Post: List-Help: List-Subscribe: List-Unsubscribe: Precedence: list Resent-Sender: debian-devel-request@lists.debian.org Resent-Date: Tue, 15 Apr 2003 11:25:56 -0500 (CDT) Status: RO Content-Length: 3899 Lines: 120 --=-=-= Hi, >> On Mon, 14 Apr 2003 22:56:06 -0400, >> Joey Hess said: Joey, I am going to be presumtuous and assume you would not mind this response going to a public list; I have condensed your questions down to the technical details, and you did imply that you considered filing a bug, so the email was not really personal and private. The reason is that there were a number of questions on IRC about similar topics. > But before the code for the provided C functions is put in the C > file, flex now includes a line that undefs yytext_ptr. Yes, part of cleaning up after itself. > I can work around it easily enough, by either moving the functions > to the top code block, or by making them explicitly call yyunput > instead of the unput macro. Putting it in the top code block won't work, since the yyunput function is first implicitly declared, and then explicitly declared static. yyunput is not an published asset; so may go away at any time. In other words, there are macros and stuff available only in the rules section, for use in actions. The `flex' input file consists of three sections, separated by a line containing only `%%'. definitions %% rules %% user code Flex sets up a number of things that are available in the rules section, and now cleans it all up before polluting the user namespace. Now, actions can be any C statement, but using functions that in turn use flex macros is going to be a problem since declaring the function is going to be problematic. Some of these changes were driven by the need to make flex scanners reentrant, others by hte requirements for having multiple scanners, perhaps with different options, in the same program. > Oh and why do I need an expliciy main() and yyweap() function now? > This code always worked before with flex generating stub functions > automatically. The real code is in the filters package, in > cockney.l, jive.l, and others. With multiple scanners now possible in the same program, flex would not know where to put main, obviously, it shoulkd not generate stubs in all the scanners generated. What is yyweap? manoj --=-=-= Content-Disposition: attachment; filename=test.l Content-Description: A version of te test input that compiles fine. %{ /* code block */ %} /* Definitions Section */ BW [ \t\n] SP [ \t]+ EW [ \t.,;!\?$] %% /* Rules Section */ foo /* after regex */ {unput(yytext[yyleng-1]);}; /* after code block */ /* Rules Section (indented) */ %% /* User Code Section */ int main(void) { yylex(); } int yywrap() { return 1; } --=-=-= ps. An important potential problem when using `unput()' is that if you are using `%pointer' (the default), a call to `unput()' _destroys_ the contents of `yytext', starting with its rightmost character and devouring one character to the left with each call. If you need the value of `yytext' preserved after a call to `unput()' (as in the above example), you must either first copy it elsewhere, or build your scanner using `%array' instead. Also, `yytext' presently does _not_ dynamically grow if a call to `unput()' results in too much text being pushed back; instead, a run-time error results. This unput behaviour is not POSIX or lex compliant. Also, The `unput()' routine is not redefinable. -- The bomb will never go off. I speak as an expert in explosives. Admiral William Leahy, U.S. Atomic Bomb Project Manoj Srivastava 1024R/C7261095 print CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C --=-=-=-- -- To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org From bounce-debian-devel=sam=via.ecp.fr@lists.debian.org Tue Apr 15 18:46:59 2003 Return-Path: Delivered-To: sam@via.ecp.fr Received: from murphy.debian.org (murphy.debian.org [65.125.64.134]) by zen.via.ecp.fr (Postfix) with ESMTP id 49EAF3ACF6 for ; Tue, 15 Apr 2003 18:46:59 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by murphy.debian.org (Postfix) with QMQP id 7A0C51F561; Tue, 15 Apr 2003 11:44:10 -0500 (CDT) Old-Return-Path: Received: from kitenet.net (client132.fre.communitycolo.net [64.62.161.42]) by murphy.debian.org (Postfix) with ESMTP id 84FA21F49B for ; Tue, 15 Apr 2003 11:43:56 -0500 (CDT) Received: from dragon.kitenet.net (host-216-184-73-197.peercom.net [216.184.73.197]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (Client CN "Joey Hess", Issuer "Joey Hess" (verified OK)) by kitenet.net (Postfix) with ESMTP id DBB43AA592; Tue, 15 Apr 2003 16:43:55 +0000 (GMT) Received: from joey by dragon.kitenet.net with local (Exim 3.35 #1 (Debian)) id 195TY9-0004v4-00; Tue, 15 Apr 2003 12:44:49 -0400 Date: Tue, 15 Apr 2003 12:44:49 -0400 From: Joey Hess To: Manoj Srivastava Cc: debian-devel@lists.debian.org Subject: Re: new version breaks some uses of unput() Message-ID: <20030415164449.GA18638@dragon.kitenet.net> Mail-Followup-To: Manoj Srivastava , debian-devel@lists.debian.org References: <20030415025606.GA9359@dragon.kitenet.net> <87istfn42g.fsf@glaurung.green-gryphon.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="bg08WKrSYDhXBjb5" Content-Disposition: inline In-Reply-To: <87istfn42g.fsf@glaurung.green-gryphon.com> User-Agent: Mutt/1.5.4i Sender: Joey Hess X-Spam-Status: No, hits=-40.1 required=4.0 tests=BAYES_01,EMAIL_ATTRIBUTION,IN_REP_TO,PGP_SIGNATURE_2, QUOTED_EMAIL_TEXT,QUOTE_TWICE_1,REFERENCES, REPLY_WITH_QUOTES,USER_AGENT_MUTT autolearn=ham version=2.53 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.53 (1.174.2.15-2003-03-30-exp) Resent-Message-ID: <5XysIC.A.6xG.abDn-@murphy> Resent-From: debian-devel@lists.debian.org X-Mailing-List: archive/latest/140890 X-Loop: debian-devel@lists.debian.org List-Post: List-Help: List-Subscribe: List-Unsubscribe: Precedence: list Resent-Sender: debian-devel-request@lists.debian.org Resent-Date: Tue, 15 Apr 2003 11:44:10 -0500 (CDT) Status: RO Content-Length: 3662 Lines: 104 --bg08WKrSYDhXBjb5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Manoj Srivastava wrote: > Joey, I am going to be presumtuous and assume you would not > mind this response going to a public list; I have condensed your > questions down to the technical details, and you did imply that you > considered filing a bug, so the email was not really personal and > private. The reason is that there were a number of questions on IRC > about similar topics.=20 That's fine. Note that the disclaimer you cropped is that I don't claim to know anything about lex. :-) Also that the code dates back to 1986.. > > But before the code for the provided C functions is put in the C > > file, flex now includes a line that undefs yytext_ptr. >=20 > Yes, part of cleaning up after itself. >=20 > > I can work around it easily enough, by either moving the functions > > to the top code block, or by making them explicitly call yyunput > > instead of the unput macro. >=20 > Putting it in the top code block won't work, since the yyunput > function is first implicitly declared, and then explicitly > declared static. yyunput is not an published asset; so may go away at > any time. In other words, there are macros and stuff available only > in the rules section, for use in actions. >=20 > The `flex' input file consists of three sections, separated by a > line containing only `%%'. > definitions > %% > rules > %% > user code >=20 > Flex sets up a number of things that are available in the > rules section, and now cleans it all up before polluting the user > namespace. Now, actions can be any C statement, but using functions > that in turn use flex macros is going to be a problem since declaring > the function is going to be problematic. >=20 > Some of these changes were driven by the need to make flex > scanners reentrant, others by hte requirements for having multiple > scanners, perhaps with different options, in the same program.=20 It seems to promote bad coding style to require that reasonable little functions like this: eos() { if (yytext[yyleng-1] =3D=3D '.') dintI(); else yyunput(yytext[yyleng-1], yytext); } Must be inlined into the rules block or defined as macros or something, instead of just being defined as regular functions. The function above is called many times in the rules block, like this: way printf("why"); [BbHh]it{EW} { printf("%ci'",yytext[0]); eos(); } ait{EW} { printf("ite"); eos(); } ime{EW} { printf("oime"); eos(); } > > Oh and why do I need an expliciy main() and yyweap() function now? > > This code always worked before with flex generating stub functions > > automatically. The real code is in the filters package, in > > cockney.l, jive.l, and others. >=20 > With multiple scanners now possible in the same program, flex > would not know where to put main, obviously, it shoulkd not generate > stubs in all the scanners generated. >=20 > What is yyweap?=20 Typo for yywrap. --=20 see shy jo --bg08WKrSYDhXBjb5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQE+nDcA3xwsXJZQhyMRAnKAAJ4rqFaWKqLVy66yzoZk937aDkRudQCeMUVO 1MF9dE3DDDa9ik2jS7UUkdY= =YEBk -----END PGP SIGNATURE----- --bg08WKrSYDhXBjb5-- -- To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org From bounce-debian-devel=sam=via.ecp.fr@lists.debian.org Tue Apr 15 20:08:35 2003 Return-Path: Delivered-To: sam@via.ecp.fr Received: from murphy.debian.org (murphy.debian.org [65.125.64.134]) by zen.via.ecp.fr (Postfix) with ESMTP id 4D5043ACEB for ; Tue, 15 Apr 2003 20:08:35 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by murphy.debian.org (Postfix) with QMQP id 11D9C1FED4; Tue, 15 Apr 2003 13:05:59 -0500 (CDT) Old-Return-Path: Received: from glaurung.green-gryphon.com (host-12-107-230-171.dtccom.net [12.107.230.171]) by murphy.debian.org (Postfix) with ESMTP id A26C51FEC4 for ; Tue, 15 Apr 2003 12:49:19 -0500 (CDT) Received: from glaurung.green-gryphon.com (srivasta@localhost [127.0.0.1]) by glaurung.green-gryphon.com (8.12.9/8.12.9/Debian-1) with ESMTP id h3FHkDlD003541 for ; Tue, 15 Apr 2003 12:46:13 -0500 Received: (from srivasta@localhost) by glaurung.green-gryphon.com (8.12.9/8.12.9/Debian-1) id h3FHkDCt003537; Tue, 15 Apr 2003 12:46:13 -0500 X-Authentication-Warning: glaurung.green-gryphon.com: srivasta set sender to srivasta@debian.org using -f X-Mailer: emacs 21.2.2 (via feedmail 8 I) To: debian-devel@lists.debian.org Subject: Re: new version breaks some uses of unput() From: Manoj Srivastava Organization: The Debian Project X-URL: http://www.debian.org/%7Esrivasta/ User-Agent: Gnus/5.090019 (Oort Gnus v0.19) Emacs/21.2 (gnu/linux) (i386-pc-linux-gnu) Mail-Copies-To: nobody X-Face: #q.#]5@vq!Jz+E0t_/;Y^gTjR\T^"B'fbeuVGiyKrvbfKJl!^e|e:iu(kJ6c|QYB57LP*|t &YlP~HF/=h:GA6o6W@I#deQL-%#.6]!z:6Cj0kd#4]>*D,|0djf'CVlXkI,>aV4\}?d_KEqsN{Nnt7 78"OsbQ["56/!nisvyB/uA5Q.{)gm6?q.j71ww.>b9b]-sG8zNt%KkIa>xWg&1VcjZk[hBQ>]j~`Wq Xl,y1a!(>6`UM{~'X[Y_,Bv+}=L\SS*mA8=s;!=O`ja|@PEzb&i0}Qp,`Z\:6:OmRi* Date: Tue, 15 Apr 2003 12:46:12 -0500 In-Reply-To: <20030415164449.GA18638@dragon.kitenet.net> (Joey Hess's message of "Tue, 15 Apr 2003 12:44:49 -0400") Message-ID: <87adermz6j.fsf@glaurung.green-gryphon.com> References: <20030415025606.GA9359@dragon.kitenet.net> <87istfn42g.fsf@glaurung.green-gryphon.com> <20030415164449.GA18638@dragon.kitenet.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Status: No, hits=-19.0 required=4.0 tests=BAYES_00,IN_REP_TO,REFERENCES,SIGNATURE_SHORT_SPARSE, USER_AGENT_GNUS_UA,X_AUTH_WARNING autolearn=ham version=2.53 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.53 (1.174.2.15-2003-03-30-exp) Resent-Message-ID: <-j0WfD.A.IuD.GoEn-@murphy> Resent-From: debian-devel@lists.debian.org X-Mailing-List: archive/latest/140894 X-Loop: debian-devel@lists.debian.org List-Post: List-Help: List-Subscribe: List-Unsubscribe: Precedence: list Resent-Sender: debian-devel-request@lists.debian.org Resent-Date: Tue, 15 Apr 2003 13:05:59 -0500 (CDT) Status: RO Content-Length: 3768 Lines: 117 --=-=-= >> On Tue, 15 Apr 2003 12:44:49 -0400, >> Joey Hess said: > Manoj Srivastava wrote: >> Flex sets up a number of things that are available in the rules >> section, and now cleans it all up before polluting the user >> namespace. Now, actions can be any C statement, but using >> functions that in turn use flex macros is going to be a problem >> since declaring the function is going to be problematic. >> >> Some of these changes were driven by the need to make flex >> scanners reentrant, others by hte requirements for having multiple >> scanners, perhaps with different options, in the same program. > It seems to promote bad coding style to require that reasonable > little functions like this: > eos() { > if (yytext[yyleng-1] == '.') > dintI(); > else > yyunput(yytext[yyleng-1], yytext); >> > Must be inlined into the rules block or defined as macros or > something, instead of just being defined as regular functions. Not quite. ---------------------------------------------------------------------- In the rules section, any indented or %{ %} enclosed text appearing before the first rule may be used to declare variables which are local to the scanning routine and (after the declarations) code which is to be executed whenever the scanning routine is entered. Other indented or %{ %} text in the rule section is still copied to the output, but its meaning is not well-defined and it may well cause compile-time errors (this feature is present for POSIX compliance. See *Note Lex and Posix::, for other such features). Any _indented_ text or text enclosed in `%{' and `%}' is copied verbatim to the output (with the %{ and %} symbols removed). The %{ and %} symbols must appear unindented on lines by themselves. ---------------------------------------------------------------------- I'll attach a file at the end that uses functions once again. Just the placement of these function definitions has changed. >> > Oh and why do I need an expliciy main() and yyweap() function >> > now? This code always worked before with flex generating stub >> > functions automatically. The real code is in the filters >> > package, in cockney.l, jive.l, and others. >> >> With multiple scanners now possible in the same program, flex >> would not know where to put main, obviously, it shoulkd not >> generate stubs in all the scanners generated. >> >> What is yyweap? > Typo for yywrap. You must supply a `yywrap()' function of your own, or link to `libfl.a' (which provides one), or use %option noyywrap in your source to say you don't want a `yywrap()' function. manoj --=-=-= Content-Type: text/x-csrc Content-Disposition: attachment; filename=test.l Content-Description: test.l %{ /* COMMENT: code block */ %} %option noyywrap /* COMMENT: Definitions Section */ BW [ \t\n] SP [ \t]+ EW [ \t.,;!\?$] %% /* COMMENT: Rules Section */ %{ /* COMMENT: Define function */ void foo () { unput(yytext[yyleng-1]); } %} foo /* COMMENT: after regex */ foo(); /* COMMENT: after code block */ /* COMMENT: Rules Section (indented) */ %% /* User COMMENT: Code Section */ int main(void) { yylex(); } --=-=-= -- If you live to the age of a hundred you have it made because very few people die past the age of a hundred. George Burns Manoj Srivastava 1024R/C7261095 print CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C --=-=-=-- -- To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org