Lisp HUG Maillist Archive

CL readtable hack help

Hi, dear Lisp-HUG guys

I'm working on Common Lisp <-> ASN.1 language mapping. I want to hack  
CL readtable to be a ASN.1 lexer, and use LispWorks' Parsergen package  
to translate ASN.1 token list into Common Lisp forms, then compile  
them into CLOS types.

Now I can translate following ASN.1 type definition (only a demo,  
parser not finished):

ASN.1 37 > (parse "Record ::= SEQUENCE {
   name   PrintableString (SIZE (1..30)),
   age    INTEGER,
   gender ENUMERATED { unknown(0)
                       male(1),
                       famale(2) }}")

(DEFCLASS |Record|
           (ASN.1-TYPE)
           ((|name| :TYPE STRING)
            (|age| :TYPE INTEGER)
            (|gender| :TYPE (MEMBER '|unknown| '|male| '|famale|))))
T

But I met some issues when hacking CL readtable:

1. In ASN.1, "--" is the beginning of A comment (to the end of current  
line or next "--" on the same line), so I must set #\- as a macro- 
character. Since I should be break a legal identifier with "-" as a  
part, #\- must be a non-terminal macro-character:

(set-macro-character #\- #'|--reader| t *asn.1-readtable*)

(defun |--reader| (stream char)
   (let ((next-char (read-char stream t nil t)))
     (case next-char
       (#\- (|---reader| stream char next-char))
       ((#\0 #\1 #\2 #\3 #\4 #\5 #\6 #\7 #\8 #\9)
        (unread-char next-char stream)
        (let ((possible-number (read stream t nil t)))
          (etypecase possible-number
            (number (- possible-number))
            (symbol (intern (concatenate 'string "-" (symbol-name  
possible-number)))))))
       (t (unread-char next-char stream)
          '|-|))))

2. In ASN.1, ".." and "..." are used as primitive symbol/token, so I  
must also set #\. as macro character too. I hope "a..b" can be read as  
|a|, |..|, |b| but since I still want to use CL to parse numbers, this  
character also must be non-terminal, so "a..b" only be read as |a..b|,  
this is a little bad:

(set-macro-character #\. #'|.-reader| t *asn.1-readtable*)

(defun |.-reader| (stream char)
   (let ((next-char (read-char stream t nil t)))
     (case next-char
       (#\. (|..-reader| stream char next-char))
       ((#\0 #\1 #\2 #\3 #\4 #\5 #\6 #\7 #\8 #\9)
        (unread-char next-char stream)
        (read-from-string (format nil "0.~D" (read stream t nil t))))
       (t (unread-char next-char stream)
          '|.|))))

3. Above two hacks caused a "minus" number cannot be read directly  
because a number begin with "-" will be done in my |--reader|  
function, my solution is: read the positive number instead and turn it  
into a minus number use #'-

4. Above 3 hacks cause "-2147483648..2147483647" cannot be read as I  
wish: |-| and |2147483648..2147483647|, this is bad, what I want is  
-2147483648, |..| and 2147483647

Any expert can give me some advice on readtable programming? I attach  
the reader source, and the whole source is in my subversion repository  
[1] (quite big)

Regards,

Chun Tian (binghe)

[1] https://cl-net-snmp.svn.sourceforge.net/svnroot/cl-net-snmp/asn.1/trunk


Re: CL readtable hack help

Unable to parse email body. Email id is 8376

Re: CL readtable hack help

On Fri, 11 Jul 2008 20:17:08 +0100, Tim Bradshaw <tfb@cley.com> wrote:

> Both true.  But it's important to realise that the CL readtable was
> not *meant* to be a general lexer.

Oops.  I should have read this one first before answering...


Re: CL readtable hack help

On Sat, 12 Jul 2008 01:11:35 +0800, "Chun Tian (binghe)" <binghe.lisp@gmail.com> wrote:

> But as a general lexer, I think CL readtable has at least two fault:

The CL reader was never intended to serve as a general lexer, so it is
not surprising that it is not the right tool for some jobs.


Updated at: 2020-12-10 08:42 UTC