Code Walkers?

I just finished studying a post by Christophe Rhodes naive vs proper code-walking , in which he rightly points out some weaknesses inherent in the quick and dirty code scanner used by Doug Hoyte’s LOL macro system. And I gather that Christophe is the current maintainer of SBCL.

So I looked in hidden places, and indeed I found a WALKER:WALK-FORM and WALKER:VARIABLE-LEXICAL-P. These appear to work identically with those in SBCL. I also examined the recommended alternates for portable code walking mentioned in his article.

WALK-FORM usefully calls a function on every subform that will be evaluated or macro expanded when the parent form is evaluated. That function receives a subform, a context, and an environment which can be introspected with VARIABLE-LEXICAL-P and a very few other functions.

One does have to bear in mind that we are higher order than LISP-1, and so you have to be careful when looking at forms like ‘(XXX) — arg lists anyone?. Those won’t get opened for examination because, I presume, it appears to have its sole symbol in function position, and would not be evaluated or macroexpanded. Only forms that would be evaluated or macroexpanded get walked. So you have to reform the query as `(LIST ,XXX) in order to scan XXX.

It strikes me that the macro system offered in CL is deficient for not also offering these code walkers. You could try to build a parallel tower on your own, and you’d likely miss a few corner cases during development. But who is the greater authority in any CL system than the code walker actually used by the compiler in that system?

I understand that code walking is likely not publicly offered because it exceeds the charter of CLTL2 and environments were considered implementation dependent. But then so is CAPI. And I’d venture to state that we are more likely to suffer changes to the public API of CAPI than we are to that of the WALKER package.

I welcome any thoughts from others.

- Cheers,

David McClain

Refined Audiometrics Laboratory, LLC

Re: Code Walkers?

David McClain <dbm@refined-audiometrics.com> writes:

> It strikes me that the macro system offered in CL is deficient for
> not also offering these code walkers. You could try to build a
> parallel tower on your own, and you’d likely miss a few corner cases
> during development. But who is the greater authority in any CL system
> than the code walker actually used by the compiler in that system?

But defining a generic code walker is not so easy.  

To give an analogy, if you try to define a generic tree walking, you
will have to provide a lot of hooks, to be able to cover all the cases,
infix, suffix, prefix, and others.  In a code walker, this is compounded
by the number of special operators.

> I understand that code walking is likely not publicly offered because
> it exceeds the charter of CLTL2 and environments were considered
> implementation dependent. But then so is CAPI. And I’d venture to
> state that we are more likely to suffer changes to the public API of
> CAPI than we are to that of the WALKER package.

AFAIK, there are only two things that prevent to write portable code
walkers:

- when implementations define (and use) implementation specific special
  operators without providing an "equivalent" macro.  

- when implementation define extensions to standard special operators.
  This is a big no-no.  For example, one implementation allows
  CL:FUNCTION to take addionnal parameters, and of course, has some
  standard macros expand to such forms.

AFAIK, as long as implementations don't do that and provide "equivalent"
macros for all their non-standard special operators, a conforming
generic code walker can be run on them.

Now, one may have some more requirement than myself for a code walker,
namely, since we are walking code that could be run in the current
implementation, (and that is expanded assumedly from macros of the
current implementation), one may want to have accesses to the
environments of the current implementation while walking code.

This indeed may be a difficulty, but it's related to defining a portable
environment API, not to code walkers per-se.  I'd be perfectly happy
with a code walker that would provide its own environment
representations.

You may also want to have a look at the work of Robert Strandh on SICP,
involving first-class environments.  

-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html