Lisp HUG Maillist Archive

Question about compiled forms...

At this point I think I need to hear from some Lisp experts...

I defined a macro to provide for early binding of functions in forms at compile time:

(defmacro defun-eb (name args &body body)
  (let ((gform (gensym)))
    `(progn
       (defun ,name ,args ,@body)
       (define-compiler-macro ,name (&whole ,gform ,@args)
         `(funcall ,(symbol-function (first ,gform)) ,@(rest ,gform))))
    ))

Use it like this:

(defun foo () (list 10)
(defun bar () (1+ (car (foo))))

then redefine foo to provide an int instead of a list;

(defun foo () 10)

and bar keeps on ticking just like before, until you redefine it as

(defun bar () (1+ (foo))

So far this works great!!

Only problem here is that you can't compile these forms to a file. Lisp complains that these substituted closures in the compiled forms are non-externalizable functions. I have run into this before, and I know that the answer requires pushing these closures in after load time. 

But how does a FASL file do it? It must be loading "non-externalizable functions", compiled to native code, nonetheless...

Any expert opinions on how to get around this issue? and how FASL files manage to pull it off?

If we could do this early-binding then we'd have a hybrid in Lisp that mimics some aspects of ML with its immutable bindings. That would be just great for deployed embedded code, where the full generality of Lisp is both costly and unwarranted. Eh?

Dr. David McClain
Chief Technical Officer
Refined Audiometrics Laboratory
4391 N. Camino Ferreo
Tucson, AZ  85750

email: dbm@refined-audiometrics.com
phone: 1.520.390.3995
web: http://refined-audiometrics.com



Re: Question about compiled forms...


But how does a FASL file do it? It must be loading "non-externalizable functions", compiled to native code, nonetheless...

Any expert opinions on how to get around this issue? and how FASL files manage to pull it off?


I'm no expert, but load-time-value is your friend here.

(defmacro defun-eb (name args &body body)
  (let ((gform (gensym)))
    `(progn
       (defun ,name ,args ,@body)
       (define-compiler-macro ,name (&whole ,gform ,@args)
         `(funcall (load-time-value (symbol-function ',(first ,gform))) ,@(rest ,gform))))))


This isn't portable however as implementations are not required to use the compiler macro.



Sean. 

Re: Question about compiled forms...

On Tue, 26 Aug 2008 17:19:18 -0700, David McClain <dbm@refined-audiometrics.com> wrote:

> But how does a FASL file do it? It must be loading "non-
> externalizable functions", compiled to native code, nonetheless...
>
> Any expert opinions on how to get around this issue? and how FASL
> files manage to pull it off?

FWIW, Clozure CL can externalize compiled closures, so you might want
to look at its source (which is available) to see how they do it.

Edi.


Re: Question about compiled forms...

[ defun's changed to defun-eb's, and missing #\)'s added ]

On Tue, Aug 26, 2008 at 05:19:18PM -0700, David McClain wrote:
> I defined a macro to provide for early binding of functions in forms
> at compile time:
> 
> (defmacro defun-eb (name args &body body)
>   (let ((gform (gensym)))
>     `(progn
>        (defun ,name ,args ,@body)
>        (define-compiler-macro ,name (&whole ,gform ,@args)
>          `(funcall ,(symbol-function (first ,gform)) ,@(rest ,gform))))
>     ))
> 
> Use it like this:
> 
> (defun-eb foo () (list 10))
> (defun-eb bar () (1+ (car (foo))))
> 
> then redefine foo to provide an int instead of a list;
> 
> (defun-eb foo () 10)
> 
> and bar keeps on ticking just like before, until you redefine it as
> 
> (defun-eb bar () (1+ (foo)))
> 
> So far this works great!!
> 
> Only problem here is that you can't compile these forms to a file.
> Lisp complains that these substituted closures in the compiled forms
> are non-externalizable functions. I have run into this before, and I
> know that the answer requires pushing these closures in after load
> time.

As near as I can tell, you have a problem with stuff in your image
overlapping with stuff in your compilation environment.  In your
image, you have a pre-existing interpreted function FOO.  You compile
the file and it tries to externalize that interpreted function into
the fasl, and fails.

To illustrate this, if you put defun-eb and foo and bar in a file and
compile it separately, in a clean image, you get an error when it gets
around to compiling BAR:

  % lispworks-5-1-0-x86-linux -build =(print '(compile-file "play-hug")')
  [ or, if you don't use zsh, do that in two steps, as follows ]

  % echo '(compile-file "play-hug.lisp")' > compile-file.play-hug.lisp
  % lispworks-5-1-0-x86-linux -build compile-file.play-hug.lisp
  LispWorks(R): The Common Lisp Programming Environment
  [snip]
  ; Loading text file /home/lmc/lisp/compile-file.play-hug.lisp
  ;;; Compiling file play-hug.lisp ...
  [snip]
  ; (TOP-LEVEL-FORM 0)
  ; (DEFPACKAGE "HUG")
  ; (TOP-LEVEL-FORM 2)
  ; HUG::DEFUN-EB
  ; HUG::FOO
  ; (DEFINE-COMPILER-MACRO HUG::FOO)

  **++++ Error in HUG::BAR:
    Undefined function FOO in form (SYMBOL-FUNCTION FOO).
  [snip]

> But how does a FASL file do it? It must be loading "non-
> externalizable functions", compiled to native code, nonetheless...
> 
> Any expert opinions on how to get around this issue? and how FASL  
> files manage to pull it off?

Fasl files don't store externalized interpreted functions, they store
compiled functions.  :)

On the other hand, this worked for me: Put this in a file and compile
it from a clean image.

  (defpackage :hug (:use :cl))
  (in-package :hug)
  (defun foo () (list 10))
  (defparameter *foo* #'foo)
  (defun bar () (1+ (car (funcall *foo*))))
  (format t "(foo) is ~S, (bar) is ~S~%" (foo) (bar))
  (defun foo () 10)
  (format t "~%(foo) is ~S, (bar) is ~S~%" (foo) (bar))

Then load the file:

  HUG 62 > (load "play-hug")
  ; Loading fasl file /home/lmc/lisp/play-hug.ufasl
  (foo) is (10), (bar) is 11
  Warning: FOO defined more than once in /home/lmc/lisp/play-hug.lisp.
  (foo) is 10, (bar) is 11
  #P"/home/lmc/lisp/play-hug.ufasl"

Now write a macro to do that.  :)  (But be careful about your
environment, and think carefully about where the function objects come
from when you store them.)

As an aside, doing something like this using compiler macros seems
like a bad idea.  The compiler is allowed to ignore compiler macros,
and just because it doesn't ignore them in one usage doesn't mean it
won't ignore them somewhere else.

-- Larry


Re: Question about compiled forms...

On Wed, 27 Aug 2008 21:23:17 +0100, Martin Simmons <martin@lispworks.com> wrote:

> Is it a bug or a feature?

I don't know.  You'll have to ask them... :)

> We know how to do it too.  We also know why we don't want to :-)

Would you care to give a short explanation for that?  I honestly have
no idea why you wouldn't want to.

Thanks,
Edi.


Re: Question about compiled forms...

Unable to parse email body. Email id is 8606

Updated at: 2020-12-10 08:42 UTC