Runtime loading of code

Hi,

Advance warning: I'm new to common lisp, learning as fast as I can, but 
I'm still easily confused and certainly overwhelmed by the amount of 
reading I have to do and the slightly different terminology. I'm trying 
to justify using LispWorks for a desktop application instead of Java 
which is the incumbent in our company.

I would like to establish that it is or is not possible to dynamically 
load/link a unit of lisp code roughly corresponding to a (set of) fasl 
file(s) into a running application that was compiled with no knowledge 
of the specific functions and CLOS classes defined in the new code. The 
new code would include brand new classes, which won't matter to the 
running application, but these classes will extend common classes and 
provide new methods to already established generic functions.

Can you compile this code without including the common classes?

Can you do this without breaching the runtime license of LispWorks?

If you can't do this without shutting down the host application, can 
you do it with a restart?

I suppose this functionality is very similar to loading new web 
applications into a web application server without restarting the 
server. As I write this, I see that I should be looking into the CL web 
servers out there and see what/if they do this.

Thanks,
Bob

Re: Runtime loading of code

Unable to parse email body. Email id is 1628

Re: Runtime loading of code

This is a very good answer :-) Thanks very much.

This is looking very very promising.

Thanks,
Bob

On Thursday, November 27, 2003, at 01:03  PM, davef@xanalys.com wrote:

>
>    Advance warning: I'm new to common lisp, learning as fast as I can
>
> Welcome!
>
>                                                                       
> , but
>    I'm still easily confused and certainly overwhelmed by the amount of
>    reading I have to do and the slightly different terminology. I'm 
> trying
>    to justify using LispWorks for a desktop application instead of Java
>    which is the incumbent in our company.
>
>    I would like to establish that it is or is not possible to 
> dynamically
>    load/link a unit of lisp code roughly corresponding to a (set of) 
> fasl
>    file(s) into a running application that was compiled with no 
> knowledge
>    of the specific functions and CLOS classes defined in the new code. 
> The
>    new code would include brand new classes, which won't matter to the
>    running application, but these classes will extend common classes 
> and
>    provide new methods to already established generic functions.
>
> Yes. A running Lisp image can load compiled code which defines new
> classes and methods. The LispWorks patch loader is one example, and a
> loadable module such as "corba" is another. There also exist LispWorks
> applications which have their own patch loaders.
>
>    Can you compile this code without including the common classes?
>
> Yes.
>
> If I understand your question, the common classes are already existant
> in the compiling image. For example CL:STANDARD-OBJECT is part of the
> language, so it's always defined at compile time.
>
> But Common Lisp is more flexible than that, as it is possible to
> define a class FOO inheriting from an as-yet-undefined class BAR. The
> restriction is that you cannot instantiate FOO until BAR is defined.
>
>    Can you do this without breaching the runtime license of LispWorks?
>
> Yes.
>
> --
> Dave Fox			
>
> Xanalys                 http://www.lispworks.com
> Compass House
> Vision Park, Chivers Way
> Histon
> Cambridge, CB4 9AD
> England
>
>
>

Re: Runtime loading of code

Bob Hutchison <hutch@recursive.ca> writes:

> I would like to establish that it is or is not possible to dynamically
> load/link a unit of lisp code roughly corresponding to a (set of) fasl
> file(s) into a running application that was compiled with no knowledge
> of the specific functions and CLOS classes defined in the new
> code. The new code would include brand new classes, which won't matter
> to the running application, but these classes will extend common
> classes and provide new methods to already established generic
> functions.
>
> Can you compile this code without including the common classes?

You got a good answer from Dave, I'd like to add a little user
experience: With my former employer, we not only did (and they still
do) exactly what you describe here, but even partly generated and
changed the classes dynamically from descriptions stored in a
database, using a little of the MOP (e.g. ensure-class).

Where I currently work, we also do live patching of servers with fasl
files (I loaded a fasl file with a bug fix into 8 running servers
during peak usage hours just yesterday), and we also distribute
patches to end user applications as fasl files, automatically down-
loaded by the end user application.

Note that the most common use of such fasl files is not only brand
new classes and functions, but redefinitions of old ones!
-- 
  (espen)

CAPI dynamicity (Re: Runtime loading of code)

Espen Vestre <ev@netfonds.no> writes:

> Note that the most common use of such fasl files is not only brand
> new classes and functions, but redefinitions of old ones!

I'd like to add a question to the LW guys here: An obvious exception
here is the case of redefining CAPI interface classes, which does
not work in a 100% dynamic way (things tend to break in already
displayed interfaces). Is there any chance you can make CAPI more
dynamic in this sense, or is it simply too difficult?
-- 
  (espen)

Re: Runtime loading of code

* Bob Hutchison wrote:
> I would like to establish that it is or is not possible to dynamically 
> load/link a unit of lisp code roughly corresponding to a (set of) fasl 
> file(s) into a running application that was compiled with no knowledge 
> of the specific functions and CLOS classes defined in the new code. The 
> new code would include brand new classes, which won't matter to the 
> running application, but these classes will extend common classes and 
> provide new methods to already established generic functions.

To add to what other people have said: yes this is possible, and it
works well.  We have an application which is essentially a core
framework with a bunch of loadable modules together with possible
patches to those modules, all of which are simply chunks of compiled
Lisp code. When it starts, the configuration file specifies which
modules to be loaded & loads them and patches to them.  This all works
fine.  It can also load stuff later in fact, although we generally
don't do that.

From the LW point of view, you can make loadable chunks of code by
concatenating systems together, which makes one huge fasl file which
can then be loaded.  If you're loading code that defines or redefines
classes which are subclasses of classes in a delivered image, there
are some options you need to DELIVER to keep enough information
around.  I'm not completely sure what they are (Dave Fox undoubtedly
knows!): what we have is

     :keep-clos :no-empty

Which I must have found either by reading the documentation or by
asking Xanalys.

--tim

Re: Runtime loading of code

Bob Hutchison wrote:

> I suppose this functionality is very similar to loading new web
> applications into a web application server without restarting the
> server. As I write this, I see that I should be looking into the CL web
> servers out there and see what/if they do this.

I always do this with my web applications but also with my industrial
applications.
As there are lots of RAM on the computers used for these applications, I
generally deliver at level 0 and I keep all. Just to be sure I'm not missing
some piece of code.

Marc

Re: CAPI dynamicity (Re: Runtime loading of code)

Unable to parse email body. Email id is 1647