Re: In lisp, what is a module?
I will put lisp functions, macros, etc. in {curly braces} because I think some people are communicating in plain text.
So, that’s a good article. But now I look at the Lisp function {require}. Seems that {require} is both deprecated and ***recommended*** for loading modules. And we have the {*modules*} global variable associated with using {require}.
I understand that a package can span multiple files. And also the reverse is true that one file can contain multiple packages ({in-package}). So, clearly, packages are not files. A Lisp program pretty much automatically is composed of multiple packages because you generally are going to at least {use-package} {common-lisp}, and {defpackage} your own package or packages. Or, if you don’t {defpackage} your own package, you are still at least developing in {cl-user} and tacitly {use-package} {common-lisp} so that your lisp code will work.
You could do something weird like {in-package} to {common-lisp}. Then you are adding your own functions and code to Lisp itself!. Seems like bad practice to me, unless your job is something like developing the next version of LispWorks.
I understand package as just a namespace of symbol names. Packages allow re-using the same symbol names in different packages.
I don’t know. Maybe today’s Common Lisp mostly doesn’t talk about or deal with modules and systems with a few glaring exceptions due to historical reasons and tradition?
I suspect that outside of Lisp, module and system have a different meaning than inside Lisp. In Lisp, is a module always exactly the same as a file? Is a system always exactly the same as a program?
You can assume that system and module are fundamentally the same things; the only difference being that modules are defined in terms of REQUIRE and PROVIDE and listed in the *MODULES* variable.
On the other hand, systems are implemented using user-level libraries, such as the various versions of defsystem, or nowadays, asdf. quicklisp implements the downloading and installation of asdf systems, so the function ql:quickload may performs the three actions (download, install and load the asdf system).
Furthermore, some implementations provide hooks into the module system, so that you can map module names to asdf system names, and quickload or asdf load them. I wouldn’t recommend it, but this gives you an integration, letting you use the REQUIRE function to load asdf systems.
There is a conforming way to use REQUIRE, which is to pass it a second argument which his the pathname of the file to be loaded. It is them equivalent to:
(defun require (module &optional pathname)
(if pathname
(unless (member module *modules*) (load pathname))
(do-something-implementation-specific module)))
The loaded module files are responsible to evaluate (provide module) so that the module be added to the *modules* variable.
The implementation specific module loading thing is not too different, in general the only implementation specific thing is the the locating of the module file.
But as mentioned above, there may also be hooks. On the other hand, you may also define an asdf system, with a single file containing a REQUIRE form (perhaps conditionalized for various implementations with #+), so that you may load modules using asdf (or quicklisp).
Now what systems (or modules) are? They may be libraries or programs. There’s not much difference between libraries and programs. In both case, you can call functions from the REPL.
However, what we would avoid, is to run lengthy processes or start threads at load time. We don’t expect in general that (ql:quickload :some-program) would run the program.
Instead it is expected to call some function named MAIN or RUN or something else, in some package, after having loaded the system. Or even, to save a lisp image using this function as top-level function to generate an executable.
We may also make the distinction between libraries and programs on details. But I would rather make that distinction on code distributed thru quicklisp (or other means) vs. code defined by the end-user (or end-developer if you will) that won’t be shared with other developers.
A library must avoid mutating global common state, such as defining stuff in a package that is not specific to that library (ie. it should avoid short package names that could easily be chosen by other libraries as well), because otherwise other libraries could also mutate the same global common state and it would render them incompatible (eg. if two libraries use a package named “OPTION” to define in it functions, say one package to process command line options, and the other package to process financial options, they could override each other functions and make a very bad mess; my advice is to use package names such as com.informatimago.command-line.option and com.richtrader.tradex.client.option ; perhaps a little longer to type, but no risk of collision: DNS registration already resolved them! (Notice, there are at least 3 systems in quicklisp to process command line options! and at least as many to process JSON! They cannot all be called CL-OPTION or CL-JSON!). This doesn’t concern only package names, there is of course a lot of global state in a lisp image, such as the *READTABLE*, (all the global CL variables with stars around their names). An example is the *features* variable which contains symbols that can be tested with the reader macros #+ and #-. Usually the implementation puts keywords in this list, that you can test easily without writing the colon, since the expression following #+ and #- is read in the KEYWORD package. But if libraries do the same, they may confuse one another. So instead, it is advised in general that system push onto *features* symbols from their very own packages, to avoid any risk of collision. They then have to be tested using the qualified symbol of course.
As mentioned above, a program would provide a function (or a few functions) to start the program. Eg. a server could provide a start and a stop function. We’re not limited to the “main” function of a C program! ;-)
Programs, or applications may also easily try to access global resources in an exclusive way (eg. with most GUI, there’s a single application object; a program would naturally create and use this singleton), but this would make it impossible to load and run different programs in the same lisp image! While it is acceptable (it is programs after all), it may be nice if possible to avoid such exclusive use, in programs that are distributed as systems (eg. thru quicklisp). Eg. instead of using the application singleton, design the applications as a “module” that can be integrated with other “modules” in a common application.
On the other hand, the end-user may write a program using global resources without any difficulty since he won’t be distributing it to other developers. Eg. he may load systems defining packages, and then rename those packages to short names (he will obviously avoid naming conflicts), and then use those short names in his own (end-user) code. He may define an application, using global and exclusive resources such as an application singleton, and integrating into it multiple libraries and programs systems, if they are modular in that way.
Well, this is my advice, but of course, there are a lot of program or application systems out there that don’t follow this recommendation. You wouldn’t be able to load both hemlock and maxima systems and integrate them easily into a single end-user application. Some heavy patching would be required.
And a final note to further confuse the terminology, asdf systems are made of components and of modules! But they’re not the same module as CL modules of course, they’re just a way of structuring complex systems. Read the documentation of asdf.
--
__Pascal J. Bourguignon__