I'm not in a position to test this myself, but my best guess about what is wrong is that you need to invoke asdf:initialize-source-registry again, after the image restarts. It sounds like you might have tried this.

Minor note: the proper API function is asdf:find-system, which has a simpler return form than locate-system.

Also, when I look at the source registry form you posted, I note that there's an unquoted top in there, but I don't know if that's just a cut-and-paste error.

You can also ask questions on the asdf-devel mailing list (see the project page for ASDF on common-lisp.net)

Finally, have a look at the FAQ in the ASDF manual on the common-lisp.net page: ISTR that I put something in there about how to debug configurations.

Ping me if that is not enough to put you on the right track.

On 9 Jun 2020, at 12:43, Tim Bradshaw wrote:

For some reason I have lost track of I decided I should make things so that various of my packages can use ASDF. I'm using 64-bit LW 7.1.1 on OSX. I think I'm using the bundled ASDF, but I might have updated it. ASDF reports itself as being 3.3.0.

To configure the source registry system I have this form:

(asdf:initialize-source-registry
'(:source-registry
:inherit-configuration
(:tree
(:root "Local" "tfb" "packages" "asdf"top ))))

I know this is not how you are meant to do that, but it means I can have the configuration in a single file which I control with git rather than some obscure config file which I have no idea where it is, and I can put my packages in "/Local/tfb/packages/asdf/...".

And this works fine:

(asdf:locate-system "org.tfeb.hax")

t
nil
#P"/Local/tfb/packages/asdf/org.tfeb.hax/org.tfeb.hax.asd"
nil
nil

All is shiny and good.

Except ... not. This is great in the LW GUI, but I also have console images which are made by

<lispworks-binary> -build make-console-image.lisp

Where make-console-image.lisp is:

(in-package :cl-user)

;;; This is not needed, but harmless
(load-all-patches)

(save-image "lw"
:clean-down t
:console t
:environment nil)
(quit)

And in these images nothing works:

CL-USER 1 > (asdf:locate-system "org.tfeb.hax")
nil
nil
nil
nil
nil

CL-USER 2 > asdf:*source-registry-parameter*
(:source-registry
:inherit-configuration
(:tree (:root "Local" "tfb" "packages" "asdf")))

As you can see the console image has read the init file. I've also tried explicitly clearing the registry and redefining it. No good.

Does anyone understand ASDF well enough to explain what's going on? Every time I try and understand it it just fills me with depression and I end up using LW's defsystem and/or writing another makefile because it's easier, but I also realise that I kind of need to have a defsystem I can use on platforms other than LW if I want to make any of my code available to people.

Just knowing what function I should trace or hook I should set to tell ASDF to show me where it's looking would be good.

Thanks

--tim

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Robert P. Goldman
Research Fellow
Smart Information Flow Technologies (d/b/a SIFT, LLC)

319 N. First Ave., Suite 400
Minneapolis, MN 55401

Voice: (612) 326-3934
Email: rpgoldman@SIFT.net

On 10 Jun 2020, at 6:09, Tim Bradshaw wrote:

I've since also discovered that find-system doesn't actually search for systems as best I can see: that search happens when the source registry is defined. So if you define the source registry at image start time, and there's a big tree to search, the image is slow to start: on my machine

(initialize-source-registry
'(:source-registry
:inherit-configuration
(:tree "/Local")))

makes the image take a great chunk of a minute to start (well, the first time: once the filesystem cache is warm it's better).

Yes, that was done by Faré a long time ago. I believe that this was done to handle very large lisp systems like the ones that ITA was building, but you are right that it doesn't make a great tradeoff for people with more modest systems.

If you like looking for systems on an as-needed basis, you can use asdf:*central-registry*, the old school way of specifying where to look for systems, but note that it does not have the equivalent of :tree. See below

Well, OK, I'm going to revive my antique hack which knew how to search for things based on their names and modify it so it will look for asd files, and make sure that that tree is not where ASDF searches by default so my images start quickly.

I have an equivalent to this hack, called asd-finder. LMK off list if you would like a copy of it.

Also note that if you would like things to start more quickly, you can provide a smaller region for the :tree search to cover, possibly by having two different regions (one for common libraries and one for the actual system you are loading), and searching two smaller trees instead of one big one.

Best,
R

On 11 Jun 2020, at 10:13, Tim Bradshaw wrote:

Also note that if you would like things to start more quickly, you can provide a smaller region for the `:tree` search to cover, possibly by having two different regions (one for common libraries and one for the actual system you are loading), and searching two smaller trees instead of one big one

Yes, in fact the thing that was irritating (and I think still is) is that if I have some external process which takes a system and then dumps it into a place where it should be found, then ASDF doesn't look for it without kicking it.

Yes, ASDF does now lock you into eagerly finding systems, rather than lazily finding them, unless you use asdf:*central-registry*, but if you do that, you can find things lazily.

The docs encourage you to use the new shiny thing, but *central-registry* is simple and has worked for me forever.