Lisp HUG Maillist Archive

Using both a restart function and multiprocessing when saving an image

Greetings Lispworks Community,

Is there a way to save an image with :multiprocessing t and also a
:restart-function, so that when the restart-function exits,
multiprocessing remains enabled?

The log included below the signature shows how multiprocessing exits,
even when I didn't ask for it to stop (I assume this is because
my-loop was called in a separate thread, and since that thread exited,
the system thinks that multiprocessing is no longer needed, which
isn't the case in our application).  I'm looking for the "right way"
to keep multiprocessing enabled, even after "my-loop" exits.  I
include a longer version of this question after the log.

Thank you,
David

=== My-loop log ===

oliphaunt-0% /u/ragerdl/lisps/lispworks-6.0.1/lw-terminal -init - -siteinit -
LispWorks(R): The Common Lisp Programming Environment
Copyright (C) 1987-2009 LispWorks Ltd.  All rights reserved.
Version 6.0.1
Saved by ragerdl as lw-terminal, at 11 Apr 2011 14:01
User kaufmann on oliphaunt-0

CL-USER 1 > (defun my-loop ()
             (loop
              (princ "prompt> ")
              (let ((x (read)))
                (format t "Just read: ~s~%" x)
                (when (eq x :q)
                  (return-from my-loop nil)))))
MY-LOOP

CL-USER 2 > (system::save-image "lw-mp" :multiprocessing t
:restart-function 'my-loop)
#P"lw-mp"

CL-USER 3 > (quit)
oliphaunt-0% ./lw-mp -init - -siteinit -
LispWorks(R): The Common Lisp Programming Environment
Copyright (C) 1987-2009 LispWorks Ltd.  All rights reserved.
Version 6.0.1
Saved by kaufmann as lw-mp, at 17 May 2011 15:11
User kaufmann on oliphaunt-0
prompt> 3
Just read: 3
prompt> 4
Just read: 4
prompt> :q
Just read: :Q

;; No live processes except internal servers - stopping multiprocessing

CL-USER 3 > mp::*multiprocessing*
NIL

CL-USER 4 >


=== Longer version of the question ===

I have an experimental modification to ACL2 that allows the main proof
process of ACL2 to evaluate in parallel.  ACL2 contains a
read-eval-print-loop.  Thus far our system has been designed for CCL
and SBCL, which have compile-time options for enabling
multiprocessing.  Lispworks has a similar option for when one saves
the image.  However, it's still possible to "return from"
multiprocessing once you start that image.  This ability to exit the
multi-processing loop without explicitly typing
"mp::stop-multiprocessing" is problematic for us.

So, I'm looking into a new implementation of our multi-threading for
Lispworks, which is based off the multi-processing example in the
Lispworks manual
[http://www.lispworks.com/documentation/lw60/LW/html/lw-287.htm#pgfId-890248].
 We have a function, "lp", which, when called, causes the prompt to
enter the ACL2 read-eval-print-loop.  So, I've renamed "lp" to "lp1"
and added the following Lispworks code:

(defvar *acl2-repl-thread*
 '("acl2-read-eval-print-loop" nil lp1))

(defun guarantee-process ()
 (unless mp::*multiprocessing*
   (pushnew *acl2-repl-thread*
            mp:*initial-processes*)
   (mp:initialize-multiprocessing)))

And then upon saving the image, I set the restart function to
"guarantee-process".

My question is: is there another way?  I still have to define "lp" to
call "lp1", and I'd like to avoid having to rename "lp" itself to
"lp1".  Based on an earlier thread, there isn't a way to disable the
"return from multiprocessing" option.


Re: Using both a restart function and multiprocessing when saving an image

Unable to parse email body. Email id is 11016

Re: Using both a restart function and multiprocessing when saving an image

Unable to parse email body. Email id is 11018

Re: Using both a restart function and multiprocessing when saving an image

Hi Martin and Nick,

Calling lw:start-tty-listener is very close to what I was looking for.
 Thus far it seems nearly impossible to return from multiprocessing
when I do this (I'm guessing that I could interrupt "The idle process"
with a function that causes an error, and return that way, but I
haven't tried).

To answer Nick's question: having multiprocessing quit when there
aren't any more processes isn't necessarily a bad thing for us.  Our
problem lies in when we want to reinitialize-multiprocessing.  One
pretty much has to use a solution similar to the example found in
section 15.3 of the LW6 manual, where you need to spawn a thread each
time you want to reenter "my-loop."  Otherwise any time you call
mp:initialize-multiprocessing from within a function that is also
supposed to call "my-loop", "my-loop" is never entered.

Anyway, problem solved for now.

Thanks!
David

On Wed, May 18, 2011 at 7:24 AM, Martin Simmons <martin@lispworks.com> wrote:
>
> I'm not quite sure what you want it to do when you enter :q to quit acl2::lp.
>
> If you want it to run a normal REPL, then you could call lw:start-tty-listener
> when acl2::lp returns.  That will make a new thread running the REPL, which
> will prevent multiprocessing from stopping.
>
> --
> Martin Simmons
> LispWorks Ltd
> http://www.lispworks.com/
>
>
>
>
>>>>>> On Tue, 17 May 2011 16:32:21 -0500, David L Rager said:
>>
>> Greetings Lispworks Community,
>>
>> Is there a way to save an image with :multiprocessing t and also a
>> :restart-function, so that when the restart-function exits,
>> multiprocessing remains enabled?
>>
>> The log included below the signature shows how multiprocessing exits,
>> even when I didn't ask for it to stop (I assume this is because
>> my-loop was called in a separate thread, and since that thread exited,
>> the system thinks that multiprocessing is no longer needed, which
>> isn't the case in our application).  I'm looking for the "right way"
>> to keep multiprocessing enabled, even after "my-loop" exits.  I
>> include a longer version of this question after the log.
>>
>> Thank you,
>> David
>>
>> === My-loop log ===
>>
>> oliphaunt-0% /u/ragerdl/lisps/lispworks-6.0.1/lw-terminal -init - -siteinit -
>> LispWorks(R): The Common Lisp Programming Environment
>> Copyright (C) 1987-2009 LispWorks Ltd.  All rights reserved.
>> Version 6.0.1
>> Saved by ragerdl as lw-terminal, at 11 Apr 2011 14:01
>> User kaufmann on oliphaunt-0
>>
>> CL-USER 1 > (defun my-loop ()
>>              (loop
>>               (princ "prompt> ")
>>               (let ((x (read)))
>>                 (format t "Just read: ~s~%" x)
>>                 (when (eq x :q)
>>                   (return-from my-loop nil)))))
>> MY-LOOP
>>
>> CL-USER 2 > (system::save-image "lw-mp" :multiprocessing t
>> :restart-function 'my-loop)
>> #P"lw-mp"
>>
>> CL-USER 3 > (quit)
>> oliphaunt-0% ./lw-mp -init - -siteinit -
>> LispWorks(R): The Common Lisp Programming Environment
>> Copyright (C) 1987-2009 LispWorks Ltd.  All rights reserved.
>> Version 6.0.1
>> Saved by kaufmann as lw-mp, at 17 May 2011 15:11
>> User kaufmann on oliphaunt-0
>> prompt> 3
>> Just read: 3
>> prompt> 4
>> Just read: 4
>> prompt> :q
>> Just read: :Q
>>
>> ;; No live processes except internal servers - stopping multiprocessing
>>
>> CL-USER 3 > mp::*multiprocessing*
>> NIL
>>
>> CL-USER 4 >
>>
>>
>> === Longer version of the question ===
>>
>> I have an experimental modification to ACL2 that allows the main proof
>> process of ACL2 to evaluate in parallel.  ACL2 contains a
>> read-eval-print-loop.  Thus far our system has been designed for CCL
>> and SBCL, which have compile-time options for enabling
>> multiprocessing.  Lispworks has a similar option for when one saves
>> the image.  However, it's still possible to "return from"
>> multiprocessing once you start that image.  This ability to exit the
>> multi-processing loop without explicitly typing
>> "mp::stop-multiprocessing" is problematic for us.
>>
>> So, I'm looking into a new implementation of our multi-threading for
>> Lispworks, which is based off the multi-processing example in the
>> Lispworks manual
>> [http://www.lispworks.com/documentation/lw60/LW/html/lw-287.htm#pgfId-890248].
>>  We have a function, "lp", which, when called, causes the prompt to
>> enter the ACL2 read-eval-print-loop.  So, I've renamed "lp" to "lp1"
>> and added the following Lispworks code:
>>
>> (defvar *acl2-repl-thread*
>>  '("acl2-read-eval-print-loop" nil lp1))
>>
>> (defun guarantee-process ()
>>  (unless mp::*multiprocessing*
>>    (pushnew *acl2-repl-thread*
>>             mp:*initial-processes*)
>>    (mp:initialize-multiprocessing)))
>>
>> And then upon saving the image, I set the restart function to
>> "guarantee-process".
>>
>> My question is: is there another way?  I still have to define "lp" to
>> call "lp1", and I'd like to avoid having to rename "lp" itself to
>> "lp1".  Based on an earlier thread, there isn't a way to disable the
>> "return from multiprocessing" option.
>>
>
>


Updated at: 2020-12-10 08:37 UTC