Summary of multi-processing...
Hi, Somebody asked me about multiprocessing in LispWorks, and I quickly wrote down a summary. The summary, as far as I can tell, is below. Anything I’m missing? - mp:process-run-function calls a function in a new process. You can pass some parameters to such a function, but that’s usually not necessary. This essentially moves a function to a different core (not exactly correct, but a good enough intuition). - 95% or more of all synchronization needs are covered my mailboxes (mp:make-mailbox, mp:mailbox-send, mp:mailbox-peek, mp:mailbox-read). You can send any lisp object from one process to other processes through such mailboxes in a safe manner. mp:process-run-function allows you to create process-specific mailboxes, but I always find that confusing. I usually set up global mailboxes that the different processes can refer to (shared in closures, or in global variables). - It’s typically not a good idea to call mp:process-run-function for short-lived functions. Better call a function that runs a loop, fetches some object from a mailbox, does something with it, and then fetches some more, until it fetches a special object / symbol that signals that the process should shut down. Shutting down a process is achieved by just returning from the process function as normal. If a process needs to know whether some other process has successfully shut down, mp:process-join is your friend. - If you need more fine-grained synchronization, you can use locks (mp:make-lock, mp:with-lock). Locks are nothing to be afraid of: Unless there is high contention, they don’t cost that much anymore in modern processors in terms of performance. - Some data structures provide locked / atomic operations by default, like hash tables and vectors with fill-pointers. In can be interesting to tell them to not use locks if you know that they are used only by one thread at a time (option :single-thread in make-array and make-hash-table). Some CLOS operations need to take locks, and it’s not possible to prevent that from happening, but that’s only for rare cases anyway (for example, for redefining classes). - LispWorks provides even more synchronization primitives, but you rarely need them. (But it’s good that they are there!) Specifically: Barriers (in package mp), semaphores (MP), condition variables (mp), atomic operations (package system), memory order synchronization (system). - Good to know: memory allocation is by default thread-local. It’s always better to allocate memory in the threads that use that memory most. - The garbage collector freezes all processes. This may or may not matter. Pascal -- Pascal Costanza The views expressed in this email are my own, and not those of my employer. _______________________________________________ Lisp Hug - the mailing list for LispWorks users lisp-hug@lispworks.com http://www.lispworks.com/support/lisp-hug.html