thread pool

Greetings, Lispers!

For my web project, I've written a web application server that "talks" with the web server via the scgi protocol. The web application server can process more than one request from the server at the same time. First, I used one thread per request, but the overhead of creating and destroying thread was not worth to me and I decided to use a thread pool.

Here is some code to show the idea of how I did a thread pool to process requests:


(comm:start-up-server :function #'accept-scgi-connection :service port)


(defparameter *sockets-pool* nil)

(defun accept-scgi-connection (socket)
      (when *cleanup-function*
        (funcall *cleanup-function*))
      (sys:atomic-push socket *sockets-pool*))

(defun create-pool-worker ()
  (mp:process-run-function
   "Pool Worker"
   nil
   #'(lambda ()
       (loop 
        (mp:process-wait "Waiting for connection" 
                         #'(lambda () (> (length *sockets-pool*) 0)))
        (let ((socket (sys:atomic-pop *sockets-pool*)))
          (when socket
            (do-scgi-connection socket)))))))

In words: when there is a request from the web server, I put the socket into the socket list. There are some pool workers threads. They monitor the length of the socket list and if its length is not zero, they pop the socket from the socket list and process the request.

Question: are there other ways for the dispatcher to notify workers that there is a work to do? Effective ways to dispatch work to worker thread? LW ways to do a thread pool? Any suggestions, thoughts, code, links to articles are appreciated.


Best,
 Art