Re: Supported Linux Versions

> Is a 2.6 kernel on Red Hat Enterprise Linux 3 or 4 also supported ?
> Are dual processor machines with hyperthreading supported ?

I don't know what's supported, but I do run on 2.6 dual processor
Debian (no hyperthreading). 

Regards,
Chris Dean

Re: Supported Linux Versions

On Mon, 19 Dec 2005 11:02:55 +0100, Sven Van Caekenberghe <sven@beta9.be> wrote:

> The official requirements for LispWorks on Linux state that a 2.2 or
> 2.4 kernel is required. RedHat up to version 9 is mentioned as being
> supported. What about newer kernels or distributions ? Is a 2.6
> kernel on Red Hat Enterprise Linux 3 or 4 also supported ?

I can't speak for LispWorks in terms of "supported" but I've used LWL
on various Debian installations with 2.6.x kernels without problems.

> Are dual processor machines with hyperthreading supported ?

I would assume that in this case only one processor is used by LW.

Does anyone on this list have experience with multi-processor machines
and LWW/LWM, BTW?  Will a multi-threaded LW application deploy more
than one processor on these machines?

> BTW, I think I read somewhere in the docs that LispWorks on Mac OS X
> and Windows is using native multi-threading, but not on Linux (where
> it is using soft multi-threading). Is that correct ?

Yes, but LispWorks announced they're going to change this for 5.0:

  <ftp://ftp.lispworks.com/pub/software_tools/downloads/Amsterdam2005/LW.pdf>

> And how does this relate to (web) server performance as compared to
> the other platforms ?

Web server performance with LWL has been "good enough" for me.  I
think you must have /very/ high loads before you feel a difference
between different threading models.  If you're using foreign function
calls that have the potential to block, that's another story of
course.

Cheers,
Edi.

Re: Supported Linux Versions

Sven Van Caekenberghe <sven@beta9.be> writes:

> BTW, I think I read somewhere in the docs that LispWorks on Mac OS X  
> and Windows is using native multi-threading, but not on Linux (where  
> it is using soft multi-threading). Is that correct ? And how does  
> this relate to (web) server performance as compared to the other  
> platforms ?

My http-based server is not exactly a web server, but my experience is
that the non-native threads of LispWorks on linux work excellently
with even more than 100 threads. There are some problems when one of
the threads is doing really intensive I/O, and global GC (mark-and-sweep)
tends to be very heavy when there are many threads working on the same
data, but in general, I'm happy with it. I just tested now, and got very
good response times (< 15ms for most requests) when accessing my LWL
server running on an Opteron server, with currently 127 threads.

LW 5.0 will get native threads on linux, in addition to a lot of other
improvements on the x68 platform. I'm really looking forward to testing
it!
-- 
  (espen)

Re: Supported Linux Versions

On Mon, 19 Dec 2005 17:57:58 +0100, "Marc Battyani" <marc.battyani@fractalconcept.com> wrote:

> But even with native threading LW only has one Lisp thread active at
> any time.

Hmm, so a "pure" Lisp application (no Apache etc.) with several
threads will only ever use one processor even if you have two or four?
Not good... :(

Re: Supported Linux Versions

At 12/19/2005 12:17 PM, David Tolpin wrote:
>>Hmm, so a "pure" Lisp application (no Apache etc.) with several
>>threads will only ever use one processor even if you have two or four?
>>Not good... :(
>
>
>Is there any lisp that does better?

OpenMCL (www.clozure.com) supports real multi-processing.

Gail Zacharias
Clozure Associates

Re: Supported Linux Versions

On Mon, 19 Dec 2005 21:17:04 +0400, David Tolpin <dvd@davidashen.net> wrote:

> Is there any lisp that does better?

Scieneer Common Lisp:

  <http://scieneer.com/scl/threads.html>

I also remember from playing around with SBCL that each "native"
thread appears as a Linux process in the ps(1) output.  I guess one
can deduce from this that SBCL/Linux will employ several processors at
the same time.

Don't know about Allegro/Windows or OpenMCL.

Re: Supported Linux Versions

On Mon, 19 Dec 2005 19:22:29 +0100, "Marc Battyani" <marc.battyani@fractalconcept.com> wrote:

> From: "Edi Weitz" <edi@agharta.de>
>
>> Scieneer Common Lisp:
>>
>>   <http://scieneer.com/scl/threads.html>
>
> From this description it's not obvious that they do better.  In
> particular it's not stated: "multiple lisp threads can be active
> simultaneously on SMP systems" (Though like everybody thay say that
> foreign code can be active at the same time)

I'm pretty sure that they do better, I think that's their main selling
point.  Also note this sentence from their website:

  "Applications written in the Scieneer Common Lisp implementation can
   grow as the demands of the application grow by simply moving the
   application to a multi-processor system."

That wouldn't make much sense if only one processor can be active at
the same time.

Also, Christophe Rhodes reminded me of this c.l.l message

  <http://groups.google.com/group/comp.lang.lisp/msg/30626f0005b0315c>

which seems to confirm that Scieneer is SMP-capable (and AllegroCL is
not).

I'll ask Douglas Crosher for confirmation.

So far, I /think/ we have to following situation.

  Native threads, can employ more than one processor at the same time:
  --------------------------------------------------------------------

  - Scieneer Common Lisp
  - OpenMCL
  - Armed Bear Common Lisp
  - SBCL (on Linux/x86)
  - Corman Lisp (???)

  Native threads, but only one active Lisp thread:
  ------------------------------------------------

  - AllegroCL (Windows)
  - LispWorks (Windows, OS X)

As a LispWorks customer I certainly wish that LispWorks moves up to
the first group in version 5.0.  I think multi-processor or at least
multi-core systems will be ubiquitous very soon.

Cheers,
Edi.

Re: Supported Linux Versions

> Von: Edi Weitz <edi@agharta.de>
> Antworten an: Edi Weitz <edi@agharta.de>
> Datum: Mon, 19 Dec 2005 21:37:33 +0100
> An: Marc Battyani <marc.battyani@fractalconcept.com>
> Cc: David Tolpin <dvd@davidashen.net>, <lisp-hug@lispworks.com>
> Betreff: Re: Supported Linux Versions
> 
.....
> I'll ask Douglas Crosher for confirmation.
> 
> So far, I /think/ we have to following situation.
> 
>   Native threads, can employ more than one processor at the same time:
>   --------------------------------------------------------------------
> 
>   - Scieneer Common Lisp
>   - OpenMCL
>   - Armed Bear Common Lisp
>   - SBCL (on Linux/x86)
>   - Corman Lisp (???)

Question: are the Common Lisp implementations thread safe? Where and where
not? Defining functions? Classes? Redefining classes? Methods?
Redefining method combinations? Interning symbols in packages?
Update instance for a changed class? The compiler? Did any
implementor actually tried to deliver answers or are these things
left to the user/programmer?

>   Native threads, but only one active Lisp thread:
>   ------------------------------------------------
> 
>   - AllegroCL (Windows)
>   - LispWorks (Windows, OS X)
> 
> As a LispWorks customer I certainly wish that LispWorks moves up to
> the first group in version 5.0.  I think multi-processor or at least
> multi-core systems will be ubiquitous very soon.
> 
> Cheers,
> Edi.
>

Re: Supported Linux Versions

Rainer Joswig <joswig@lisp.de> writes:

> Question: are the Common Lisp implementations thread safe? 

Only one data point, but I thought I'd mention it anyway: I've asked
both Franz and LispWorks on different occasions whether gethash and
(setf gethash) are thread safe. And they are.

However, be aware that (at least LW's) maphash will lock the hash
table for the duration of the maphash execution. I discovered this the
hard way - it took me some debugging before I understood that (maphash
#'pretty-slow-function *important-table*) was making my server grind
to a halt ;)
-- 
  (espen)

Re: Supported Linux Versions

On Tue, 20 Dec 2005 07:39:45 +0100, Rainer Joswig <joswig@lisp.de> wrote:

> Question: are the Common Lisp implementations thread safe? Where and
> where not? Defining functions? Classes? Redefining classes? Methods?
> Redefining method combinations? Interning symbols in packages?
> Update instance for a changed class? The compiler? Did any
> implementor actually tried to deliver answers or are these things
> left to the user/programmer?

Yes, good questions.  This is precisely what has bothered me for some
time - basically nobody will tell you what exactly is thread safe in
their Lisp and what's not.  If one of the implementations out there
has a comprehensive documentation that answers these questions then
I've missed it - my apologies.  My impression is that all this is
mostly undocumented and if you're lucky you stumble upon some details
by chance, like here:

  <http://thread.gmane.org/gmane.lisp.lispworks.general/3999>

IIRC the Java documentation shows that one can document these issues
in a pretty precise way.

Cheers,
Edi.

Re: Supported Linux Versions

Espen Vestre <ev@netfonds.no> writes:

> However, be aware that (at least LW's) maphash will lock the hash
> table for the duration of the maphash execution. I discovered this the
> hard way - it took me some debugging before I understood that (maphash
> #'pretty-slow-function *important-table*) was making my server grind
> to a halt ;)

I was unclear here: The lock is only a write-lock. You can still read
from the hash table during the maphash.
-- 
  (espen)

Re: maphash

Clint Hyde <chyde@cox.net> writes:

> would you *want* the hashtable modified while you're map'ing on it? that 
> just sounds dangerous to me.

Well, in my case, I needed to go through the hash table and output (to
a socket) every y that matched certain criteria. My solution was to collect
those y first, and then output them (since the collection itself was fast,
it was only the i/o during the maphash that would stall it).
-- 
  (espen)

Re: Supported Linux Versions

On Mon, 19 Dec 2005 21:37:33 +0100, Edi Weitz <edi@agharta.de> wrote:

> I'll ask Douglas Crosher for confirmation.

He just answered and confirmed that several different threads on
different processors can execute Lisp code at the same time.  The only
exception (similar to SBCL) is the garbage collector which stops the
other threads.

Re: Supported Linux Versions

On Mon, 19 Dec 2005 21:37:33 +0100, Edi Weitz <edi@agharta.de> wrote:

> So far, I /think/ we have the following situation.
>
>   Native threads, can employ more than one processor at the same time:
>   --------------------------------------------------------------------
>
>   - Scieneer Common Lisp
>   - OpenMCL
>   - Armed Bear Common Lisp
>   - SBCL (on Linux/x86)
>   - Corman Lisp (???)
>
>   Native threads, but only one active Lisp thread:
>   ------------------------------------------------
>
>   - AllegroCL (Windows)
>   - LispWorks (Windows, OS X)
>
> As a LispWorks customer I certainly wish that LispWorks moves up to
> the first group in version 5.0.  I think multi-processor or at least
> multi-core systems will be ubiquitous very soon.

I have to revive this topic for a moment.

Yesterday I was bitten by the fact that threads created by "foreign"
code can't call back into Lisp.  I hope this is also something that
would be history if LispWorks were fully SMP-capable.  FWIW, it
doesn't work on AllegroCL either (although their documentation seems
to suggest it should) while it works fine with Corman Lisp.

See here (and the following messages in the same thread) for an
example:

  <http://common-lisp.net/pipermail/rdnzl-devel/2005-December/000041.html>

How do other Lisps with "native" threads (SBCL, OpenMCL, etc.) deal
with this?

Cheers,
Edi.

PS: Yes, I know that one can deliver to a DLL as a workaround.
    However, you'll lose interactive development in this case.

Re: Supported Linux Versions

Unable to parse email body. Email id is 5093

Re: Supported Linux Versions

On Fri, 23 Dec 2005 21:23:16 GMT, Martin Simmons <martin@lispworks.com> wrote:

> We've fixed it for LW 5 to allow calls rom "foreign" threads in .exe
> too

Great, thanks.

> (it's not really related to SMP).

Well, one step at a time then... :)

Re: Supported Linux Versions

Unable to parse email body. Email id is 5116

Re: Supported Linux Versions

On Tue, 3 Jan 2006 11:41:23 GMT, Nick Levine <ndl@ravenbrook.com> wrote:

> I don't know whether I'm missing the point here, but: you can
> save-image to a dll and still retain the IDE.

Oh, I didn't know that.  So, you use SAVE-IMAGE with :DLL-EXPORTS and
:RESTART-FUNCTION set to what?  And you start the resulting DLL with
rundll32 and what starts up is the usual LispWorks IDE?

Thanks,
Edi.

Re: Supported Linux Versions

Unable to parse email body. Email id is 5119

Re: Supported Linux Versions

Unable to parse email body. Email id is 5120

Re: Supported Linux Versions

On Tue, 3 Jan 2006 13:14:53 GMT, Nick Levine <ndl@ravenbrook.com> wrote:

>    Starting via rundll32 is a little tricky
>
> ... not to mention, pointless. 

???

Re: Supported Linux Versions

Unable to parse email body. Email id is 5123

Re: Supported Linux Versions

On Tue, 3 Jan 2006 13:30:00 GMT, Nick Levine <ndl@ravenbrook.com> wrote:

> Surely you need a more sophisticated harness than rundll32 to start
> lisp, something that won't call exit() as soon as the first call
> into lisp returns.

OK, I've fiddled around a bit and I now managed to build a tiny C
"driver" that starts the (DLL) IDE and enables me to play with foreign
threads calling back into Lisp.  For those of you who are C palookas
like me I've written down how I did it:

  <http://weitz.de/lw-callbacks/>

Nick, thanks a lot for your help!

Cheers,
Edi.