comm:open-tcp-socket returning nil

On OS X I'm getting a situation where open-tcp-socket is returning nil 
when (I assume) I try and open too many sockets at once in parallel 
processes. Is there a limit that's built into LW? I can mitigate this by 
using a semaphore and limiting the number of open sockets to - say - 
20-ish, but this is terrible and not even close to the ulimit set by the 
file system for open handles.

So, I'm curious as to whether or not there is a hard-limit of # sockets 
that can be opened at once in LW? If so, what is that limit (and can I 
increase it)? If not, what could be causing open-tcp-socket to return 
nil? Note that I've ruled out a bad server/host; if I set the semaphore 
count to something super small (like 10) then all connections eventually 
work once they can grab hold of it.

Jeff M.

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: comm:open-tcp-socket returning nil

Hi Jeff,

On 22 nov. 2013, at 14:38, Jeffrey Massung <massung@gmail.com> wrote:

> On OS X I'm getting a situation where open-tcp-socket is returning nil when (I assume) I try and open too many sockets at once in parallel processes. Is there a limit that's built into LW? I can mitigate this by using a semaphore and limiting the number of open sockets to - say - 20-ish, but this is terrible and not even close to the ulimit set by the file system for open handles.
> 
> So, I'm curious as to whether or not there is a hard-limit of # sockets that can be opened at once in LW? If so, what is that limit (and can I increase it)? If not, what could be causing open-tcp-socket to return nil? Note that I've ruled out a bad server/host; if I set the semaphore count to something super small (like 10) then all connections eventually work once they can grab hold of it.

As you said, this seems well under OS X’s limits.
I’d be surprised if LispWorks had such a low limit too!

Have you had a look at the previous sockets state? (lsof -i)
There is an interesting article about this here:
http://superuser.com/questions/145989/does-mac-os-x-throttle-the-rate-of-socket-creation


Best,
Cam

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: comm:open-tcp-socket returning nil

On 22 Nov 2013, at 13:38, Jeffrey Massung wrote:

> On OS X I'm getting a situation where open-tcp-socket is returning nil when (I assume) I try and open too many sockets at once in parallel processes. Is there a limit that's built into LW? I can mitigate this by using a semaphore and limiting the number of open sockets to - say - 20-ish, but this is terrible and not even close to the ulimit set by the file system for open handles.

I presume you mean open-tcp-stream?

If so, are you opening a lot of connections to the same port on the same machine rather quickly?  You may be overwhelming its listen backlog in that case which will cause connections to be dropped.  The maximum backlog you can have is 128 on OSX I think.  if the server is written in LW then I can't see how to control the backlog at all, so presumably you get whatever is hardwired (or I am missing something in the documentation, which is quite likely).

--tim


_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: comm:open-tcp-socket returning nil


* Tim Bradshaw <8181E842-6456-41A5-84FB-AD13C71A57FA@cley.com> :
Wrote on Sat, 23 Nov 2013 17:49:21 +0000:

| The maximum backlog you can have is 128 on OSX I think.  if the server
| is written in LW then I can't see how to control the backlog at all,
| so presumably you get whatever is hardwired (or I am missing something
| in the documentation, which is quite likely).

I think this is a concern in both the COMM and MP[1] packages.  A lot of
important often crucial functionality is available only in unexported
symbols for which there is no open documentation.  However one would
have assumed that they were just not available; unless one gives credit
to LW and then reasons from there that these are basic issues and must
have been solved at some point.  Perhaps there is an alternative way of
discovering these functions.

[Of course these are necessarily platform/OS dependenet issues, and no
 doubt the devs havd good reasons for `deprecating' them and disclaiming
 support.  ---Madhu

[1] maybe to a lesser extent now as the OS/hardware landscape
converges..

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: comm:open-tcp-socket returning nil

Unable to parse email body. Email id is 12645