"Missing" comm:shutdown

Hi,

I've been struggling for a few hours withn something that should be simple: open a network connection, write a file to the socket, then read back a response. Whatever I did, the read operation would either hang, or return with 0 bytes read. I tried interleaving reads and writes (based on some personal experience in creating deadlocks). I also tried calling force-output and finish-output on the socket stream: no help. I also tried closing the socket, which just meant that the read operation failed.

Finally, I found an implementation in Python of what I as trying to do, and noticed that they were calling shutdown on the write part of the socket. I found an fli definition for shutdown in http://gbbopen.org/svn/GBBopen/trunk/source/tools/portable-sockets.lisp , under the heading "add missing shutdown to lispworks". By calling shutdown, I got my code to work the way it should.

As far as I can see, there is no way of doing this except by going through the fli.

Is there any chance that shutdown can be added to the COMM package?

Re: "Missing" comm:shutdown

Unable to parse email body. Email id is 12552

Re: "Missing" comm:shutdown

Sure; see http://pastebin.com/bcRsqzz1

On Fri, Nov 1, 2013 at 11:31 AM, Nick Levine <ndl@ravenbrook.com> wrote:

> Date: Fri, 1 Nov 2013 10:32:03 +0100
> From: Raymond Wiker <rwiker@gmail.com>

>
> Hi,
>
> I've been struggling for a few hours withn something that should be
> simple: open a network connection, write a file to the socket, then
> read back a response. Whatever I did, the read operation would

> either hang, or return with 0 bytes read. [...]

Any chance we can see your code?

- nick

Re: "Missing" comm:shutdown

Unable to parse email body. Email id is 12554

Re: "Missing" comm:shutdown

The shutdown is necessary because the Java side reads until it gets an EOF, which is a valid strategy. There are alternatives, like an HTTP-based service, but the "raw" tika service should be more efficient.

The reason I would like to have this be efficient is because it will be used as part of a "document processing pipeline" used for feeding a search engine (Apache SOLR, but it will be possible to support other search engines).

On Fri, Nov 1, 2013 at 12:02 PM, Nick Levine <ndl@ravenbrook.com> wrote:

> Date: Fri, 1 Nov 2013 11:51:48 +0100
> From: Raymond Wiker <rwiker@gmail.com>
> Cc: Lisp Hug Lispworks <lisp-hug@lispworks.com>
>
> Sure; see http://pastebin.com/bcRsqzz1

Nothing wrong with the lisp, as far as I can tell. Call to shutdown is
unexpected. My guess is there's some quirk in the Java side; I suggest
you write a dummy server in lisp to demonstrate that.

By the way, paste.lisp.org (a) is written in lisp which makes it
cooler and (b) doesn't throw those nasty adverts at you...

- nick

Re: "Missing" comm:shutdown

Unable to parse email body. Email id is 12556

Re: "Missing" comm:shutdown

That won't help in solving the actual problem I have here, but would certainly be required for production use. I've added that to the to-be-production version of the code; thanks!

On Fri, Nov 1, 2013 at 12:18 PM, Camille Troillard <camille@osculator.net> wrote:

Hi Raymond,

Just a guess...
Wouldn’t need to call close on the socket-stream returned by open-tcp-stream?

Best,
Cam

On 01 Nov 2013, at 12:09, Raymond Wiker <rwiker@gmail.com> wrote:

> The shutdown is necessary because the Java side reads until it gets an EOF, which is a valid strategy. There are alternatives, like an HTTP-based service, but the "raw" tika service should be more efficient.
>
> The reason I would like to have this be efficient is because it will be used as part of a "document processing pipeline" used for feeding a search engine (Apache SOLR, but it will be possible to support other search engines).
>
>
> On Fri, Nov 1, 2013 at 12:02 PM, Nick Levine <ndl@ravenbrook.com> wrote:
> > Date: Fri, 1 Nov 2013 11:51:48 +0100
> > From: Raymond Wiker <rwiker@gmail.com>
> > Cc: Lisp Hug Lispworks <lisp-hug@lispworks.com>
> >
> > Sure; see http://pastebin.com/bcRsqzz1
>
> Nothing wrong with the lisp, as far as I can tell. Call to shutdown is
> unexpected. My guess is there's some quirk in the Java side; I suggest
> you write a dummy server in lisp to demonstrate that.
>
> By the way, paste.lisp.org (a) is written in lisp which makes it
> cooler and (b) doesn't throw those nasty adverts at you...
>
> - nick
>
>

Re: "Missing" comm:shutdown

Camille Troillard <camille@osculator.net> writes:

> Hi Raymond,
>
> Just a guess...
> Wouldn’t need to call close on the socket-stream returned by open-tcp-stream?

http://cr.yp.to/tcpip/twofd.html has some good information about the
issue.

Zach

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: "Missing" comm:shutdown

If I recall close() is the same as shutdown(socket,2)

On Fri, Nov 1, 2013 at 3:46 PM, Zach Beane <xach@xach.com> wrote:
>
> Camille Troillard <camille@osculator.net> writes:
>
>> Hi Raymond,
>>
>> Just a guess...
>> Wouldn’t need to call close on the socket-stream returned by open-tcp-stream?
>
> http://cr.yp.to/tcpip/twofd.html has some good information about the
> issue.
>
> Zach
>
> _______________________________________________
> Lisp Hug - the mailing list for LispWorks users
> lisp-hug@lispworks.com
> http://www.lispworks.com/support/lisp-hug.html
>

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: "Missing" comm:shutdown

If so, then I want something that is equivalent to shutdown(socket, 1) - only close the write side of the socket.

On Fri, Nov 1, 2013 at 1:01 PM, Ala'a Mohammad <amalawi@gmail.com> wrote:

If I recall close() is the same as shutdown(socket,2)

On Fri, Nov 1, 2013 at 3:46 PM, Zach Beane <xach@xach.com> wrote:
>
> Camille Troillard <camille@osculator.net> writes:
>
>> Hi Raymond,
>>
>> Just a guess...
>> Wouldn’t need to call close on the socket-stream returned by open-tcp-stream?
>
> http://cr.yp.to/tcpip/twofd.html has some good information about the
> issue.
>
> Zach
>

> _______________________________________________
> Lisp Hug - the mailing list for LispWorks users
> lisp-hug@lispworks.com
> http://www.lispworks.com/support/lisp-hug.html
>

Re: "Missing" comm:shutdown

Unable to parse email body. Email id is 12563