Re: Piping to/from external program

Re: Piping to/from external program

Hi Erik,

For binary data I would use open-pipe.
run-shell-command is basically running command in a shell.


Best,
Cam


> On 20 Sep 2017, at 09:41, Erik Ronström <erik.ronstrom@doremir.com> wrote:
> 
> Hi,
> 
> I have binary data in a vector, and I want to use an external program to process it and read the result into another vector. What is the best practices for doing that kind of operation?
> 
> I need guarantees that all the data has been consumed and the process has finished before continuing. I’ve been trying using sys:run-shell-command, but I don’t understand how to make the call synchronous while using streams.
> 
> There seems to be a bunch of related functions (run-shell-command, open-pipe, call-system, call-system-showing-output …), and I’m not sure which is the most appropriate in this case. Any tips greatly appreciated!
> 
> Erik
> 
> 
> 
> _______________________________________________
> Lisp Hug - the mailing list for LispWorks users
> lisp-hug@lispworks.com
> http://www.lispworks.com/support/lisp-hug.html


_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: Piping to/from external program

On 20 Sep 2017, at 2:41, Erik Ronström wrote:

> Hi,
>
> I have binary data in a vector, and I want to use an external program 
> to process it and read the result into another vector. What is the 
> best practices for doing that kind of operation?
>
> I need guarantees that all the data has been consumed and the process 
> has finished before continuing. I’ve been trying using 
> sys:run-shell-command, but I don’t understand how to make the call 
> synchronous while using streams.
>
> There seems to be a bunch of related functions (run-shell-command, 
> open-pipe, call-system, call-system-showing-output …), and I’m not 
> sure which is the most appropriate in this case. Any tips greatly 
> appreciated!
>
> Erik
>

I think you could try UIOP:RUN-PROGRAM, and read carefully the 
documentation about Faré's OUTPUT-SLURPER.  You could certainly make a 
slurper that would fill a vector for you.

I am afraid it's not that well documented; you may have to look at the 
source to really get it, although the documentation string for 
UIOP:RUN-PROGRAM would be a good first step.

Best,
r

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: Piping to/from external program

Unable to parse email body. Email id is 14530

Re: Piping to/from external program

Unable to parse email body. Email id is 14533

Re: Piping to/from external program

> If anyone’s interested, I’m sending raw audio data to an external ogg converter, reading from stdin and writing back on stdout. From a shell, I could easily feed it with a file or pipe something into it, and then on to further processing, so my initial thought was that there has to be a simple way to do it from Lisp. But it looks like it wasn’t so simple after all! Although I guess it’s not a Lisp problem per se.

Since it is raw audio data, there would seem to be massive amounts of data - 10 MB/min for 44.1 kHz stereo. So using a thin straw to pipe the data between programs seems inefficient. 

A better way would be binary bulk storage. And since all modern OS use VM with massive in-core buffering, there wouldn’t seem to be much of a speed hit, as long as you can read / write large binary blocks, rather than a serial stream format - i.e., READ/WRITE vs FREAD/FWRITE. Nothing has to wait for physical I/O unless your memory is severely cramped.

 - DM

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html