Lisp HUG Maillist Archive

sys:open-pipe oddities...

Hello all,
I'm playing with pipe streams again, and I'm having a bit
of an issue with command line arguments. I'm on
Mac OS X 10.4.6, on Lispworks 4.4.6 Professional.

Quoth the docs:

"Signature
open-pipe command &key direction element-type interrupt-off shell-type => stream
....
If command  is a string then it is passed to the shell as the command
to run. If command is a list, then the first element is the command to
run directly and the other elements are passed as arguments on the
command line. If command  is nil , then the shell is run."

So, something like

(setq qq (sys:open-pipe "/bin/ls -l"))
(loop while (print (read-line qq nil nil)))

works, as I'd expect it to, the shell interprets the -l correctly.
This example works if I just give "ls -l" as well, which also
seems good; the shell is making sure PATH is consulted.

Evaluating
(setq qq (sys:open-pipe (list "/bin/ls")))
in the listener causes a terminal window to come up,
with a shell prompt at which

/tmp/LWtemp.Wright.5021.1.command; exit

has been entered. The call in the listener returns
a pipe stream as expected.
However,

(loop while (print (read-line qq nil nil)))

gives me

"ng fasl file /Users/tayloj/.lisp-src/cl-ppcre-1.2.11/errors.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/cl-ppcre-1.2.11/lexer.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/cl-ppcre-1.2.11/parser.nfasl"
";  Loading fasl file
/Users/tayloj/.lisp-src/cl-ppcre-1.2.11/regex-class.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/cl-ppcre-1.2.11/convert.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/cl-ppcre-1.2.11/optimize.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/cl-ppcre-1.2.11/closures.nfasl"
";  Loading fasl file
/Users/tayloj/.lisp-src/cl-ppcre-1.2.11/repetition-closures.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/cl-ppcre-1.2.11/scanner.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/cl-ppcre-1.2.11/api.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-doc/packages.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-doc/specials.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-doc/util.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-doc/parse.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-doc/pages.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-add-ons-0.2.0/packages.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-add-ons-0.2.0/specials.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-add-ons-0.2.0/misc.nfasl"
";  Loading fasl file
/Users/tayloj/.lisp-src/lw-add-ons-0.2.0/documentation.nfasl"
";  Loading fasl file
/Users/tayloj/.lisp-src/lw-add-ons-0.2.0/completions.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-add-ons-0.2.0/systems.nfasl"
";; Creating system Common-Lisp-User::Cl-Ppcre"
";; Creating system Common-Lisp-User::Cl-Who"
";; Creating system Common-Lisp-User::Lw-Doc"
";; Creating system Common-Lisp-User::Lw-Add-Ons"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-add-ons-0.2.0/editor.nfasl"
";  Loading fasl file /Users/tayloj/.lisp-src/lw-add-ons-0.2.0/commands.nfasl"
""
"Error: Signal : Bus error(10)"
"  1 (abort) Return to level 0."
"  2 Return to top loop level 0."
""
"Type :b for backtrace, :c <option number> to proceed,  or :? for
other options"
""

and the call doesn't actually return. That's the (normally unseen) output of
loading some things from my .lispworks file, and clearly not the output
of ls.

To make the whole issue even more complicated, if I run the darwin-motif
lispworks binary from the console, I don't have this issue at all. Both forms
of comand "ls" "/bin/ls" and ("/bin/ls") seem to work without a problem.
The output of that follows. Any ideas?

CL-USER 1 > (setq qq (sys:open-pipe "/bin/ls"))
#<SYSTEM::PIPE-STREAM "/bin/ls">

CL-USER 2 > (loop while (print (read-line qq nil nil)))

"Applications"
"Applications (Mac OS 9)"
"Classic System Folder"
"Desktop"
"Desktop DB"
"Desktop DF"
"Developer"
"Library"
"Network"
"System"
"TheVolumeSettingsFolder"
"User Guides And Information"
"Users"
"Volumes"
"automount"
"bin"
"cores"
"dev"
"etc"
"lost+found"
"mach"
"mach.sym"
"mach_kernel"
"opt"
"private"
"sbin"
"sw"
"tmp"
"usr"
"var"
NIL
NIL

CL-USER 3 > (close qq)
T

CL-USER 4 > (setq qq (sys:open-pipe (list "/bin/ls")))
#<SYSTEM::PIPE-STREAM "/bin/ls">

CL-USER 5 > (loop while (print (read-line qq nil nil)))

"Applications"
"Applications (Mac OS 9)"
"Classic System Folder"
"Desktop"
"Desktop DB"
"Desktop DF"
"Developer"
"Library"
"Network"
"System"
"TheVolumeSettingsFolder"
"User Guides And Information"
"Users"
"Volumes"
"automount"
"bin"
"cores"
"dev"
"etc"
"lost+found"
"mach"
"mach.sym"
"mach_kernel"
"opt"
"private"
"sbin"
"sw"
"tmp"
"usr"
"var"
NIL
NIL

CL-USER 6 >
-- 
=====================
Joshua Taylor
tayloj@rpi.edu


Re: sys:open-pipe oddities...


"Taylor, Joshua" <tayloj@rpi.edu> writes: 
> I'm on Mac OS X 10.4.6, on Lispworks 4.4.6 Professional.
...
> (setq qq (sys:open-pipe (list "/bin/ls")))
> (loop while (print (read-line qq nil nil)))

FWIW, confirmed.  I'm on the same versions of OS X and LispWorks and
it works in my normal tty image, but does bad things in the GUI
Listener.

Cheers,
Chris Dean


Re: sys:open-pipe oddities...

Unable to parse email body. Email id is 5639

Re: sys:open-pipe oddities...

Ah, great, tihs is easy to work around then.

By the way, is there somewhere on the Lispworks
website where one can search for outstanding bugs?
I found the area to report them, and the place to download
patches, but I'm always worried when I bring up something
that seems to be a bug that i've not searched some existing
store of known bugs. Is there such a searchable store?
(Aside from the list archives?)

On 6/9/06, Martin Simmons <martin@lispworks.com> wrote:
>
> >>>>> On Fri, 9 Jun 2006 11:23:16 -0400, "Taylor, Joshua" <tayloj@rpi.edu> said:
> >
> > Hello all,
> > I'm playing with pipe streams again, and I'm having a bit
> > of an issue with command line arguments. I'm on
> > Mac OS X 10.4.6, on Lispworks 4.4.6 Professional.
> >
> > Quoth the docs:
> >
> > "Signature
> > open-pipe command &key direction element-type interrupt-off shell-type => stream
> > ...
> > If command  is a string then it is passed to the shell as the command
> > to run. If command is a list, then the first element is the command to
> > run directly and the other elements are passed as arguments on the
> > command line. If command  is nil , then the shell is run."
> >
> > So, something like
> >
> > (setq qq (sys:open-pipe "/bin/ls -l"))
> > (loop while (print (read-line qq nil nil)))
> >
> > works, as I'd expect it to, the shell interprets the -l correctly.
> > This example works if I just give "ls -l" as well, which also
> > seems good; the shell is making sure PATH is consulted.
> >
> > Evaluating
> > (setq qq (sys:open-pipe (list "/bin/ls")))
> > in the listener causes a terminal window to come up,
>
> This is caused by a bug in sys:open-pipe when the arguments are not
> base-strings.  It works in some situations because the Lisp reader creates
> strings with the same element type as the stream being read.
>
> To work around this, you can do
>
> (setq qq (sys:open-pipe (list (coerce "/bin/ls" 'base-string))))
>
> --
> Martin Simmons
> LispWorks Ltd
> http://www.lispworks.com/
>
>


-- 
=====================
Joshua Taylor
tayloj@rpi.edu


Re: sys:open-pipe oddities...

Unable to parse email body. Email id is 5641

Re: sys:open-pipe oddities...

Well, the base-string answer got me to a point where I
can get the external excutable running. Now I'm facing
a slightly more challenging problem, which might be more
of an OS I/O issue, but I'm not sure.

The program to which I'm opening a pipe accepts a
filename as an argument, and it reads its input from this
file. As I'm opening a pipe, I'm going to be writing that
input directly. (I may run into the inability to close just
one side of a pipe, an issue I've had in the past, but
I'll attack that problem when I come to it.) When I'm
working on a command prompt, it's no problem to
give /dev/stdin as the file argument, or even /dev/fd/0.
However, if I built my command line, or my list of
command strings, with such an argument, I get a
FILE NOT FOUND error.

Here's an example with cat:
-----------------------------------
;; here's an example with 'cat -' where an '-' tells cat to
;; read from standard input
AS 84 > (setf pipe (sys:open-pipe (list (coerce "/bin/cat"
'base-string) (coerce "-" 'base-string)) :direction :io))
#<System::Pty-Stream "/bin/cat">

AS 85 > (write-line "line 1" pipe)
"line 1"

AS 86 > (write-line "line 2" pipe)
"line 2"

AS 87 > (force-output pipe)
Nil

AS 88 > (read-line pipe)
"line 1"
Nil

AS 89 > (read-line pipe)
"line 2"
Nil

;; here's 'cat /dev/stdin' which fails with
;; cat: /dev/stdin: Permission denied
AS 92 > (setf pipe (sys:open-pipe (list (coerce "/bin/cat"
'base-string) (coerce "/dev/stdin" 'base-string)) :direction :io))
#<System::Pty-Stream "/bin/cat">

AS 93 > (write-line "line 1" pipe)
"line 1"

AS 94 > (write-line "line 2" pipe)
"line 2"

AS 95 > (force-output pipe)

Error: Error while writing on stream #<System::Pty-Stream "/bin/cat">:
 Broken pipe [Unix Errno = 32].
  1 (continue) Try writing to #<Editor::Rubber-Stream #<Editor:Buffer
CAPI interactive-pane 2> 11AE4B7F> again.
  2 (abort) Return to level 0.
  3 Return to top loop level 0.

Type :b for backtrace, :c <option number> to proceed,  or :? for other options

AS 96 : 1 > :a

AS 97 > (read-line pipe)
"cat: /dev/stdin: Permission denied"
Nil

AS 98 > (read-line pipe)

Error: End of file while reading stream #<System::Pty-Stream "/bin/cat">.
  1 (abort) Return to level 0.
  2 Return to top loop level 0.

Type :b for backtrace, :c <option number> to proceed,  or :? for other options

AS 99 : 1 >
-------------------------------------

I suspect that this might not be a LispWorks issue so
much as a 'I'm naive about pipes' issue, so do let me know
if that's the case.

Thanks!


On 6/9/06, Taylor, Joshua <tayloj@rpi.edu> wrote:
> Ah, great, tihs is easy to work around then.
>
> By the way, is there somewhere on the Lispworks
> website where one can search for outstanding bugs?
> I found the area to report them, and the place to download
> patches, but I'm always worried when I bring up something
> that seems to be a bug that i've not searched some existing
> store of known bugs. Is there such a searchable store?
> (Aside from the list archives?)
>
> On 6/9/06, Martin Simmons <martin@lispworks.com> wrote:
> >
> > >>>>> On Fri, 9 Jun 2006 11:23:16 -0400, "Taylor, Joshua" <tayloj@rpi.edu> said:
> > >
> > > Hello all,
> > > I'm playing with pipe streams again, and I'm having a bit
> > > of an issue with command line arguments. I'm on
> > > Mac OS X 10.4.6, on Lispworks 4.4.6 Professional.
> > >
> > > Quoth the docs:
> > >
> > > "Signature
> > > open-pipe command &key direction element-type interrupt-off shell-type => stream
> > > ...
> > > If command  is a string then it is passed to the shell as the command
> > > to run. If command is a list, then the first element is the command to
> > > run directly and the other elements are passed as arguments on the
> > > command line. If command  is nil , then the shell is run."
> > >
> > > So, something like
> > >
> > > (setq qq (sys:open-pipe "/bin/ls -l"))
> > > (loop while (print (read-line qq nil nil)))
> > >
> > > works, as I'd expect it to, the shell interprets the -l correctly.
> > > This example works if I just give "ls -l" as well, which also
> > > seems good; the shell is making sure PATH is consulted.
> > >
> > > Evaluating
> > > (setq qq (sys:open-pipe (list "/bin/ls")))
> > > in the listener causes a terminal window to come up,
> >
> > This is caused by a bug in sys:open-pipe when the arguments are not
> > base-strings.  It works in some situations because the Lisp reader creates
> > strings with the same element type as the stream being read.
> >
> > To work around this, you can do
> >
> > (setq qq (sys:open-pipe (list (coerce "/bin/ls" 'base-string))))
> >
> > --
> > Martin Simmons
> > LispWorks Ltd
> > http://www.lispworks.com/
> >
> >
>
>
> --
> =====================
> Joshua Taylor
> tayloj@rpi.edu
>



-- 
=====================
Joshua Taylor
tayloj@rpi.edu


Re: Code obfuscator (also, encryption)


> Also, if there are encryption libraries people like to use with
> Lispworks, please feel free to mention them... (Ironclad looks good; or
> perhaps we'll wrap our own...)

IIRC Lispworks has a few problems compiling various
files in Ironclad (eg ripemd-160) due to the size of some of the
functions (hopefully it'll just work in LW 5.0), otherwise it's
quite a nice crypto lib.

Cheers,
 Sean.

Updated at: 2020-12-10 08:48 UTC