How to pass unicode string to sys:call-system

Hi,

I'm failing to pass a path containing unicode characters as an argument
to call-system on Linux:

```
CL-USER 60 > (sys:call-system '("/usr/bin/xdg-open" "/home/fourier/мой_файл.png"))

Error: #\м (of type character) is not of type base-char.
  1 (abort) Return to top loop level 0.
```

I've tried to convert the string to utf8 and then to base-string but
apparently the argument was not accepted by the system either:

(defun string-to-base-string (str)
  (loop with len = (length str)
        with result = (make-string len :element-type 'base-char)
        for n across (babel:string-to-octets str :encoding :utf-8)
        for i below len 
        do (setf (schar result i) (code-char n))
        finally (return result)))

CL-USER 66 > (sys:call-system (list "/usr/bin/xdg-open" (string-to-base-string "/home/fourier/мой_файл.png")))
4

Even then the file is clearly there. 
System locale is UTF-8

LANG=en_US.UTF-8


Any ideas how to pass the unicode string as a command line argument to
call-system ?

-- 
Br,
/Alexey

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: How to pass unicode string to sys:call-system

Hi Alexey,

I got bit by this a couple years ago on Mac OS. So here’s what I came up with…

Re: How to pass unicode string to sys:call-system

... I didn’t remember how I did it. But after looking at the code I just sent you, I scan the proposed command for any “errant” characters. If some are found, then I construct a temporary text file containing the command string, and then have the system shell execute that script for me. 

I needed this when I was scanning Music CD’s which often had European characters embedded in the names of the extracted files. It works for me...

- DM

> On Jun 25, 2020, at 11:19 AM, dbm@refined-audiometrics.com wrote:
> 
> Hi Alexey,
> 
> I got bit by this a couple years ago on Mac OS. So here’s what I came up with…
> 
> <safe-call-system.lisp>
> 
> - DM
> 
>> On Jun 25, 2020, at 10:23 AM, Alexey Veretennikov <txm.fourier@gmail.com> wrote:
>> 
>> Hi,
>> 
>> I'm failing to pass a path containing unicode characters as an argument
>> to call-system on Linux:
>> 
>> ```
>> CL-USER 60 > (sys:call-system '("/usr/bin/xdg-open" "/home/fourier/мой_файл.png"))
>> 
>> Error: #\м (of type character) is not of type base-char.
>> 1 (abort) Return to top loop level 0.
>> ```
>> 
>> I've tried to convert the string to utf8 and then to base-string but
>> apparently the argument was not accepted by the system either:
>> 
>> (defun string-to-base-string (str)
>> (loop with len = (length str)
>>       with result = (make-string len :element-type 'base-char)
>>       for n across (babel:string-to-octets str :encoding :utf-8)
>>       for i below len 
>>       do (setf (schar result i) (code-char n))
>>       finally (return result)))
>> 
>> CL-USER 66 > (sys:call-system (list "/usr/bin/xdg-open" (string-to-base-string "/home/fourier/мой_файл.png")))
>> 4
>> 
>> Even then the file is clearly there. 
>> System locale is UTF-8
>> 
>> LANG=en_US.UTF-8
>> 
>> 
>> Any ideas how to pass the unicode string as a command line argument to
>> call-system ?
>> 
>> -- 
>> Br,
>> /Alexey
>> 
>> _______________________________________________
>> Lisp Hug - the mailing list for LispWorks users
>> lisp-hug@lispworks.com
>> http://www.lispworks.com/support/lisp-hug.html
> 


_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: How to pass unicode string to sys:call-system

No guarantees, but try:

(eval-when (:compile-toplevel :load-toplevel :execute)
  #+lispworks (lw:set-default-character-element-type 'character))


> On Jun 25, 2020, at 1:23 PM, Alexey Veretennikov <txm.fourier@gmail.com> wrote:
> 
> Hi,
> 
> I'm failing to pass a path containing unicode characters as an argument
> to call-system on Linux:
> 
> ```
> CL-USER 60 > (sys:call-system '("/usr/bin/xdg-open" "/home/fourier/мой_файл.png"))
> 
> Error: #\м (of type character) is not of type base-char.
>  1 (abort) Return to top loop level 0.
> ```
> 
> I've tried to convert the string to utf8 and then to base-string but
> apparently the argument was not accepted by the system either:
> 
> (defun string-to-base-string (str)
>  (loop with len = (length str)
>        with result = (make-string len :element-type 'base-char)
>        for n across (babel:string-to-octets str :encoding :utf-8)
>        for i below len 
>        do (setf (schar result i) (code-char n))
>        finally (return result)))
> 
> CL-USER 66 > (sys:call-system (list "/usr/bin/xdg-open" (string-to-base-string "/home/fourier/мой_файл.png")))
> 4
> 
> Even then the file is clearly there. 
> System locale is UTF-8
> 
> LANG=en_US.UTF-8
> 
> 
> Any ideas how to pass the unicode string as a command line argument to
> call-system ?
> 
> -- 
> Br,
> /Alexey
> 
> _______________________________________________
> Lisp Hug - the mailing list for LispWorks users
> lisp-hug@lispworks.com
> http://www.lispworks.com/support/lisp-hug.html


_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: How to pass unicode string to sys:call-system

As DM mentioned, writing the command to a temporary script file is the
simplest workaround to this problem.

Your function string-to-base-string doesn't work because the utf-8 encoded
string in longer than str.  You need something starting like this (untested):

  (loop with bytes = (babel:string-to-octets str :encoding :utf-8)
        with len = (length bytes)
        with result = (make-string len :element-type 'base-char)
        for n across bytes
        ...)

-- 
Martin Simmons
LispWorks Ltd
http://www.lispworks.com/



>>>>> On Thu, 25 Jun 2020 19:23:59 +0200, Alexey Veretennikov said:
> 
> Hi,
> 
> I'm failing to pass a path containing unicode characters as an argument
> to call-system on Linux:
> 
> ```
> CL-USER 60 > (sys:call-system '("/usr/bin/xdg-open" "/home/fourier/мой_файл.png"))
> 
> Error: #\м (of type character) is not of type base-char.
>   1 (abort) Return to top loop level 0.
> ```
> 
> I've tried to convert the string to utf8 and then to base-string but
> apparently the argument was not accepted by the system either:
> 
> (defun string-to-base-string (str)
>   (loop with len = (length str)
>         with result = (make-string len :element-type 'base-char)
>         for n across (babel:string-to-octets str :encoding :utf-8)
>         for i below len 
>         do (setf (schar result i) (code-char n))
>         finally (return result)))
> 
> CL-USER 66 > (sys:call-system (list "/usr/bin/xdg-open" (string-to-base-string "/home/fourier/мой_файл.png")))
> 4
> 
> Even then the file is clearly there. 
> System locale is UTF-8
> 
> LANG=en_US.UTF-8
> 
> 
> Any ideas how to pass the unicode string as a command line argument to
> call-system ?
> 
> -- 
> Br,
> /Alexey
> 
> _______________________________________________
> Lisp Hug - the mailing list for LispWorks users
> lisp-hug@lispworks.com
> http://www.lispworks.com/support/lisp-hug.html
> 

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: How to pass unicode string to sys:call-system

Hi Martin,

Your finding for string-to-base-string actually solved the problem. Now
with the code as simple as

(defun string-to-base-string (str)
  (loop with bytes = (babel:string-to-octets str :encoding :utf-8)
        with len = (length bytes)
        with result = (make-string len :element-type 'base-char)
        for n across bytes 
        for i below len 
        do (setf (schar result i) (code-char n))
        finally (return result)))

with the call

(sys:call-system (list "/usr/bin/xdg-open" (string-to-base-string "/home/fourier/мой_файл.png")))

finally works. Thanks everyone for the contribution!


Martin Simmons <martin@lispworks.com> writes:

> As DM mentioned, writing the command to a temporary script file is the
> simplest workaround to this problem.
>
> Your function string-to-base-string doesn't work because the utf-8 encoded
> string in longer than str.  You need something starting like this (untested):
>
>   (loop with bytes = (babel:string-to-octets str :encoding :utf-8)
>         with len = (length bytes)
>         with result = (make-string len :element-type 'base-char)
>         for n across bytes
>         ...)

-- 
Br,
/Alexey

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html