RE: Browser (client) side: bring up cPanel website, success and problem

I used Drakma to get a cPanel login page. "https://assigned-email-manager.com:2083/" 
It worked easily and perfectly when my function returned a string that displayed in the Listener. That's the success part.

So, then I modified the code to output to a file. It would not output the file. Raised an error. I added (finish-output out-stream) to my code and found that the output to file stopped with error when it got some high value Unicode character. It reported U+0167.

I guessed that the characters were coming from the cPanel page's drop down box that allows you to select the language you want. Each language is displayed in its own character set (Arabic in Arabic letters, etc.).

What I wound up doing is (mod chr 256) to force any character to be within range.

Even though I got the file to write "without errors", I think my solution is a poor solution. Maybe it is okay for problem isolation. Seems like I should be able to get characters from all languages to write to a file.

(format t "~c" chr) and (write-char chr) write without problem to the Listener.
(format out-stream "~c" chr) and (write-char chr out-stream) to a file result in bouncing out to the debugger with the error that complains about not being able to write U+0167.

Here is the main part of the code:

=============== start code ==================================
;;;; 
;;;; Package xcomm-drakma
;;;; 
;;;; experiment with drakma library
;;;; 
;;;; 
;;;; 
;;;; 

(in-package xcomm-drakma)

(ql:quickload "drakma")



(defparameter url-str "https://assigned-email-manager.com:2083/")

(defparameter data-path "D:/Users/Ron/Documents/LispWorks/comm/data/")


(defun out-file ()
  (concatenate 'string 
               data-path
               "cpanel-response.html"))



(defun go-to-cpanel ()
  (drakma:http-request url-str))


(defun output-str-to-file-2 (chr-lst outstrm)
  (mapcar #'(lambda (chr)
              (let ((chr-1 (if (< (char-code chr) 256)   
                               chr 
                             (code-char (mod (char-code chr) 256)))))  ; poor solution!!!
                (write-char chr-1 outstrm)
                (finish-output outstrm)))
          chr-lst))




(defun output-str-to-file-1 (str outstrm)
  (cond ((null outstrm)
         (format t "~%output file did not open.~%"))

        (t
         (output-str-to-file-2 (coerce str 'list) outstrm))))



(defun output-str-to-file (str file)
  (with-open-file (out-strm file
                            :direction :output
                            :if-exists :supersede
                            :if-does-not-exist :create)   ; maybe need to specify more here?
                            
    
    (output-str-to-file-1 str out-strm)))


(defun out-cpanel-to-file ()
  (let ((response (go-to-cpanel)))
    (output-str-to-file response (out-file))
    (format t "~%cpanel response output to file cpanel-response.html~%")))
================ end code ============================

Invoking the last function, out-cpanel-to-file, is supposed to output cPanel's response to a file.

This is the kind of problem I think I would solve by reading documentation. I'm looking here: http://www.lispworks.com/documentation/lw70/CLHS/Body/m_w_open.htm 

Is this where I should be reading? 

Ron Lewis



_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: Browser (client) side: bring up cPanel website, success and problem

Yes, you should read through that, paying particular attention to what it says about :external-format. You should also skim through the chapter on "The external-format package" in the Lispworks Reference Manual.

Note: the external format that I personally would use for this (:utf-8) is not mentioned in the external-formats chapter - I'm not sure if this is just an omission, or if there's a valid reason for it.

On Tue, Mar 20, 2018 at 4:01 AM, Ron Lewis <rlewis-4d@indinfer.com> wrote:

I used Drakma to get a cPanel login page. "https://assigned-email-manager.com:2083/"
It worked easily and perfectly when my function returned a string that displayed in the Listener. That's the success part.

So, then I modified the code to output to a file. It would not output the file. Raised an error. I added (finish-output out-stream) to my code and found that the output to file stopped with error when it got some high value Unicode character. It reported U+0167.

I guessed that the characters were coming from the cPanel page's drop down box that allows you to select the language you want. Each language is displayed in its own character set (Arabic in Arabic letters, etc.).

What I wound up doing is (mod chr 256) to force any character to be within range.

Even though I got the file to write "without errors", I think my solution is a poor solution. Maybe it is okay for problem isolation. Seems like I should be able to get characters from all languages to write to a file.

(format t "~c" chr) and (write-char chr) write without problem to the Listener.
(format out-stream "~c" chr) and (write-char chr out-stream) to a file result in bouncing out to the debugger with the error that complains about not being able to write U+0167.

Here is the main part of the code:

=============== start code ==================================
;;;;
;;;; Package xcomm-drakma
;;;;
;;;; experiment with drakma library
;;;;
;;;;
;;;;
;;;;

(in-package xcomm-drakma)

(ql:quickload "drakma")

(defparameter url-str "https://assigned-email-manager.com:2083/")

(defparameter data-path "D:/Users/Ron/Documents/LispWorks/comm/data/")

(defun out-file ()
(concatenate 'string
data-path
"cpanel-response.html"))

(defun go-to-cpanel ()
(drakma:http-request url-str))

(defun output-str-to-file-2 (chr-lst outstrm)
(mapcar #'(lambda (chr)
(let ((chr-1 (if (< (char-code chr) 256)
chr
(code-char (mod (char-code chr) 256))))) ; poor solution!!!
(write-char chr-1 outstrm)
(finish-output outstrm)))
chr-lst))

(defun output-str-to-file-1 (str outstrm)
(cond ((null outstrm)
(format t "~%output file did not open.~%"))

(t
(output-str-to-file-2 (coerce str 'list) outstrm))))

(defun output-str-to-file (str file)
(with-open-file (out-strm file
:direction :output
:if-exists :supersede
:if-does-not-exist :create) ; maybe need to specify more here?

(output-str-to-file-1 str out-strm)))

(defun out-cpanel-to-file ()
(let ((response (go-to-cpanel)))
(output-str-to-file response (out-file))
(format t "~%cpanel response output to file cpanel-response.html~%")))
================ end code ============================

Invoking the last function, out-cpanel-to-file, is supposed to output cPanel's response to a file.

This is the kind of problem I think I would solve by reading documentation. I'm looking here: http://www.lispworks.com/documentation/lw70/CLHS/Body/m_w_open.htm

Is this where I should be reading?

Ron Lewis

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: Browser (client) side: bring up cPanel website, success and problem

You may have set lw:*default-character-element-type* in your startup file; the default on OSX at least is base-char (for 6.1.1, 7.0 and 7.1).

> On 20 Mar 2018, at 21:39 , Erik Ronström <erik.ronstrom@doremir.com> wrote:
> 
> It is strange that you have to explicitly specify :element-type ’character. From the listener, I can successfully write unicode characters to a file using this code:
> 
> (with-open-file (out-strm "~/Desktop/test.txt"
>                           :direction :output
>                           :if-exists :supersede
>                           :if-does-not-exist :create
>                           :external-format :utf-8)
>  (format out-strm "づ”))
> 
> What platform are you on? (Don’t know if that matters, but just maybe)
> 
> Erik
> 
> 
> 
>> 20 mars 2018 kl. 15:34 skrev Ron Lewis <rlewis-4d@indinfer.com>:
>> 
>> 
>> Erik and Raymond,
>> 
>> Sounds like I was a little bit on the right direction. Seems like the Lisp Standard documentation (http://www.lispworks.com/documentation/lw70/CLHS/Body/f_open.htm) only tells me about :default as a value possibility for :external-format key value. I suppose only telling about :default makes sense for documentation of standard Lisp because :default is the only value that is required by the standard.
>> 
>> You have led me to look at LispWorks specific documentation which I hope I can remember for future self-sufficient problem solving. Looking at LispWorks specific documentation makes sense because the value I want will be specific to  LispWorks.
>> 
>> And Bingo! I started up LispWorks IDE. In the Listener I typed “(open” and left the cursor on ‘n’. Then I clicked on Help >> Symbol, etc. This time, instead of going to ANSI Common Lisp Standard, I went to “open function 689” in LispWorks User Guide and Reference Manual. I thought, “So far, doesn’t seem to help enough. Where are my options?” But at the bottom I saw “See also” and clicked on “valid-external-format-p”.
>> 
>> valid-external-format-p looks to be valuable to tell me whether or not I am specifying an :external-format with valid syntax. And it has an example of what I might specify, “ ‘(:Unicode :eol-style :lf)) ”. 
>> 
>> So, anyway I will interrupt my blow by blow account to get to what worked.
>> 
>>  (with-open-file (out-strm file
>>                            :direction :output
>>                            :if-exists :supersede
>>                            :if-does-not-exist :create 
>>                            :external-format :utf-8
>>                            :element-type 'character)
>> 
>> I am surprised that nothing worked until I specified both :external-format and :element-type. I thought :character was the default. And then :character was incorrect. I needed ‘character (apostrophe instead of colon).
>> 
>> My working code above is based on the LispWorks User Guide and Reference Manual (thank you for pointing me to it), Chapter-Section 24.2.6 “Instantiating the stream”. And a list of :external-format’s I could use are at 26.6 “External Formats”. :utf-8 looks best in epsilon (my emacs-like text editor). But :unicode and :utf-16 also work (no errors and the file gets written), just that only :utf-8 looks good in epsilon. (I think I could configure epsilon to display the other :external-formats, if I want to bother with it.)
>> 
>> I don’t know why I need to specify :element-type ‘character since character is supposed to be the default, right? Maybe my specifying :external-format affects :element-type somehow?
>> 
>> In any case, thank you for helping me to become more self-sufficient (I hope) in Lisp problem solving.
>> 
>> Now I have success and success instead of success with a problem.
>> 
>> Ron Lewis
>> 
>> From: owner-lisp-hug@lispworks.com <owner-lisp-hug@lispworks.com> On Behalf Of Raymond Wiker
>> Sent: Tuesday, March 20, 2018 3:27 AM
>> To: Ron Lewis <rlewis-4d@indinfer.com>
>> Cc: Lisp HUG <lisp-hug@lispworks.com>
>> Subject: Re: Browser (client) side: bring up cPanel website, success and problem
>> 
>> Yes, you should read through that, paying particular attention to what it says about :external-format. You should also skim through the chapter on "The external-format package" in the Lispworks Reference Manual.
>> 
>> Note: the external format that I personally would use for this (:utf-8) is not mentioned in the external-formats chapter - I'm not sure if this is just an omission, or if there's a valid reason for it.
>> 
>> On Tue, Mar 20, 2018 at 4:01 AM, Ron Lewis <rlewis-4d@indinfer.com> wrote:
>>> I used Drakma to get a cPanel login page. "https://assigned-email-manager.com:2083/"
>>> It worked easily and perfectly when my function returned a string that displayed in the Listener. That's the success part.
>>> 
>>> So, then I modified the code to output to a file. It would not output the file. Raised an error. I added (finish-output out-stream) to my code and found that the output to file stopped with error when it got some high value Unicode character. It reported U+0167.
>>> 
>>> I guessed that the characters were coming from the cPanel page's drop down box that allows you to select the language you want. Each language is displayed in its own character set (Arabic in Arabic letters, etc.).
>>> 
>>> What I wound up doing is (mod chr 256) to force any character to be within range.
>>> 
>>> Even though I got the file to write "without errors", I think my solution is a poor solution. Maybe it is okay for problem isolation. Seems like I should be able to get characters from all languages to write to a file.
>>> 
>>> (format t "~c" chr) and (write-char chr) write without problem to the Listener.
>>> (format out-stream "~c" chr) and (write-char chr out-stream) to a file result in bouncing out to the debugger with the error that complains about not being able to write U+0167.
>>> 
>>> Here is the main part of the code:
>>> 
>>> =============== start code ==================================
>>> ;;;;
>>> ;;;; Package xcomm-drakma
>>> ;;;;
>>> ;;;; experiment with drakma library
>>> ;;;;
>>> ;;;;
>>> ;;;;
>>> ;;;;
>>> 
>>> (in-package xcomm-drakma)
>>> 
>>> (ql:quickload "drakma")
>>> 
>>> 
>>> 
>>> (defparameter url-str "https://assigned-email-manager.com:2083/")
>>> 
>>> (defparameter data-path "D:/Users/Ron/Documents/LispWorks/comm/data/")
>>> 
>>> 
>>> (defun out-file ()
>>>  (concatenate 'string
>>>               data-path
>>>               "cpanel-response.html"))
>>> 
>>> 
>>> 
>>> (defun go-to-cpanel ()
>>>  (drakma:http-request url-str))
>>> 
>>> 
>>> (defun output-str-to-file-2 (chr-lst outstrm)
>>>  (mapcar #'(lambda (chr)
>>>              (let ((chr-1 (if (< (char-code chr) 256)
>>>                               chr
>>>                             (code-char (mod (char-code chr) 256)))))  ; poor solution!!!
>>>                (write-char chr-1 outstrm)
>>>                (finish-output outstrm)))
>>>          chr-lst))
>>> 
>>> 
>>> 
>>> 
>>> (defun output-str-to-file-1 (str outstrm)
>>>  (cond ((null outstrm)
>>>         (format t "~%output file did not open.~%"))
>>> 
>>>        (t
>>>         (output-str-to-file-2 (coerce str 'list) outstrm))))
>>> 
>>> 
>>> 
>>> (defun output-str-to-file (str file)
>>>  (with-open-file (out-strm file
>>>                            :direction :output
>>>                            :if-exists :supersede
>>>                            :if-does-not-exist :create)   ; maybe need to specify more here?
>>> 
>>> 
>>>    (output-str-to-file-1 str out-strm)))
>>> 
>>> 
>>> (defun out-cpanel-to-file ()
>>>  (let ((response (go-to-cpanel)))
>>>    (output-str-to-file response (out-file))
>>>    (format t "~%cpanel response output to file cpanel-response.html~%")))
>>> ================ end code ============================
>>> 
>>> Invoking the last function, out-cpanel-to-file, is supposed to output cPanel's response to a file.
>>> 
>>> This is the kind of problem I think I would solve by reading documentation. I'm looking here: http://www.lispworks.com/documentation/lw70/CLHS/Body/m_w_open.htm
>>> 
>>> Is this where I should be reading?
>>> 
>>> Ron Lewis
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Lisp Hug - the mailing list for LispWorks users
>>> lisp-hug@lispworks.com
>>> http://www.lispworks.com/support/lisp-hug.html
> 
> 
> _______________________________________________
> Lisp Hug - the mailing list for LispWorks users
> lisp-hug@lispworks.com
> http://www.lispworks.com/support/lisp-hug.html


_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html