Lisp HUG Maillist Archive

open-url

Hello,
 
I use open-url to open a web page from within a lisp application.
Occasionally, I find that the URL has changed, specifically to a general caterory of discontinued urls.
How can I algorithmically go through my application to find such changed URLs?
 
For example a function such as:
 
(defun find-discontinued-URLS (URL-in)
  ;; ouput as URL-out (a string) if the URL is redirected.
)
 
Sheldon

Re: open-url

On Mar 21, 2012, at 20:58 , Sheldon Ball wrote:
> Hello,
>  
> I use open-url to open a web page from within a lisp application.
> Occasionally, I find that the URL has changed, specifically to a general caterory of discontinued urls.
> How can I algorithmically go through my application to find such changed URLs?
>  
> For example a function such as:
>  
> (defun find-discontinued-URLS (URL-in)
>   ;; ouput as URL-out (a string) if the URL is redirected.
> )

Looks like you might be able to use Edit Weitz' drakma package for this; the http-request method takes a :redirect parameter that you may set to 0 to raise an error if there is a redirect. Alternatively, if you use the default setting for this variable, you could examine the return headers to see if any redirects have been applied.

For example:

(defun check-uri (uri)
              (multiple-value-bind (content-stream code headers new-uri headers-stream close-p status) 
                  (drakma:http-request uri :want-stream t)
                (unless (puri:uri= new-uri (puri:uri uri))
                  (princ-to-string new-uri))))

Note: you probably want to add some parameters (or change some of the values I've used), change the names used for the return values, close the return stream(s), etc.

I guess it would also be possible to call this function from a defadvice over #'open-url, and log all redirected urls somewhere...

Updated at: 2020-12-10 08:36 UTC