FLI: pointer to lisp object in FLI callback

Hi all,

I'm trying to use FLI for some C library which uses callbacks.

I create foreign-callable and transfer it to C function. However the C API supports sending opaque pointers as an argument to callback function. I want to supply some of my Lisp objects as this opaque pointer, so my Lisp foreign callable, being called from C code, will receive a Lisp object as an argument.

How to specify this argument ?

BR,

/Alexey

Re: FLI: pointer to lisp object in FLI callback

That won't be generally possible, because you'd need to coerce the object into a pointer value or some such.

The approach I've seen most often (and used myself) is to keep a hash-table of id=>CL objects. Generate a unique ID and give thas as the opaque pointer.

Then in your callback:

(let ((my-object (gethash (fli:pointer-address opaque-ptr) *objects-list*)))
;; Do stuff with your object
)

.. Where *objects-list* is this hash table, and opaque-ptr is the argument to the callback

On Mon, Nov 2, 2020 at 2:31 PM Alexey Veretennikov <txm.fourier@gmail.com> wrote:

Hi all,

I'm trying to use FLI for some C library which uses callbacks.
I create foreign-callable and transfer it to C function. However the C API supports sending opaque pointers as an argument to callback function. I want to supply some of my Lisp objects as this opaque pointer, so my Lisp foreign callable, being called from C code, will receive a Lisp object as an argument.
How to specify this argument ?

BR,
/Alexey

Wilfredo Velázquez-Rodríguez

Unable to parse email body. Email id is 15408

Re: FLI: pointer to lisp object in FLI callback

Thanks,

Apparently all solutions require some sort of the global variable. However during my evening experiments I was able to get the solution which does not require explicit creation of the global variable, but using allocation in the particular GC generation.

Here is the code for the call of the foreign function supplying Lisp callback. I create the object (Lisp struct) in long-living GC generation (2), take the address to it, allocate memory to hold the address and supply pointer to the foreign function(note the code is for 32bit LW):

;; create the object in "static" memory. generation 2 supposed to be safe from GC move
(allocation-in-gen-num 2
(let* ((helper (make-visitor-helper))
(helper-addr (sys:object-address helper))) ; get the object address
;; allocate dynamic memory for address and copy it here
(fli:with-integer-bytes (p len) helper-addr
;; call the foreign function 'visitchildren' supplying our callback 'visitor'
(visitchildren visitor

p))))

In my Lisp callable function I do the reverse: dereference the pointer to get the integer value (address in the generation 2) and then get the object by using this address:

;; check if the data supplied is not a null pointer
(unless (fli:null-pointer-p client-data)
;; dereference the opaque pointer to integer and get the value
(when-let* ((addr (fli:dereference client-data :type '(:unsigned :int)))
;; get the lisp object from address
(obj (sys:pointer-from-address addr)))
;; works!
(format t "filename: ~a~%" (visitor-helper-filename obj))))

/Alexey

On Mon, Nov 2, 2020 at 8:36 PM HZ- <964411818@qq.com> wrote:

You may use the third library which called "CFFI" ,or refer to the book <common lisp recipes > for help.

------------------ 原始邮件 ------------------
发件人: "Alexey Veretennikov"<txm.fourier@gmail.com>;
发送时间: 2020年11月3日(星期二) 凌晨3:28
收件人: "Lispworks HUG"<lisp-hug@lispworks.com>;
主题: FLI: pointer to lisp object in FLI callback

Hi all,

I'm trying to use FLI for some C library which uses callbacks.
I create foreign-callable and transfer it to C function. However the C API supports sending opaque pointers as an argument to callback function. I want to supply some of my Lisp objects as this opaque pointer, so my Lisp foreign callable, being called from C code, will receive a Lisp object as an argument.
How to specify this argument ?

BR,
/Alexey

Re: FLI: pointer to lisp object in FLI callback

That approach will work most of the time but it is not robust because you
can't 100% guarantee that the GC will not move the object.  Also, the layout
of bytes from fli:with-integer-bytes in unspecified so pairing it with
fli:dereference is not supported.  Finally, wrapping allocation-in-gen-num
around the whole form means that many other objects will be allocated in
generation 2.

Does the callback happen inside visitchildren?  If so, then the simplest
solution is to bind a special variable around the call and use that inside the
callback:

(defvar *current-visitor-helper*)

(let ((*current-visitor-helper* helper))
  (visitchildren visitor nil))

Then use *current-visitor-helper* in the callback.

-- 
Martin Simmons
LispWorks Ltd
http://www.lispworks.com/



>>>>> On Mon, 2 Nov 2020 23:02:00 +0100, Alexey Veretennikov said:
> 
> Thanks,
> 
> Apparently all solutions require some sort of the global variable. However
> during my evening experiments I was able to get the solution which does not
> require explicit creation of the global variable, but using allocation in
> the particular GC generation.
> Here is the code for the call of the foreign function supplying Lisp
> callback. I create the object (Lisp struct) in long-living GC generation
> (2), take the address to it, allocate memory to hold the address and supply
> pointer to the foreign function(note the code is for 32bit LW):
> 
>       ;; create the object in "static" memory. generation 2 supposed to be
> safe from GC move
>       (allocation-in-gen-num 2
>         (let* ((helper (make-visitor-helper))
>                (helper-addr (sys:object-address helper))) ; get the object
> address
>           ;; allocate dynamic memory for address and copy it here
>           (fli:with-integer-bytes (p len) helper-addr
>             ;; call the foreign function 'visitchildren' supplying our
> callback 'visitor'
>             (visitchildren visitor
>                            p))))
> 
> In my Lisp callable function I do the reverse: dereference the pointer to
> get the integer value (address in the generation 2) and then get the object
> by using this address:
> 
>   ;; check if the data supplied is not a null pointer
>   (unless (fli:null-pointer-p client-data)
>     ;; dereference the opaque pointer to integer and get the value
>     (when-let* ((addr (fli:dereference client-data :type '(:unsigned :int)))
>                 ;; get the lisp object from address
>                 (obj (sys:pointer-from-address addr)))
>       ;; works!
>       (format t "filename: ~a~%" (visitor-helper-filename obj))))
> 
> BR
> /Alexey
> 
> 
> 
> On Mon, Nov 2, 2020 at 8:36 PM HZ- <964411818@qq.com> wrote:
> 
> > You may use the third library which called "CFFI" ,or refer to the book
> > <common lisp recipes > for help.
> >
> >
> > ------------------ 原始邮件 ------------------
> > *发件人:* "Alexey Veretennikov"<txm.fourier@gmail.com>
> > *发送时间:* 2020年11月3日(星期二) 凌晨3:28
> > *收件人:* "Lispworks HUG"<lisp-hug@lispworks.com>
> > *主题:* FLI: pointer to lisp object in FLI callback
> >
> > Hi all,
> >
> > I'm trying to use FLI for some C library which uses callbacks.
> > I create foreign-callable and transfer it to C function. However the C API
> > supports sending opaque pointers as an argument to callback function. I
> > want to supply some of my Lisp objects as this opaque pointer, so my Lisp
> > foreign callable, being called from C code, will receive a Lisp object as
> > an argument.
> > How to specify this argument ?
> >
> > BR,
> > /Alexey
> >
> 

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: FLI: pointer to lisp object in FLI callback

Thanks Martin,

This is much simpler and works.

Br,

/Alexey

On Tue, Nov 3, 2020 at 11:35 AM Martin Simmons <martin@lispworks.com> wrote:

That approach will work most of the time but it is not robust because you
can't 100% guarantee that the GC will not move the object. Also, the layout
of bytes from fli:with-integer-bytes in unspecified so pairing it with
fli:dereference is not supported. Finally, wrapping allocation-in-gen-num
around the whole form means that many other objects will be allocated in
generation 2.

Does the callback happen inside visitchildren? If so, then the simplest
solution is to bind a special variable around the call and use that inside the
callback:

(defvar *current-visitor-helper*)

(let ((*current-visitor-helper* helper))
(visitchildren visitor nil))

Then use *current-visitor-helper* in the callback.

--
Martin Simmons
LispWorks Ltd
http://www.lispworks.com/

>>>>> On Mon, 2 Nov 2020 23:02:00 +0100, Alexey Veretennikov said:
>
> Thanks,
>
> Apparently all solutions require some sort of the global variable. However
> during my evening experiments I was able to get the solution which does not
> require explicit creation of the global variable, but using allocation in
> the particular GC generation.
> Here is the code for the call of the foreign function supplying Lisp
> callback. I create the object (Lisp struct) in long-living GC generation
> (2), take the address to it, allocate memory to hold the address and supply
> pointer to the foreign function(note the code is for 32bit LW):
>
> ;; create the object in "static" memory. generation 2 supposed to be
> safe from GC move
> (allocation-in-gen-num 2
> (let* ((helper (make-visitor-helper))
> (helper-addr (sys:object-address helper))) ; get the object
> address
> ;; allocate dynamic memory for address and copy it here
> (fli:with-integer-bytes (p len) helper-addr
> ;; call the foreign function 'visitchildren' supplying our
> callback 'visitor'
> (visitchildren visitor
> p))))
>
> In my Lisp callable function I do the reverse: dereference the pointer to
> get the integer value (address in the generation 2) and then get the object
> by using this address:
>
> ;; check if the data supplied is not a null pointer
> (unless (fli:null-pointer-p client-data)
> ;; dereference the opaque pointer to integer and get the value
> (when-let* ((addr (fli:dereference client-data :type '(:unsigned :int)))
> ;; get the lisp object from address
> (obj (sys:pointer-from-address addr)))
> ;; works!
> (format t "filename: ~a~%" (visitor-helper-filename obj))))
>
> BR
> /Alexey
>
>
>
> On Mon, Nov 2, 2020 at 8:36 PM HZ- <964411818@qq.com> wrote:
>
> > You may use the third library which called "CFFI" ,or refer to the book
> > <common lisp recipes > for help.
> >
> >
> > ------------------ 原始邮件 ------------------
> > *发件人:* "Alexey Veretennikov"<txm.fourier@gmail.com>;
> > *发送时间:* 2020年11月3日(星期二) 凌晨3:28
> > *收件人:* "Lispworks HUG"<lisp-hug@lispworks.com>;
> > *主题:* FLI: pointer to lisp object in FLI callback
> >
> > Hi all,
> >
> > I'm trying to use FLI for some C library which uses callbacks..
> > I create foreign-callable and transfer it to C function. However the C API
> > supports sending opaque pointers as an argument to callback function. I
> > want to supply some of my Lisp objects as this opaque pointer, so my Lisp
> > foreign callable, being called from C code, will receive a Lisp object as
> > an argument.
> > How to specify this argument ?
> >
> > BR,
> > /Alexey
> >
>

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html