Re: FLI and :REFERENCE types

Hi Edi,

even if the memory is freed upon garbage collection it would be an  
error to do so.

Imagine the situation if the C data structure you get returned via a  
pointer itself contains further data structures that have been  
allocated by malloc() and its friends... In Lisp land you wouldn't  
know about these and therefore the Lisp shouldn't free automatically  
any object without calling a deep-free function that recursively walks  
down the data structure and frees any memory having been allocated  
dynamically.

When looking at your example with ULONG * there might be argued that  
this is a trivial case because the returned object is known not to  
contain any further dynamically allocated memory - or is the :unsigned- 
long meant to be a pointer address ?
If not then deallocating the object would be ok - no memory leaks  
resulting.

Not really the answer you were looking for but when Lispworks' garbage  
collector is programmed properly it would know that it can't free a  
NULL address. So a safe way to handle it would be:

Free the data yourself.
Set the address of the pointer to NULL.
Done.

I'd like to hear comments - I do so myself in another Lisp and there  
this works. And I have to port to LW at some point in time ...

;-)

Frank

Am 24.03.2008 um 12:57 schrieb Edi Weitz:
>
> I'm suddenly confused about :REFERENCE types in LispWorks' FLI
> although I've used them quite often in the past.  Maybe someone can
> deconfuse me.  I've defined some C struct and a corresponding foreign
> function like this:
>
>  (define-c-struct my-config
>    (type :unsigned-long)
>    (interface :unsigned-long)
>    (software-version :unsigned-long)
>    (hardware-version :unsigned-long)
>    (serial-number :unsigned-long)
>    (memory-size :unsigned-long))
>
>  (define-foreign-function (read-config "ReadConfig")
>      ((product-id :unsigned-long)
>       (category :unsigned-long)
>       (data (:reference-return sgl-config)))
>    :result-type :unsigned-long
>    :module :foo
>    :lambda-list (product-id &optional (category 0) &aux data))
>
> The original C function's prototype looks like so
>
>  ULONG ReadConfig(
>    ULONG ProductId,
>    ULONG Category,
>    ULONG *Data);
>
> and the C programmer is of course supposed to allocate memory for Data
> and take care of it.
>
> Now, if I define and call READ-CONFIG as above, I'll automatically get
> the FLI structure as the second return value and can look at its
> values.  But - do I have to call FREE-FOREIGN-OBJECT once I'm finished
> with it?  The documentation for :REFERENCE says that
>
>  "an object of TYPE is dynamically allocated across the scope of the
>   foreign function, and is automatically de-allocated once the
>   foreign function terminates"
>
> and I think the behaviour is quite clear when I have something like
>
>  (:reference-return :unsigned-long)
>
> where I'll end up with a Lisp integer being returned and don't have to
> care about the intermediate C unsigned long integer.  But what exactly
> is allocated and de-allocated and when in the case of aggregate types
> as above?
>
> Looking at the macro expansion of DEFINE-FOREIGN-FUNCTION for
> READ-CONFIG, it seems to me as if the struct for DATA is originally
> stack-allocated but then copied into memory which is allocated and not
> freed.  But maybe the memory will be automatically freed when the Lisp
> object which is the second return value of READ-CONFIG is
> garbage-collected?
>
> Thanks,
> Edi.
>

Re: FLI and :REFERENCE types

Hi Frank,

On Mon, 24 Mar 2008 13:43:32 +0100, Frank Goenninger <frgo@mac.com> wrote:

> even if the memory is freed upon garbage collection it would be an
> error to do so.
>
> Imagine the situation if the C data structure you get returned via a
> pointer itself contains further data structures that have been
> allocated by malloc() and its friends... In Lisp land you wouldn't
> know about these and therefore the Lisp shouldn't free automatically
> any object without calling a deep-free function that recursively
> walks down the data structure and frees any memory having been
> allocated dynamically.
>
> When looking at your example with ULONG * there might be argued that
> this is a trivial case because the returned object is known not to
> contain any further dynamically allocated memory - or is the
> :unsigned-long meant to be a pointer address ?  If not then
> deallocating the object would be ok - no memory leaks resulting.
>
> Not really the answer you were looking for but when Lispworks'
> garbage collector is programmed properly it would know that it can't
> free a NULL address. So a safe way to handle it would be:
>
> Free the data yourself.
> Set the address of the pointer to NULL.
> Done.
>
> I'd like to hear comments - I do so myself in another Lisp and there
> this works. And I have to port to LW at some point in time ...

I understand your concerns, but I think you're expecting too much.
We're talking about C, after all... :)

I would argue that a) if you declare something to be of type
:UNSIGNED-LONG and use it as a pointer you should know what you're
doing and b) if you free the space a C struct occupies which itself
points to other objects, that's not a memory leak.  In the case of b),
another pointer could point to the same object, so your "deep-free"
function would be wrong unless it was a full GC with all bells and
whistles - and even then it would be wrong as it wouldn't be able to
know what happens in C land.

In other words, for things like this, there should be a clear
contract.  If a C function asks you to allocate space for a struct and
then puts a pointer to some other object into this struct, it should
be clearly specified who is responsible for the memory occupied by
this object.  This has nothing to do with the memory for the struct
itself, I'd say.

Coming back to my original question, I'd like to see the :REFERENCE
type as a kind of "translator" between C and Lisp.  Once it has done
its work (in the case of :REFERENCE-RETURN), what I get back is a Lisp
object (which might "wrap" a C struct) that can be treated like any
other Lisp object - specifically, I can forget about it when it goes
out of scope.  And I'm asking for confirmation that this POV is
correct... :)

Cheers,
Edi.

Re: FLI and :REFERENCE types

Hello Edi,

|   (define-c-struct my-config
|     (type :unsigned-long)
|     (interface :unsigned-long)
|     (software-version :unsigned-long)
|     (hardware-version :unsigned-long)
|     (serial-number :unsigned-long)
|     (memory-size :unsigned-long))
| 
|   (define-foreign-function (read-config "ReadConfig")
|       ((product-id :unsigned-long)
|        (category :unsigned-long)
|        (data (:reference-return sgl-config)))
|     :result-type :unsigned-long
|     :module :foo
|     :lambda-list (product-id &optional (category 0) &aux data))
| 
| The original C function's prototype looks like so
| 
|   ULONG ReadConfig(
|     ULONG ProductId,
|     ULONG Category,
|     ULONG *Data);
| 
| and the C programmer is of course supposed to allocate memory for Data
| and take care of it.

Yes, :reference sounds a bit vague. That is why I prefer proven patterns
like :pointer and decoding foreign structures into Lisp ones ASAP.

(define-foreign-function (|ReadConfig| "ReadConfig")
     ((product-id :unsigned-long)
      (category :unsigned-long)
      (data (:pointer sgl-config)))
   :result-type :unsigned-long
   :module :foo)

(defun read-config (product-id &optional (category 0))
  (fli:with-dynamic-foreign-objects ((sc sql-config))
    (let* ((result (|ReadConfig| product-id categor sc)))
      (values result
                 (fli:foreign-slot-value sc 'type)
                 ...)))))
--
Sincerely,
Dmitriy Ivanov
lisp.ystok.ru

Re: FLI and :REFERENCE types

Unable to parse email body. Email id is 7895