Lisp HUG Maillist Archive

Need help with efficient passing of strings via FLI

Dear Lisp Huggers,

I have a question concerning the intricacies of LW's FLI.
Here's the setup: I'm thinking of writing a set of FLI bindings
for Sleepycat's Berkeley DB (http://www.sleepycat.com) based
on the code published by Paul Foley (mycroft@actrix.gen.nz).
That code currently only runs on CMUCL.

That code has function which look like this:
    (defun db-get (db key &optional datum default
		      &key (start 0) end db-start db-end for-update
		      (transaction *transaction*) (flags 0))
      ... body... )

The idea is you do things like:
    (db-get db "foo")  => "bar"

But also, if you need to be really efficient (which I do)
you have some buffer buf, which you reuse, and which you
send in as the field DATUM.

    (db-get db "foo" buf) => "bar"
    buf => "bar"

Okay.  Now, db-get is implemented to call the following foreign function,

    (def-alien-routine ("dbx_get" %db-get) c-call:int
      (handle sys:system-area-pointer)
      (key c-call:c-string)
      (kstart c-call:unsigned-int)
      (kend c-call:unsigned-int)
      (datum c-call:c-string)
      (dstart c-call:unsigned-int)
      (dend c-call:unsigned-int)
      (txnid sys:system-area-pointer)
      (partial c-call:int)
      (start c-call:unsigned-int)
      (end c-call:unsigned-int)
      (flags c-call:unsigned-int)
      (result sys:system-area-pointer :out)
      (rlength c-call:unsigned-int :out))

This all seems reasonable, except for one thing: What to do
with the c-call:c-string bits.

So far, all I have to work on are the FLI doc and entry 17004
in LW's knowlegebase, entitled:
  What to do about "Cannot pass foreign aggregate type :EF-MB-STRING" errors

With that as a basis, I'm thinking of writing something like this:

    (define-foreign-function (%db-get "dbx_get")
      ((handle (:pointer :void))
       (key    (:reference-pass (:ef-mb-string :limit +key-limit-size+)))
       (kstart (:unsigned :int))
       (kend   (:unsigned :int))
       (datum  (:reference (:ef-mb-string :limit +datum-limit-size+)))
       (dstart (:unsigned :int))
       (dend   (:unsigned :int))
       (txnid  (:pointer :void))
       ;; Rest omitted...
       )
      :result-type :int)

Where the hope is to indicate:
 a)  that the key is passed "read-only"
 b)  that the datum is writable by C
 c)  that, when Lisp arranges for %db-get to be called, the actual
     memory buffers used by the lisp strings are what is sent to C
     (i.e. there is not consing in either direction)
 d)  let C coerce the actual DB* to/from void*.

AFAIK, all 3 of a, b, c and d are probably wrong, and I would very much
value your collective inputs.  :-)  So far, writing little test
destructive functions in C, I am just getting segmentation violations,
so clearly I'm missing (at least one) step.

Am I on the right track?



p.s. Finally, note that this code will be given back on the same terms as
     Paul's original code.


--
			Alain Picard
			Memetrics


Re: Need help with efficient passing of strings via FLI

Alain,

>I have a question concerning the intricacies of LW's FLI.
>Here's the setup: I'm thinking of writing a set of FLI bindings
>for Sleepycat's Berkeley DB (http://www.sleepycat.com) based
>on the code published by Paul Foley (mycroft@actrix.gen.nz).
>That code currently only runs on CMUCL.


You might consider using UFFI (http://uffi.med-info.com/) to specify 
the foreign bindings. This way, the same code can be used to support 
LispWorks and any other environment that has a UFFI implementation 
(currently LispWorks, ACL, CMUCL, and MCL).

I created a Lisp interface for the Valentina relational database -- 
you can find it here: http://www.paradigmasoft.com/files/vcsdk.html. 
This might provide some examples to help you in your efforts.

Best,

John DeSoi, Ph.D.


Updated at: 2020-12-10 09:01 UTC