Creative exploring in Lisp...
Hi,Decades ago, I came from the Forth world. And one of the very first things you do in that world, when bringing up a new Forth system, is to write a memory dump routine. You live and die by that diagnostic.
Not very much at all in the Lisp world -- as long as things are working as you expect them to. This morning I discovered some old code that started breaking a while back in an older implementation of LW. I decided to see if it now works okay in LW 6. The answer is no, but that isn't important. What is important is to have this handy utility in your arsenal to help diagnose why things do or don't work. And while we're at it, we can do some creative exploring...
So I thought I'd share this useful routine with you...
---------------------------------------
(defun dump (obj)
(let* ((addr (sys:object-address obj))
(nb (find-object-size obj)))
(dump-raw addr nb)))
(defun dump-raw (addr nb)
(let* ((limit (+ addr nb))
(p (fli:make-pointer :address addr :type '(:unsigned :byte))))
(labels ((dump-line (addr)
(let ((base (* 16 (truncate addr 16)))
(pb (fli:copy-pointer p))
(pos 0))
(format t "~&~x: " base)
(loop repeat (- addr base) do
(format t " ")
(incf pos 3))
(loop repeat (- (min limit (+ base 8)) addr) do
(format t "~2,'0X " (fli:dereference pb))
(fli:incf-pointer pb)
(incf pos 3))
(format t " ")
(incf pos)
(loop repeat (- (min limit (+ base 16)) (max addr (+ base 8))) do
(format t "~2,'0X " (fli:dereference pb))
(fli:incf-pointer pb)
(incf pos 3))
(format t " ")
(incf pos)
(loop repeat (- 50 pos) do
(format t " "))
(loop repeat (- addr base) do
(format t " "))
(loop repeat (- (min limit (+ base 16)) addr) do
(princ
(let ((ch (code-char (fli:dereference p))))
(if (graphic-char-p ch)
ch
#\.)))
(fli:incf-pointer p))
)))
(loop for base from (* 16 (truncate addr 16)) by 16
for a = addr then base
while (< a limit)
do (dump-line a))
)))
---------------------------------------
Have a look at some simple vectors and watch something very interesting happen... (LWM 32-bit)
CL-USER 1 > (dump #(1 2 3))
22C3BE60: 7E 80 00 80 ~...
22C3BE70: 0C 00 00 00 04 00 00 00 08 00 00 00 0C 00 00 00 ................
It is easy to recognize the Fixnums embedded in the simple-vector.
Now try some floats: (32-bit LWM)
CL-USER 2 > (dump #(1.2 2.3 3.4))
22AB0B30: 7E 80 00 80 ~...
22AB0B40: 0C 00 00 00 AB 0B AB 22 8B 0B AB 22 6B 0B AB 22 ....«.«"..«"k.«"
Now we see that the floats have been boxed, and the vector contains pointers to those boxes.
But from LWM 64-bit, this same dump shows something really interesting...
CL-USER 1 > (dump #(1.2 2.3 3.4))
40200310F0: 2E 00 00 00 18 00 00 00 F7 00 00 00 9A 99 99 3F ........÷......?
4020031100: F7 00 00 00 33 33 13 40 F7 00 00 00 9A 99 59 40 ÷...33.@÷.....Y@
Now, instead of full-blown boxing, we see a kind of "fast-boxing". It is easy to recognize the (little-endian) single-floats in the vector itself, each one preceded by a tag word. Do this again for double-floats and we see full boxing again:
CL-USER 2 > (dump #(1.2d0 2.3d0 3.4d0))
40200647A0: 2E 00 00 00 18 00 00 00 DB 1F 03 30 40 00 00 00 ........Û..0@...
40200647B0: EB 1F 03 30 40 00 00 00 FB 1F 03 30 40 00 00 00 ë..0@...û..0@...
Now try a non-simple array: (LWM 64-bit)
CL-USER 3 > (dump (make-array 3 :element-type 'double-float :initial-contents '(1.2d0 2.3d0 3.4d0)))
4030000A00: 36 1E 00 00 18 00 00 00 33 33 33 33 33 33 F3 3F 6.......333333ó?
4030000A10: 66 66 66 66 66 66 02 40 33 33 33 33 33 33 0B 40 ffffff.@333333.@
... and we see that we are into a really efficient un-boxed representation. Very nice!
-------------------------------------
Here's where this utility shines in diagnosing problems: I want to store a Lisp string, converted to ASCII, into a fixed-length buffer, no null termination:
(fli:with-dynamic-foreign-objects ()
(let* ((str "Hello Dave!")
(nel (length str))
(pdst (fli:allocate-dynamic-foreign-object
:type '(:unsigned :byte)
:nelems (1+ nel) ;; leave room in case terminating null is stored
:initial-element 99)))
(format t "~&Before:")
(dump-raw (fli:pointer-address pdst) (1+ nel))
(unwind-protect
(fli:convert-to-foreign-string str
:external-format :ascii
:null-terminated-p nil
:allow-null nil
;; ------------------------------------------
;; set limit to (1+ nel) and it doesn't crash
;; but it does store a terminating null
:limit nel
;; ------------------------------------------
:into pdst)
(format t "~&After:")
(dump-raw (fli:pointer-address pdst) (1+ nel))
(force-output))))
Oops! Bombs out... complains it overran the buffer...
Before:
1019E00: 63 63 63 63 63 63 63 63 63 63 63 63 cccccccccccc
After:
1019E00: 48 65 6C 6C 6F 20 44 61 76 65 21 63 Hello Dave!c
But luckily, despite the complaint, it didn't do anything to the final byte, as per the limit request. Change that limit to (1+ nel) and we see that the routine does not bomb, but it does write a terminating null byte despite our request not to do so:
Before:
100831200: 63 63 63 63 63 63 63 63 63 63 63 63 cccccccccccc
After:
100831200: 48 65 6C 6C 6F 20 44 61 76 65 21 00 Hello Dave!.
eh? Handy!