Writing and reading multidimensional arrays of floats

I have a nine dimensional array of single-floats (specialized).

Writing this with write-with-fasl-file, writes this out quite quickly, but 
writes a 20Mb fasl file.
Reading this fasl file takes a looong time. Presumably because it creates a 
non-specialized array and doesn't know the size in advance.

How do people generally read and write such arrays? I would also prefer to 
use a compressed format.

I am quite happy to loop over the array, but how do I best write out the 
data?

Rene.



______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________

Re: Writing and reading multidimensional arrays of floats

Rene de Visser wrote:

> I have a nine dimensional array of single-floats (specialized).

Based on what I have seen below try to dump this nine-dimensional
array as a specialized one-dimensional single-float array.

It seems you cannot specialize a multi-dimensional array of floats in LW,

Simple example,

CL-USER 26 > (type-of (make-array 10 :element-type 'single-float))
(SIMPLE-ARRAY SINGLE-FLOAT (10))

CL-USER 27 > (type-of (make-array '(10 10) :element-type 'single-float))
SIMPLE-ARRAY

With the following functions defined,

(defvar *working-array* nil)

(defun dump-marray (array)
  (hcl:dump-forms-to-file
   (compile-file-pathname "marray")
   (list `(setf *working-array* ,array))))

(defun load-marray ()
  (sys:load-data-file (compile-file-pathname "marray")))

CL-USER 19 > (progn (setf array (make-array '(2 2 2 7 7 7 7 7 7)
                                           :element-type 'single-float
                                           :initial-element 1.0s10))
              nil)
NIL

CL-USER 20 > (dump-marray array)
2

CL-USER 21 > (load-marray)
; Loading fasl file d:\cygwin\home\wade\marray.fsl

..... Takes forever, (breaking into the debugger shows load-array calling 
adjust-array (alot))

However doing this instead is much faster, (single dimension array)

CL-USER 28 > (progn (setf array (make-array (* 2 2 2 7 7 7 7 7 7)
                                           :element-type 'single-float
                                           :initial-element 1.0s10))
              nil)
NIL

CL-USER 29 > (type-of array)
(SIMPLE-ARRAY SINGLE-FLOAT (941192))

CL-USER 30 > (dump-marray array)
2

CL-USER 31 > (load-marray)
; Loading fasl file d:\cygwin\home\wade\marray.fsl
#P"d:/cygwin/home/wade/marray.fsl"

Is nearly instantaneous.

Wade


______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________

Re: Writing and reading multidimensional arrays of floats

>From: Wade Humeniuk <whumeniu@telus.net>
>It seems you cannot specialize a multi-dimensional array of floats in LW,
>
>Simple example,
>
>CL-USER 26 > (type-of (make-array 10 :element-type 'single-float))
>(SIMPLE-ARRAY SINGLE-FLOAT (10))
>
>CL-USER 27 > (type-of (make-array '(10 10) :element-type 'single-float))
>SIMPLE-ARRAY

I think it is type-of that gives the wrong information here. Try and store 
something else in that 2d array and you will see what I mean.

>.... Takes forever, (breaking into the debugger shows load-array calling 
>adjust-array (alot))
>
>However doing this instead is much faster, (single dimension array)
>
>CL-USER 28 > (progn (setf array (make-array (* 2 2 2 7 7 7 7 7 7)
>                                           :element-type 'single-float
>                                           :initial-element 1.0s10))
>              nil)
>NIL
>
>CL-USER 29 > (type-of array)
>(SIMPLE-ARRAY SINGLE-FLOAT (941192))
>
>CL-USER 30 > (dump-marray array)
>2
>
>CL-USER 31 > (load-marray)
> Loading fasl file d:\cygwin\home\wade\marray.fsl
>#P"d:/cygwin/home/wade/marray.fsl"
>
>Is nearly instantaneous.
>
>Wade
>
Interesting, thankyou for your reply.

I since written my own serializer that works at byte level. It is fast 
enough for me and results in a much smaller file. However it is special 
purpose, not a general solutuion.

Rene.



______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________