Failed to enlarge memory

Hi there,

I don't normally concern myself with garbage collection and memory
allocation as I have no need for it in my day-to-day code However, Ive
recently been working on a bit of code where I'm running into
problems: Lispworks becomes unresponsive, and I need to force it
close.

My code has a list of 1000 objects, each of which has three vectors of
length 150, 150 and 1000. I need to do pairwise comparisons of all the
objects in the list, and store a list of the pairs of objects and
dot-products over the vectors, as well as merged vectors. Whether this
is a lot to ask, I don't know, but I'd be a little surprised if it
was.

Either way: After about 300 items through my list Lispworks becomes
unresponsive the console fills with 'Failed to enlarge memory'
messages, and may eventually fail with a 'fail to allocate object of
size x'. Reading through the documentation, it is suggested to
relocate the image using '--relocate-image address. I tried doing
this, but find I still have the same problem (though I should add, I'm
not sure where I should relocate the image).

So I started reading up on Garbage Collection and Memory Allocation
and playing around with (room)

 Generation 0:  Total Size 30229K, Allocated 4599K, Free 25621K
 Generation 1:  Total Size 108852K, Allocated 107762K, Free 1061K
 Generation 2:  Total Size 1004292K, Allocated 1004090K, Free 91K
 Generation 3:  Total Size 1072031K, Allocated 1071834K, Free 172K

Total Size 2215744K, Allocated 2188286K, Free 26946K

Looking at room it strikes me that Generation 2 and Generation 3
simply fill up. Calls to hcl:cleandown make space in Generation 2, but
appear to do so by promoting objects to Generation 3, which just fills
up. When allocated memory starts getting over 2GB, Lispworks just
freezes. And as far as I can tell Generation3 shouldn't normally grow
that large?

Any thoughts on what's going on here? Is this a memory clash? My poor
understanding of Memory Allocation and Garbage collection? Clumsy
coding? Something else altogether?

Lispworks 6.0, Mac OS 10.5.8, 4GB of Ram.

Thanks,

Dan.

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: Failed to enlarge memory

Unable to parse email body. Email id is 11677

Re: Failed to enlarge memory

Thank you Martin,

Yes, as I'm slowly getting my head around this, I realise I was indeed
simply running out of heap space. Luckily, I realised that my vector
notation, while needed for computing dot-products, was rather wasteful
as it contained many zeros. So, I threw that away and replaced it with
a sparser notation that simply notes dimension number and accompanying
value where the value is not zero, thus allowing me to recompute
vectors on the fly. That enabled me to run through the entire list of
1000 items whilst eating up 'only' 500Mb of heap, instead of running
out of heap after 300 items.

Daniel
>
> It is unlikely to be a memory clash -- the upper limit for the heap size in
> 32-bit LispWorks on Mac OS X is about 2.7GB, so you are not far off that.
>
> The first thing to do is to see how much of that data is live.  Call
>
> (room)
> (hcl:mark-and-sweep 3)
> (room)
>
> after the heap has grown too large (e.g. after object allocation failure) and
> compare the total free space in the two calls to room.  If the total free
> space doesn't decrease significantly (in particular in Generation 3) then you
> have a problem with the total amount of live data (either by accidentally
> keeping the objects or because the algorithm requires that amount of data).
>
> --
> Martin Simmons
> LispWorks Ltd
> http://www.lispworks.com/
>
> _______________________________________________
> Lisp Hug - the mailing list for LispWorks users
> lisp-hug@lispworks.com
> http://www.lispworks.com/support/lisp-hug.html
>

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html