RE: Internal Lispworks limit on size of fasls?
That's what I'm doing right now; a simple binary protocol for our clos objects. Please mail me your serialization library thought, I'd love to take a look. Thanks much! Cheers, David -----Original Message----- From: Arthur Lemmens [mailto:alemmens@xs4all.nl] Sent: Wednesday, September 01, 2004 5:16 PM To: davef@xanalys.com; deyoung@bloodhoundinc.com Cc: lisp-hug@xanalys.com Subject: Re: Internal Lispworks limit on size of fasls? Dave Fox wrote: > Sounds as if you hit an internal limit in the fasl loader. See the > thread on this list in February 2004 titled "Saving and loading big > objects (or: limits for FASL files)" I started that thread. I solved my problem by writing my own serializer & deserializer that assumes non-circularity as the default. That means that it doesn't need to be as general as the FASL-dumper. It worked quite well for my application: I got rid of the "pig in the snake" symptom, where memory use increases enormously while creating and dumping all the load forms and then goes back to normal again. Speed was also much better (a factor of 10 or 20, IIRC) than the FASL-dumper. David Young wrote: > > We've worked around this problem by partitioning these troublesome tables > > into multiple parts. However, I'd like to understand this issue; there are a > > few "smallish" tables where the error occurs when I would not have expected > > it, and some larger tables that had no trouble at all. That may be related to the amount of 'cross references' in the data. An array of floats may be easier to dump than a graph, for example. If you're interested in my serialization library, I can mail it to you. There's also something called CL-STORE on common-lisp.net that you may want to look at. Regards, Arthur Lemmens This email message is for the sole use of the intended recipients(s) and may contain confidential and privileged information of Bloodhound Software, Inc.. Any unauthorized review, use, disclosure is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.