Size of delivered executables

Hi!

I have two comments/questions about the size of delivered executables
both of which apply to Linux as well as Windows:

1. I've delivered my "Regex Coach" with level 5 and managed to get it
   fairly small I think. The funny thing is that now, even if I make
   changes to the code, add or delete some lines or whatever, the size
   of the executable stays _exactly_ the same on both platforms. How
   can that be? I would have thought that if the executable was as
   small as possible adding a function (which is really used) should
   increase the size by at least a couple of bytes.

2. Applying gzip to the executable reduces the size significantly -
   from about 6MB to 2MB. I seem to remember that Corman Lisp uses a
   trick where on delivery the image is gzipped and unpacked
   automatically once the app starts. Wouldn't that be an option for
   LW as well? It reduces the "perceived" size of the app
   considerably.

No, I'm not whining about the size of Lisp executables but of course I
hear comments like "Cool program, but it would probably be 90% smaller
if it were written in <insert your favorite C-like language here>."

Cheers,
Edi.

Re: Size of delivered executables

From: "Edi Weitz" <edi@agharta.de>
> I have two comments/questions about the size of delivered executables
> both of which apply to Linux as well as Windows:
>
> 1. I've delivered my "Regex Coach" with level 5 and managed to get it
>    fairly small I think. The funny thing is that now, even if I make
>    changes to the code, add or delete some lines or whatever, the size
>    of the executable stays _exactly_ the same on both platforms. How
>    can that be? I would have thought that if the executable was as
>    small as possible adding a function (which is really used) should
>    increase the size by at least a couple of bytes.

You are rather lucky, my web applications end up at >22Mb. I can't deliver
them at a level higher than 1. But it's not really a problem.
I once managed to go down to 3.5Mb for a command line program delivered at
level 5 and was rather impressed by this.

> 2. Applying gzip to the executable reduces the size significantly -
>    from about 6MB to 2MB. I seem to remember that Corman Lisp uses a
>    trick where on delivery the image is gzipped and unpacked
>    automatically once the app starts. Wouldn't that be an option for
>    LW as well? It reduces the "perceived" size of the app
>    considerably.

Sure. It would be a good psychological option and it should not be difficult
to do this.

> No, I'm not whining about the size of Lisp executables but of course I
> hear comments like "Cool program, but it would probably be 90% smaller
> if it were written in <insert your favorite C-like language here>."

As usual... Just tell them to do it. ;-)

Marc

Re: Size of delivered executables

Unable to parse email body. Email id is 1076

Re: Size of delivered executables

> I question if it is that important. Are we really so short of disk > space these days? Don't we prefer faster application startup?
In my opinion, size matters. :-) I think Edi was right--it's just a perceptual thing. But perception matters. And when you are, in a sense, coming from behind (ie, Lisp as a mainstream language) it doesn't hurt to spend some time on perceptual issues. Completely subjective I know--but fwiw.

I will say, however, that in other apps we've used compression to speed up application loading time. We discovered that CPUs are so much faster than disks that we were able to load a compressed image and unzip it faster than we could load an uncompressed image. By "image" in this case I mean data rather than code, but there you go.

(One word of warning, though it may be too obvious to mention. If you want to try to reproduce the numbers you have to do a clean start. Once something is loaded it's obviously in the file system cache, and at that point the cache dominates all other results.)

Re: Size of delivered executables

MichaelL@frogware.com writes:

> I will say, however, that in other apps we've used compression to
> speed up application loading time. We discovered that CPUs are so
> much faster than disks that we were able to load a compressed image
> and unzip it faster than we could load an uncompressed image. By
> "image" in this case I mean data rather than code, but there you go.

Interesting point...

Re: Size of delivered executables

David Fox <davef@xanalys.com> writes:

> The size of a LispWorks executable is a header plus an integral
> number of 64K chunks. So adding a function will only change the size
> of the delivered image if it causes a new 64k chunk to be added.

OK, I expected something like that. Thanks for the explanation.

> 6Mb is fairly large for a compacted delivered LWW executable.

I would be very happy if you could help me to make it significantly
smaller... :)

> Is your executable compacted (that is, what is the value of the
> delivery keyword :COMPACT)?

T

> How many lines of code?

CL-PPCRE has around 6,000 LOC with many comments, the additional code
for the "Regex Coach" (GUI stuff, some additions to the regex engine)
is at around 2,500 LOC. (Measured with wc, i.e. including empty
lines.)

> I question if it is that important. Are we really so short of disk
> space these days? Don't we prefer faster application startup?

For me at least it's not about disk space, it's about network
traffic. If I release a newer version of my app I have to upload it
(two versions) to the server which takes several minutes with my ADSL
connection (that's why they call it "asymmetric"...) and more or less
blocks all other network traffic during that period.

Not that my gzip proposal would help here because I'm already
uploading gzipped files...

> If, indeed. You have implemented Regex Coach pretty quickly, and
> will be able to maintain it with less effort than an equivalent
> C-like implementation, won't you? I think that's the best riposte,
> rather than arguing over a few Mb of (cheap) disk space.

Sure. I myself would never even think about building something similar
with C++ or Delphi. I'm just talking about psychological factors
here. Some things are just hard to argue against.

Well, never mind, none of this is really important. But if it isn't
much work to add something like a :GZIP delivery keyword you might
want to add it to the wishlist for one of the next releases.

Thanks,
Edi.

Re: Size of delivered executables

David Fox <davef@xanalys.com> writes:

> 6Mb is fairly large for a compacted delivered LWW executable. Is
> your executable compacted (that is, what is the value of the
> delivery keyword :COMPACT)? How many lines of code?

I forgot to mention that the app has to keep most of the editor
code. Would that explain its size?

FWIW, below is my delivery file. IIRC the last two KEEP options are
redundant and are only remnants from my playing around with other
delivery levels than 5.

Edi.

---------------------- snip ---------------------------------------

(in-package "CL-USER")

(hcl:toggle-source-debugging nil)

(setq system:*sg-default-size* 128000)

(load #+win32 "c:/cygwin/home/edi/lisp/defsystem"
      #-win32 "/usr/local/lisp/defsystem")

#+win32 (pushnew "c:/cygwin/home/edi/lisp/cl-ppcre" mk:*central-registry*)

(pushnew #+win32 "c:/cygwin/home/edi/lisp/cl-ppcre-interactive"
         #-win32 "/home/edi/cl-ppcre-interactive"
         mk:*central-registry*)

(mk:compile-system "cl-ppcre-interactive")

(deliver 'cl-ppcre-interactive::main
         #+win32 "The Regex Coach"
         #-win32 "regex-coach"
         5
         #+win32 :icon-file #+win32 "icon_big.ico"
         :editor-commands-to-keep '("Backward Character"
                                    "Backward Character Cancelling Selection"
                                    "Backward Character Extending Selection"
                                    "Backward List"
                                    "Backward Paragraph"
                                    "Backward Sentence"
                                    "Backward Up List"
                                    "Backward Word"
                                    "Backward Word Extending Selection"
                                    "Beginning Of Buffer Cancelling Selection"
                                    "Beginning Of Buffer Extending Selection"
                                    "Beginning of Buffer"
                                    "Beginning of Line"
                                    "Beginning of Line Cancelling Selection"
                                    "Beginning of Line Extending Selection"
                                    "Beginning of Window"
                                    "Bind Key"
                                    "Buffer Not Modified"
                                    "Capitalize Word"
                                    "Delete Blank Lines"
                                    "Delete Horizontal Space"
                                    "Delete Next Character"
                                    "Delete Previous Character"
                                    "Down List"
                                    "End Of Buffer Cancelling Selection"
                                    "End Of Buffer Extending Selection"
                                    "End Of Line Cancelling Selection"
                                    "End Of Line Extending Selection"
                                    "End of Buffer"
                                    "End of Line"
                                    "End of Window"
                                    "Exchange Point and Mark"
                                    "Fill Region"
                                    "Forward Character"
                                    "Forward Character Cancelling Selection"
                                    "Forward Character Extending Selection"
                                    "Forward Kill Sentence"
                                    "Forward List"
                                    "Forward Paragraph"
                                    "Forward Sentence"
                                    "Forward Word"
                                    "Forward Word Cancelling Selection"
                                    "Forward Word Extending Selection"
                                    "Indent New Line"
                                    "Indent"
                                    "Insert ()"
                                    "Just One Space"
                                    "Kill Line"
                                    "Kill Next Word"
                                    "Kill Previous Word"
                                    "Kill Region"
                                    "Lowercase Region"
                                    "Lowercase Word"
                                    "Mark Page"
                                    "Mark Paragraph"
                                    "Mark Whole Buffer"
                                    "Move Over )"
                                    "Move To Window Line"
                                    "Negative Argument"
                                    "New Line"
                                    "Next Line"
                                    "Next Line Cancelling Selection"
                                    "Next Line Extending Selection"
                                    "Next Page"
                                    "Open Line"
                                    "Overwrite Mode"
                                    "Pop Mark"
                                    "Previous Line"
                                    "Previous Line Cancelling Selection"
                                    "Previous Line Extending Selection"
                                    "Previous Page"
                                    "Quoted Insert"
                                    "Refresh Screen"
                                    "Rotate Kill Ring"
                                    "Save Region Preserving Selection"
                                    "Scroll Window Down"
                                    "Scroll Window Down Extending Selection"
                                    "Scroll Window Up"
                                    "Scroll Window Up Extending Selection"
                                    "Self Insert"
                                    "Self Overwrite"
                                    "Set Mark"
                                    "Set Prefix Argument"
                                    "Switch To Other Editor"
                                    "Transpose Characters"
                                    "Transpose Lines"
                                    "Transpose Words"
                                    "Un-Kill"
                                    "Undo"
                                    "Uppercase Region"
                                    "Uppercase Word")
         :keep-eval nil
         :interface :capi
         :keep-complex-numbers nil
         :compact t)
(quit)

Re: Size of delivered executables

Edi Weitz <edi@agharta.de> writes:

> For me at least it's not about disk space, it's about network
> traffic. If I release a newer version of my app I have to upload it
> (two versions) to the server which takes several minutes with my ADSL
> connection (that's why they call it "asymmetric"...) and more or less
> blocks all other network traffic during that period.

I release most new versions as patches (fasl-files) which are
automatically downloaded (confirmed by the user) at application
startup.  (However, this makes the initial image fairly large (20MB),
I don't think I can get around delivering with level 0 and all bells
and whistles, since the image needs to be fully dynamic and able to
load fasl files)
-- 
  (espen)

Re: Size of delivered executables

David Fox <davef@xanalys.com> writes:

> I question if it is that important. Are we really so short of disk
> space these days? 
I assume Edi is not thinking of disk space but bandwidth...

Regards
Friedrich