[ANN] LispWorks Port of Pierre R. Mai's MD5 Using 32bit Arithmetic

Hello Pierre and lispworkers,

I am pleased to announce the LispWorks version of  Pierre's implementation
of MD5 algorithm (http://www.pmsf.de/resources/lisp/MD5.html).

My version is based on raw 32bit arithmetic introduced in LW 4.4. It
demonstrates speed increase of about two hundred times compared to the naive
port.

The code was tested on LWW 4.4.0 and 4.4.5 beta and available at
    http://lisp.ystok.ru/projects.html

Notes:
1) Putting :md5-small-length onto *features* on LispWorks produces fixnum
instead of (unsigned-byte 29) and imposes size restriction 8 MB as
most-positive-fixnum = 8388607.

2) Invoking md5sum-sequence directly on Unicode string, e.g. lw:text-stirng,
is improper because the code takes char-codes and merge them via
assemble-ub32.
Rule of thumb: Always coerce text string to octets beforehand!

3) It would be nice having LW implemented internally 32-bit circular shift
(rotate) or at least logical right shift operations (in addition to the
arithmetic shift sys:int32>>).

Please enjoy.
--
Sincerely,
Dmitriy Ivanov
lisp.ystok.ru

Re: [ANN] LispWorks Port of Pierre R. Mai's MD5 Using 32bit Arithmetic

"Dmitriy Ivanov" <divanov@aha.ru> writes:

> Hello Pierre and lispworkers,
>
> I am pleased to announce the LispWorks version of  Pierre's implementation
> of MD5 algorithm (http://www.pmsf.de/resources/lisp/MD5.html).
>
> My version is based on raw 32bit arithmetic introduced in LW 4.4. It
> demonstrates speed increase of about two hundred times compared to the naive
> port.

FYI: 
I just tried it out of curiosity on a file of around 40 MB size
(md5 as of Debian unstable) needs:
(time (md5:md5sum-file "/home/frido/ftp/Java/jdk-1_5_0_02-linux-amd64.bin"))
Timing the evaluation of (MD5:MD5SUM-FILE "/home/frido/ftp/Java/jdk-1_5_0_02-linux-amd64.bin")
; Loading fasl file /usr/local/LW4.4/lib/4-4-0-0/load-on-demand/pcl/util/callcoun.ufsl

user time    =    154.538
system time  =      0.076
Elapsed time =   0:02:39
Allocation   = 21154609232 bytes standard / 22220 bytes conses
0 Page faults
#(219 13 20 113 56 84 251 59 233 53 15 214 139 242 103 79)

The new stuff 
time (md5:md5sum-file "/home/frido/ftp/Java/jdk-1_5_0_02-linux-amd64.bin"))
Timing the evaluation of (MD5:MD5SUM-FILE "/home/frido/ftp/Java/jdk-1_5_0_02-linux-amd64.bin")

user time    =      1.420
system time  =      0.017
Elapsed time =   0:00:02
Allocation   = 214624 bytes standard / 4708 bytes conses
0 Page faults
#(219 13 20 113 56 84 251 59 233 53 15 214 139 242 103 79)

This is IMHO very remarkable...

Regards
Friedrich

Re: [ANN] LispWorks Port of Pierre R. Mai's MD5 Using 32bit Arithmetic

On Sun, 10 Apr 2005 13:26:47 +0400, "Dmitriy Ivanov" <divanov@aha.ru> wrote:

> I am pleased to announce the LispWorks version of Pierre's
> implementation of MD5 algorithm
> (http://www.pmsf.de/resources/lisp/MD5.html).
>
> My version is based on raw 32bit arithmetic introduced in LW 4.4. It
> demonstrates speed increase of about two hundred times compared to
> the naive port.

Great, thanks.  Is there a chance that your modifications can be
merged back into the ASDF-installable version which is maintained by
Kevin Rosenberg?

  <http://www.cliki.net/md5>
  <http://files.b9.com/md5/md5-1.8.5.tar.gz>

It'd be nice if one could use the same code for different Lisps.

Thanks again,
Edi.