Lisp HUG Maillist Archive

Plea for Unsigned Binary operators and a few more...

In doing some crypto programming in Lisp I ran into a need for bit rotate operators. And just as surely, stumbled into the demand that INT32 and INT64 represent SIGNED-BYTE quantities… uh oh…

Signed arithmetic is asymmetric in left / right shift operators. In left shift a zero bit is shifted into the LSB of the register, while in right shifts the sign bit is propagated rightward. Hence, whenever a bit pattern (usually unsigned in crypto work) has its sign bit set, you can simply OR together left shifted and right shifted components to achieve a bit pattern rotate.

I came up with a “fast” (?) solution that dispenses with any test and branch by inventing a CNOT (conditional one’s complement) function:

(defun cnot (x)
(int64-logxor x
(int64>> x 63)))

so that the propagating sign bit effects are removed when defining rotate operators:

(defun rotl (x n)
(int64-logxor
(int64>> x (- 64 n))
(int64<< (cnot x) n)))

(defun rotr (x n)
(int64-logxor
(int64>> x n)
(int64<< (cnot x) (- 64 n))))

But it sure would be nice to have some ROT operators (ROTL, ROTR).

In addition, I can think of no instance where defaulting to unsigned behavior would be any different in arithmetic than signed, except in the right shift operation. So for this one case, there is the Symmetric case of Unsigned right-shift, and the Asymmetric case of Signed right-shift. Why not just default to unsigned operands always, and implement two different kinds of right-shift?

Am I missing any cases?

- DM

Re: Plea for Unsigned Binary operators and a few more...

err… damn autocorrect… should have read:

“ in rotate you *cannot* simply OR together left shifted and … “


On Jun 27, 2015, at 4:22 PM, David McClain <DBM@REFINED-AUDIOMETRICS.COM> wrote:

In doing some crypto programming in Lisp I ran into a need for bit rotate operators. And just as surely, stumbled into the demand that INT32 and INT64 represent SIGNED-BYTE quantities… uh oh…

Signed arithmetic is asymmetric in left / right shift operators. In left shift a zero bit is shifted into the LSB of the register, while in right shifts the sign bit is propagated rightward. Hence, whenever a bit pattern (usually unsigned in crypto work) has its sign bit set, you can simply OR together left shifted and right shifted components to achieve a bit pattern rotate.

I came up with a “fast” (?) solution that dispenses with any test and branch by inventing a CNOT (conditional one’s complement) function:

(defun cnot (x)
(int64-logxor x
(int64>> x 63)))

so that the propagating sign bit effects are removed when defining rotate operators:

(defun rotl (x n)
(int64-logxor
(int64>> x (- 64 n))
(int64<< (cnot x) n)))

(defun rotr (x n)
(int64-logxor
(int64>> x n)
(int64<< (cnot x) (- 64 n))))

But it sure would be nice to have some ROT operators (ROTL, ROTR).

In addition, I can think of no instance where defaulting to unsigned behavior would be any different in arithmetic than signed, except in the right shift operation. So for this one case, there is the Symmetric case of Unsigned right-shift, and the Asymmetric case of Signed right-shift. Why not just default to unsigned operands always, and implement two different kinds of right-shift?

Am I missing any cases?

- DM

Re: Plea for Unsigned Binary operators and a few more...

I made a request for efficient 32/64-bit unsigned arithmetic some time ago as well. I likewise came up on this for crypto-like applications. Most recently, I ran across it in my implementation of Murmurhash 3..

https://bitbucket.org/tarballs_are_good/murmurhash3

You can see the bit arithmetic code here

https://bitbucket.org/tarballs_are_good/murmurhash3/raw/22430ee1a8daf81f3165bf0592ef30a4f29c4583/utilities.lisp

SBCL has SB-ROTATE-BYTE:ROTATE-BYTE which, similar to DPB, allows rotation of arbitrary sub-bytes of an unsigned-byte.

Cheers,

Robert

On Sat, Jun 27, 2015 at 4:36 PM, David McClain <dbm@refined-audiometrics.com> wrote:
… along those lines, the demand for SIGNED-BYTE quantities also causes a stumbling block on conversion to / from INT32 and INT64. Any INTEGERs larger than #x7fff…. in whichever width causes heartburn to the conversion routine which barfs and says the bit pattern is too large, even though it explicitly fits within the stated bit-width of 32 or 64 bits. 

Hence, the SIGNED-BYTE viewpoint requires a precautionary use of ENSURE-64BITS ahead of handing off any unsigned quantities, where

(define ensure-64bits (x)
(if (>= x #.(ash 1 63))
(- x #.(ash 1 64))
    x))

There is really no difference in the resulting bit pattern, but this kind of nonsense is required to get past the bounds checking police…

- DM


On Jun 27, 2015, at 4:25 PM, David McClain <DBM@REFINED-AUDIOMETRICS.COM> wrote:

err… damn autocorrect… should have read:

“ in rotate you *cannot* simply OR together left shifted and … “


On Jun 27, 2015, at 4:22 PM, David McClain <DBM@REFINED-AUDIOMETRICS.COM> wrote:

In doing some crypto programming in Lisp I ran into a need for bit rotate operators. And just as surely, stumbled into the demand that INT32 and INT64 represent SIGNED-BYTE quantities… uh oh…

Signed arithmetic is asymmetric in left / right shift operators. In left shift a zero bit is shifted into the LSB of the register, while in right shifts the sign bit is propagated rightward. Hence, whenever a bit pattern (usually unsigned in crypto work) has its sign bit set, you can simply OR together left shifted and right shifted components to achieve a bit pattern rotate.

I came up with a “fast” (?) solution that dispenses with any test and branch by inventing a CNOT (conditional one’s complement) function:

(defun cnot (x)
(int64-logxor x
(int64>> x 63)))

so that the propagating sign bit effects are removed when defining rotate operators:

(defun rotl (x n)
(int64-logxor
(int64>> x (- 64 n))
(int64<< (cnot x) n)))

(defun rotr (x n)
(int64-logxor
(int64>> x n)
(int64<< (cnot x) (- 64 n))))

But it sure would be nice to have some ROT operators (ROTL, ROTR).

In addition, I can think of no instance where defaulting to unsigned behavior would be any different in arithmetic than signed, except in the right shift operation. So for this one case, there is the Symmetric case of Unsigned right-shift, and the Asymmetric case of Signed right-shift. Why not just default to unsigned operands always, and implement two different kinds of right-shift?

Am I missing any cases?

- DM



Updated at: 2020-12-10 08:33 UTC