How to compare a signed value and an unsigned value in x86 assembly

Question

I am having trouble finding a way to compare a positive number and a negative number in x86 assembly code.

For example: when I compare -1 and 1 I always get -1 as greater. I know that it's because 2's complement format makes the -1 bigger than 1 in underlying binary.

But can anyone provide a snippet of x86 assembly to compare positive number with a negative one and get it mathematically correct? (e.g 1 > -1)

Thanks!

Solution 1

You're probably using one of the unsigned variants like:

cmp  eax, ebx
jb   lesser

There are equivalents for checking signed numbers against each other, such as:

cmp  eax, ebx
jl   lesser

This link gives a good run down on the jump variations, including their signed-ness and the flags they check, partially copied here for self-containment:

Instruction  Jump if ...           Signed?   Flags
-----------  -----------           --------  -----
JO           overflow                        OF=1
JNO          not overflow                    OF=0
JS           sign                            SF=1
JNS          not sign                        SF=0
JE/JZ        equal
             zero                            ZF=1
JNE/JNZ      not-equal
             not-zero                        ZF=0
JB/JNAE/JC   below
             not-above-or-equal
             carry                 unsigned  CF=1
JNB/JAE/JNC  not-below
             above-or-equal
             no-carry              unsigned  CF=0
JBE/JNA      below-or-equal
             not-above             unsigned  CF=1 or ZF=1
JA/JNBE      above
             not-below-or-equal    unsigned  CF=0 and ZF=0
JL/JNGE      less
             not-greater-or-equal  signed    SF<>OF
JGE/JNL      greater-or-equal
             not-less              signed    SF=OF
JLE/JNG      less-or-equal
             not-greater           signed    ZF=1 or SF<>OF
JG/JNLE      greater
             not-less-or-equal     signed    ZF=0 and SF=OF
JP/JPE       parity
             parity-even                     PF=1
JNP/JPO      not-parity
             parity-odd                      PF=0
JCXZ/JECXZ   CX register is zero
             ECX register is zero

Solution 2

You can't directly compare two numbers that have different signs. Actually most software languages have that flow. C and C++ specifically mention that in their documentation and in most cases will generate a warning when you use a signed and an unsigned integer in the same expression which then can result in an unknown sign.

The only way is to first check whether the signed number is negative, if so then you know it is smaller. Then you can compare the two numbers as unsigned integers.

; is eax < ebx (eax signed, ebx unsigned)

cmp eax, $0
jl less

cmp eax, ebx
jc less

---

Side note: it is obviously possible to compare two numbers signed if their size is less than the maximum size supported by the processor. In that case you extend the bits of the signed and unsigned appropriately, then you can compare as if both values were signed.

Assuming you wanted to compare two bytes al and bl, then you could have something like this:

movsx ax, al
xor bh, bh    ; or movzx bx, bl
cmp ax, bx
jl less

(Note, I do not guarantee that jl is correct, it may be jle or jnl...)