x86 Assembly: INC and DEC instruction and overflow flag

Question

In x86 assembly, the overflow flag is set when an add or sub operation on a signed integer overflows, and the carry flag is set when an operation on an unsigned integer overflows.

However, when it comes to the inc and dec instructions, the situation seems to be somewhat different. According to this website, the inc instruction does not affect the carry flag at all.

But I can't find any information about how inc and dec affect the overflow flag, if at all.

Do inc or dec set the overflow flag when an integer overflow occurs? And is this behavior the same for both signed and unsigned integers?

============================= EDIT =============================

Okay, so essentially the consensus here is that INC and DEC should behave the same as ADD and SUB, in terms of setting flags, with the exception of the carry flag. This is also what it says in the Intel manual.

The problem is I can't actually reproduce this behavior in practice, when it comes to unsigned integers.

Consider the following assembly code (using GCC inline assembly to make it easier to print out results.)

int8_t ovf = 0;

__asm__
(
    "movb $-128, %%bh;"
    "decb %%bh;"
    "seto %b0;"
    : "=g"(ovf)
    :
    : "%bh"
);

printf("Overflow flag: %d\n", ovf);

Here we decrement a signed 8-bit value of -128. Since -128 is the smallest possible value, an overflow is inevitable. As expected, this prints out: Overflow flag: 1

But when we do the same with an unsigned value, the behavior isn't as I expect:

int8_t ovf = 0;

__asm__
(
    "movb $255, %%bh;"
    "incb %%bh;"
    "seto %b0;"
    : "=g"(ovf)
    :
    : "%bh"
);

printf("Overflow flag: %d\n", ovf);

Here I increment an unsigned 8-bit value of 255. Since 255 is the largest possible value, an overflow is inevitable. However, this prints out: Overflow flag: 0.

Huh? Why didn't it set the overflow flag in this case?

Solution 1

The overflow flag is set when an operation would cause a sign change. Your code is very close. I was able to set the OF flag with the following (VC++) code:

char ovf = 0;

_asm {
    mov bh, 127
    inc bh
    seto ovf
}
cout << "ovf: " << int(ovf) << endl;

When BH is incremented the MSB changes from a 0 to a 1, causing the OF to be set.

This also sets the OF:

char ovf = 0;

_asm {
    mov bh, 128
    dec bh
    seto ovf
}
cout << "ovf: " << int(ovf) << endl;

Keep in mind that the processor does not distinguish between signed and unsigned numbers. When you use 2's complement arithmetic, you can have one set of instructions that handle both. If you want to test for unsigned overflow, you need to use the carry flag. Since INC/DEC don't affect the carry flag, you need to use ADD/SUB for that case.

Solution 2

As many of the other answers have pointed out, INC and DEC do not affect the CF, whereas ADD and SUB do.

What has not been said yet, however, is that this might make a performance difference. Not that you'd usually be bothered by that unless you are trying to optimise the hell out of a routine, but essentially not setting the CF means that INC/DEC only write to part of the flags register, which can cause a partial flag register stall, see Intel 64 and IA-32 Architectures Optimization Reference Manual or Agner Fog's optimisation manuals.

Solution 3

Intel® 64 and IA-32 Architectures Software Developer's Manuals

Look at the appropriate manual Instruction Set Reference, A-M. Every instruction is precisely documented.

Here is the INC section on affected flags:

The CF flag is not affected. The OF, SZ, ZF, AZ, and PF flags are set according to the result.

Solution 4

try changing your test to pass in the number rather than hard code it, then have a loop that tries all 256 numbers to find the one if any that affects the flag. Or have the asm perform the loop and exit out when it hits the flag and or when it wraps around to the number it started with (start with something other than 0x00, 0x7f, 0x80, or 0xFF).

EDIT

.globl inc
inc:
    mov $33, %eax

top:
    inc %al
    jo done
    jmp top

done:
    ret

.globl dec
dec:
    mov $33, %eax

topx:
    dec %al
    jo donex
    jmp topx

donex:
    ret

Inc overflows when it goes from 0x7F to 0x80. dec overflows when it goes from 0x80 to 0x7F, I suspect the problem is in the way you are using inline assembler.

Solution 5

Except for the carry flag inc sets the flags the same way as add operand 1 would.

The fact that inc does not affect the carry flag is very important.

http://oopweb.com/Assembly/Documents/ArtOfAssembly/Volume/Chapter_6/CH06-2.html#HEADING2-117

View more solutions

Comments

[Peter Cordes]

Incrementing -1 to 0 is not signed overflow, so OF is cleared. teaching.idallen.com/dat2343/10f/notes/040_overflow.txt

[Christian - Reinstate Monica C]

Awesome link, @PeterCordes! That document provides one of the best explanations of anything ever.

[Channel72]

Yes, that was my understanding as well. However, in practice this doesn't seem to actually happen. See my edited question for assembly code demonstrating this.

[Channel72]

Except add also sets the overflow flag if an integer overflow occurs, but as far as I can see, inc does not do the same. See my edited question for assembly code demonstrating this.

[Andrey]

@Channel72 you understand flags incorrectly. CF means carry out of MSB (most significant byte) like 1111 1111 + 1 = 1 | 0000 0000 CF here. CF means invalid (overflow) unsigned operation. (because it is 256, which can't fit into byte). But it is pretty valid signed operation (-1 + 1 = 0). OF is when carry is to MSB, like 0111 1111 + 1 = 1000 0000 this means valid unsigned and invalid signed op (127 + 8 = -128 oops, this is overflow in case of signed, because range is -128; 127)

[old_timer]

signed and unsigned integers are normally a manifestation of programming languages and not reflected in hardware. There isnt a signed inc and unsigned inc, it is just inc one type, universal. That is why OF exists for this instruction. Some processors are not normally that nice and dont provide a way to detect unsigned vs signed results. I was surprised to see this feature actually.

[Channel72]

Well, there is mul versus imul and div versus idiv with the x86 instruction set

[old_timer]

good...normally add and subtract, inc, dec, etc are not sign based because it doesnt make sense.

[Peter Cordes]

It can lead to a partial flag stall if a later instruction reads CF, e.g. in an adc loop. See Problems with ADC/SBB and INC/DEC in tight loops on some CPUs. But unlike Pentium 4, it's usually not a problem unless you're specifically taking advantage of inc/dec leaving CF unmodified by reading CF later. INC instruction vs ADD 1: Does it matter? (usually no). Partial-flag stalls are only a thing on Intel in P6 family; Sandybridge did fairly efficient flag merging.

[Peter Cordes]

Also note that recent Intel (Skylake and maybe earlier) don't have flag-merging at all. CF is still renamed separately, but instructions like cmova that want CF and ZF just have 2 separate flag inputs. (That's why cmova costs 2 uops, because it has 4 inputs, but most other cmov conditions only need either CF or some flags from the SPAZO group, not both) See @BeeOnRope (Travis Downs') thread agner.org/optimize/blog/read.php?i=960