Checking Overflow with Inline Assembly

This question often comes up in interviews, and real-world projects also frequently need to address the same issue: how to determine whether an integer operation has overflowed. C# provides the checked keyword to handle this problem; in C, we can use inline assembly statements to check for overflow.

In computer processors, the overflow flag (sometimes called V flag) is usually a single bit in a system status register used to indicate when an arithmetic overflow has occurred in an operation, indicating that the signed two’s-complement result would not fit in the number of bits used for the operation (the ALU width).

In C, registers cannot be accessed directly, but their values can be accessed through inline assembly. One drawback of doing this is that assembly code depends on the computer architecture, so portability requires additional consideration.

After researching this, I found that on the X86 architecture, the assembly instructions JO and JNO can be used to implement conditional jumps based on the Overflow flag. JO (Jump on Overflow) means to jump when overflow occurs, while JNO (Jump on Not Overflow) is the opposite. The following small example explains how to perform overflow checking.

Input: a string that matches the regular expression [+|-]?\n+.

Output: whether the input string can be converted to an int constant without overflow.

bool TryParseInt(const char *str, int *pn)
{
    bool isAddOp = true;
    if (*str == '+') {
        str++;
    } else if (*str == '-') {
        isAddOp = false;
        str++;
    }

    int n = 0;
    while (*str != '\0') {
        n *= 10;
        __asm__ goto ("jo %l0"
            : /* no output */
            : /* no input */
            : /* no clobber */
            : PARSE_INT_FAILED
        );
        if (isAddOp) {
            n += *str - '0';
        } else {
            n -= *str - '0';
        }
        __asm__ goto ("jo %l0"
            : /* no output */
            : /* no input */
            : /* no clobber */
            : PARSE_INT_FAILED
        );
        str++;
    }
    *pn = n;
    return true;

PARSE_INT_FAILED:
    return false;
}

Failure Cases

Sometimes the compiler uses the LEA instruction to optimize calculations. Because the LEA instruction does not set any flags, JO cannot be used to detect arithmetic overflow in that case.