Add keyword argument for `dis.dis()` displaying the grouped opcode?

Python Help
1

I tried to use dis module to print the opcode for explaining the difference between “a = a + b” and “a += b”, but sadly I got the same one: “BINARY_OP”.

It seems that displaying opcodes like INPLACE_ADD for __iadd__() and BINARY_ADD for __add__() is far more understandable than the only BINARY_OP.

Is there a switch to display those?

Or I misunderstood something important?

2

AFAIK, INPLACE_ADD and BINARY_ADD just don’t exists as opcodes in the current version. They all got folded into BINARY_OP. dis does display the operation that is performed based on the argument for the opcode (i.e. 0 -> +, 13 -> +=) Otherwise there is no difference between the two. Or what do you mean?

3

INPLACE_* and BINARY_* were replaced by BINARY_OP in Python 3.11:

You can still distinguish between the two in the output of dis by looking at the arguments in the right hand column:

$ python3.12 -m dis <<< "a + 1"
  0           0 RESUME                   0

  1           2 LOAD_NAME                0 (a)
              4 LOAD_CONST               0 (1)
              6 BINARY_OP                0 (+)    # <== note: 0 (+) 
             10 POP_TOP
             12 RETURN_CONST             1 (None)

$ python3.12 -m dis <<< "a += 1"
  0           0 RESUME                   0

  1           2 LOAD_NAME                0 (a)
              4 LOAD_CONST               0 (1)
              6 BINARY_OP               13 (+=)    # <== note: 13 (+=)
             10 STORE_NAME               0 (a)
             12 RETURN_CONST             1 (None)
3 Likes
4

I mean the two operations of + and += are very different from each other, the different opcode did indicate this difference.

def test_add():
    l = [1, 2]
    print(id(l))
    l = l + [3, 4]
    print(id(l))


def test_inplace_add():
    l = [1, 2]
    print(id(l))
    l += [3, 4]
    print(id(l))


if __name__ == "__main__":
    # import dis
    print("--- test for __add__ ---")
    test_add()

    print("--- test for __iadd__ ---")
    test_inplace_add()

then the results respectively

--- test for __add__ ---
4543449408
4543807232
--- test for __iadd__ ---
4543807232
4543807232

Now they are scaffolded into BINARY_ADD. the difference was weaken.

5

Thanks for the detailed response.

I knew the different opcode(0 for +/13 for +=). Like I said above, the scaffolding may weaken the difference between add and inplace add.

I noticed these dramatical changes in ceval.c, the huge switch was moved into bytecodes.c, and many opcodes was grouped for better performance.

In terms of BINARY_OP:

    switch (opcode) {        
        ...
        family(BINARY_OP, INLINE_CACHE_ENTRIES_BINARY_OP) = {
            BINARY_OP_MULTIPLY_INT,
            BINARY_OP_ADD_INT,
            BINARY_OP_SUBTRACT_INT,
            BINARY_OP_MULTIPLY_FLOAT,
            BINARY_OP_ADD_FLOAT,
            BINARY_OP_SUBTRACT_FLOAT,
            BINARY_OP_ADD_UNICODE,
            // BINARY_OP_INPLACE_ADD_UNICODE,  // See comments at that opcode.
        };
        ...
        op(_BINARY_OP_INPLACE_ADD_UNICODE, (left, right --)) {
            _Py_CODEUNIT true_next = next_instr[INLINE_CACHE_ENTRIES_BINARY_OP];
            assert(true_next.op.code == STORE_FAST);
            PyObject **target_local = &GETLOCAL(true_next.op.arg);
            DEOPT_IF(*target_local != left, BINARY_OP);
            STAT_INC(BINARY_OP, hit);
            /* Handle `left = left + right` or `left += right` for str.
             *
             * When possible, extend `left` in place rather than
             * allocating a new PyUnicodeObject. This attempts to avoid
             * quadratic behavior when one neglects to use str.join().
             *
             * If `left` has only two references remaining (one from
             * the stack, one in the locals), DECREFing `left` leaves
             * only the locals reference, so PyUnicode_Append knows
             * that the string is safe to mutate.
             */
            assert(Py_REFCNT(left) >= 2);
            _Py_DECREF_NO_DEALLOC(left);
            PyUnicode_Append(target_local, right);
            _Py_DECREF_SPECIALIZED(right, _PyUnicode_ExactDealloc);
            ERROR_IF(*target_local == NULL, error);
            // The STORE_FAST is already done.
            SKIP_OVER(INLINE_CACHE_ENTRIES_BINARY_OP + 1);
        }
        ...
    }

I propose to add a switch like show_family=True to show the family detail like BINARY_OP, just like the show_caches=True did.

dis.dis("a += b", show_family=True)
6

The opcode is BINARY_OP, the argument is 0 or 13. The dis module should not display anything different here, it should displaying exactly what is being executed.

1 Like
7

Thanks for the response.

Like I said above, turning the (keyword arguments) show_caches or adaptive of dis.dis() on, there may be some hidden opcodes displayed.

That’s exactly what I mean: a new option for displaying the grouped/hidden opcode.

… it should displaying exactly what is being executed.

Partially agree. In my opinion, BINARY_OP is not the actual opcode, member n of _PyEval_BinaryOps is.

Ref:

  • _PyEval_BinaryOps in file ceval.c
const binaryfunc _PyEval_BinaryOps[] = {
    [NB_ADD] = PyNumber_Add,
    ...
    [NB_INPLACE_ADD] = PyNumber_InPlaceAdd,
    ...
};
  • and the opcode.h
#define NB_ADD                                   0
...
#define NB_INPLACE_ADD                          13
...
8

Ok, but then you are wrong. The term “opcode” has a clear and well defined meaning here. For example, check the members of opcode.opmap. IMO it would be very confusing for dis.dis to display stuff that isn’t the opcode in the slot where the opcode normally goes, even if it’s behind an option. Nothing stops you from implementing your own display code for the opcode lists.

2 Likes
9

opcode.opmap

You convinced me, I tried and “NB_INPLACE_ADD” is not a valid key of opmap.