Python summing in loop vs sum() performance

Python Help
1

Hey everyone!

Recently I was playing with python. And noticed in my opinion strange behaviour.

I have 4 functions:

def t1():
    s = time.perf_counter_ns()
    counter = 0
    for _ in range(10_000_000):
        counter += 1
    e = time.perf_counter_ns()
    print(counter)
    return int((e - s) / 1_000_000)

def t2():
    s = time.perf_counter_ns()
    counter = sum(1 for _ in range(10_000_000)
    e = time.perf_counter_ns()
    print(counter)
    return int((e - s) / 1_000_000)

def t11():
    s = time.perf_counter_ns()
    counter = 0
    for i in range(10_000_000):
        counter += i
    e = time.perf_counter_ns()
    print(counter)
    return int((e - s) / 1_000_000)

def t22():
    s = time.perf_counter_ns()
    counter = sum(range(10_000_000))
    e = time.perf_counter_ns()
    print(counter)
    return int((e - s) / 1_000_000)

When I run t11 and t22 I get that time t11 > t22. But when I run t1 and t2 I get the opposite.

I tried to run it on different machines - Intel and AMD, both are Win11 with Python 3.11.0 and get the same result in both cases. Below is result for AMD run.
image

So, I just wonder why this is happening.

2

The comparisons are computationally not the same. Functions t1 and t11 directly iterate the range. In t22, sum directly iterates the range. In t2, sum iterates a generator that iterates the range, so there in effect twice as many next calls, which is slower. Change t2 to directly iterate 1 iterator, as t1 does, such as

def t2b():
    onelist = [1] * 10_000_000
    s = ...
    counter = sum(onelist)
    ....

and I strongly suspect that it will beat t1.

EDIT: spelling above, plus add … Or slow down t1 by doing the same double iteration:

def t1b():
    s = ...
    counter = 0
    for _ in (1 for i in range(10_000_000))
    ...
1 Like
3

Thank you for reply! I understand that summing a list as you proposed is faster, but it consumes more memory and I tried to avoid that.

I thought that compiler could optimise double generator to something similar to t22 :sweat_smile:

4

See edit for a way to do a fair comparison without the list.