Anybody got any idea what's going on with this bizarre and nasty slowdown?

Python Help
1

I’m getting the hang of Python at least reasonably well, but I’ve bumped into something that’s over my head…

To fully tinker with this mess I’ve gotten myself into, you’ll need to hit pypi and grab John Zelle’s “Graphics.py” package. I’m also using Numpy, so that’ll be needed, too, if you don’t already have it on your rig.

Two files of code, neither very large, but nicely demonstrates the headache I’m having.

First a bit of support code… Fiddle with the window height/width as needed for your rig

"""
GraphicsWindow.py
Handle setting up the graphics window we'll be drawing into.
""" 
import graphics # John Zelle's graphics.py package from <https://pypi.org/project/graphics.py/>

def drawAxisLines(win, h=0,v=0):
    tempv = v/2
    temph = h/2
    negatedv = tempv*(-1)
    negatedh = temph*(-1)
    
    theVertical = graphics.Line(graphics.Point(0,negatedv), graphics.Point(0,tempv))
    theVertical.setOutline('blue')
    theVertical.setFill('blue')
    theVertical.draw(win)
    
    theHorizon = graphics.Line(graphics.Point(temph, 0), graphics.Point(negatedh, 0))
    theHorizon.setOutline('red')
    theHorizon.setFill('red')
    theHorizon.draw(win)

def MakeWindow(width=2560, height=1440): # Default to main screen size if not specified
    # Size of the window we'll be drawing into
    Hor = width
    Ver = height
    # Precalculate the offset needed to center the origin
    HOff = Hor/2
    VOff = Ver/2
    # Create the window using the dimensions given above, naming it "Window".
    win = graphics.GraphWin("Window", Hor, Ver)
    # Set the background to white
    win.setBackground("white")
    # Origin defaults to being in the lower left corner, but we want origin at center of window, so we offset it
    # appropriately based on the input dimensions
    win.setCoords(-HOff, -VOff, HOff, VOff)
    # Give us a set of axes (as well as making the center of the window fairly obvious)
    drawAxisLines(win,Hor,Ver)
    return win

And now, the “main course”:

import numpy as np          # The Numpy math package <www.numpy.org>
import graphics as g        # John Zelle's graphics.py package <https://pypi.org/project/graphics.py/>
import GraphicsWindow as gw # Set up my customized output window for the graphics.py package.
import datetime as dt       # Used to time execution

def PlotPoint(atLocation=(0,0), inColor="black", scaleFactor=10): # Remember to default to sane input values...
    thePoint = g.Point(atLocation[0]*scaleFactor,atLocation[1]*scaleFactor)
    thePoint.setFill(inColor)
    thePoint.setOutline(inColor)
    thePoint.draw(theWindow)

def CalculateXY(forRadius=0, andAngle=0, centeredAt=(0,0)):
    X=forRadius*np.cos(andAngle)+centeredAt[0]
    Y=forRadius*np.sin(andAngle)+centeredAt[1]
    return (X,Y)

def PlotCircle(atCenter=(0,0), withRadius=5, inColor="black", Draw=True):
    for Theta in ThetaList:
        returnedPoint=CalculateXY(withRadius, Theta, atCenter)
        if Draw==True:
            PlotPoint(returnedPoint, inColor)

theWindow = gw.MakeWindow()

# Higher Resolution value = better circle approximation, but takes longer to calculate/draw and can massively
# bloat the results list I hope to collect. We'll go with 1 step = 1 degree for now.
Resolution=360

"""
Build ThetaList as a global so anybody interested can get to it easily - once built, it should never need
changing unless Resolution is changed. Otherwise, treat it as a read-only global.
"""
ThetaList=np.linspace(0,360*(np.pi/180),Resolution)

Z1=6
Z2=Z1+1

Or=50

OCenterX=0
OCenterY=0
OCenter = (OCenterX,OCenterY)

OCircumf=2*np.pi*Or


# Plot the base circle
radius=Or
start = dt.datetime.now()
PlotCircle(OCenter ,radius, "black", True)
finish = dt.datetime.now()
print("Base circle plotted in "+str(finish-start))

Ore=((OCircumf/Z2)/np.pi/2)/3 # Calculate e-generator radius that will accomplish closure
radius2 = Ore
workingRadius=radius+radius2
count=0
start2 = dt.datetime.now() # Start the timer on the full set of 360 secondary circles
# Where will the center of the current secondary circle be when angle=Theta?
for Theta in ThetaList:
    atSecondaryCircleCenter=CalculateXY(workingRadius, Theta, (OCenterX,OCenterY))
    # Plot this secondary circle
    start3=dt.datetime.now()
    PlotCircle(atSecondaryCircleCenter, radius2, "green", True)
    finish3=dt.datetime.now()
    print("Secondary circle #"+str(count)+"(Theta="+str(Theta)+") plotted in "+str(finish3-start3))
    count = count+1
    # NEXT!

finish2 = dt.datetime.now()
print("360 secondary circles plotted in "+str(finish2-start2))

Now, the two files above (with help from John Zelle’s Graphics.py, and Numpy) do EXACTLY what’s intended - Namely, draw a “base circle” in black, and surround it with 360 smaller green circles, each one centered on a radius of the base circle, and tangent to the base circle.

A partial run looks like this:
It’s exactly as expected, when you allow for the fact that it was interrupted before it completed - if it had completed, the green circles would totally surround the base circle - but would that be within my lifetime?!?!?!?

Screen Shot 2024-01-29 at 12.38.15 AM
Screen Shot 2024-01-29 at 12.38.15 AM800×768 6.08 KB

That’s not the issue - it IS working, and producing the correct results, it’s just taking forever to happen.

Here’s the “how long is it taking?” output from the run that produced the image above - after running for roughly half an hour, maybe 45 minutes:

Base circle plotted in 0:00:03.009240
Secondary circle #0(Theta=0.0) plotted in 0:00:05.397818
Secondary circle #1(Theta=0.01750190893364787) plotted in 0:00:06.001109
Secondary circle #2(Theta=0.03500381786729574) plotted in 0:00:05.211121
Secondary circle #3(Theta=0.05250572680094361) plotted in 0:00:06.006330
Secondary circle #4(Theta=0.07000763573459148) plotted in 0:00:06.069943
Secondary circle #5(Theta=0.08750954466823935) plotted in 0:00:06.001568
Secondary circle #6(Theta=0.10501145360188723) plotted in 0:00:06.001786
Secondary circle #7(Theta=0.1225133625355351) plotted in 0:00:06.984882
Secondary circle #8(Theta=0.14001527146918297) plotted in 0:00:08.784452
Secondary circle #9(Theta=0.15751718040283083) plotted in 0:00:10.118885
Secondary circle #10(Theta=0.1750190893364787) plotted in 0:00:09.911766
Secondary circle #11(Theta=0.1925209982701266) plotted in 0:00:11.011903
Secondary circle #12(Theta=0.21002290720377445) plotted in 0:00:11.609469
Secondary circle #13(Theta=0.22752481613742231) plotted in 0:00:12.007553
Secondary circle #14(Theta=0.2450267250710702) plotted in 0:00:12.032156
Secondary circle #15(Theta=0.26252863400471804) plotted in 0:00:12.004355
Secondary circle #16(Theta=0.28003054293836593) plotted in 0:00:12.020168
Secondary circle #17(Theta=0.2975324518720138) plotted in 0:00:12.715211
Secondary circle #18(Theta=0.31503436080566166) plotted in 0:00:14.619337
Secondary circle #19(Theta=0.33253626973930955) plotted in 0:00:15.966642
Secondary circle #20(Theta=0.3500381786729574) plotted in 0:00:15.666782
Secondary circle #21(Theta=0.3675400876066053) plotted in 0:00:16.498355
Secondary circle #22(Theta=0.3850419965402532) plotted in 0:00:17.682649
Secondary circle #23(Theta=0.402543905473901) plotted in 0:00:18.006286
Secondary circle #24(Theta=0.4200458144075489) plotted in 0:00:18.001633
Secondary circle #25(Theta=0.4375477233411968) plotted in 0:00:18.001407
Secondary circle #26(Theta=0.45504963227484463) plotted in 0:00:18.018168
Secondary circle #27(Theta=0.4725515412084925) plotted in 0:00:18.403419
Secondary circle #28(Theta=0.4900534501421404) plotted in 0:00:20.749675
Secondary circle #29(Theta=0.5075553590757883) plotted in 0:00:21.617934
Secondary circle #30(Theta=0.5250572680094361) plotted in 0:00:22.156216
Secondary circle #31(Theta=0.542559176943084) plotted in 0:00:22.786201
Secondary circle #32(Theta=0.5600610858767319) plotted in 0:00:23.146197
Secondary circle #33(Theta=0.5775629948103798) plotted in 0:00:23.911411
Secondary circle #34(Theta=0.5950649037440277) plotted in 0:00:24.002968
Secondary circle #35(Theta=0.6125668126776754) plotted in 0:00:24.053116
Secondary circle #36(Theta=0.6300687216113233) plotted in 0:00:24.023977
Secondary circle #37(Theta=0.6475706305449712) plotted in 0:00:25.020135
Secondary circle #38(Theta=0.6650725394786191) plotted in 0:00:25.943358
Secondary circle #39(Theta=0.682574448412267) plotted in 0:00:27.619222
Secondary circle #40(Theta=0.7000763573459148) plotted in 0:00:27.972040
Secondary circle #41(Theta=0.7175782662795627) plotted in 0:00:28.826091
Secondary circle #42(Theta=0.7350801752132106) plotted in 0:00:29.301352
Secondary circle #43(Theta=0.7525820841468585) plotted in 0:00:29.484138
Secondary circle #44(Theta=0.7700839930805063) plotted in 0:00:30.284609
Secondary circle #45(Theta=0.7875859020141542) plotted in 0:00:30.031661
Secondary circle #46(Theta=0.805087810947802) plotted in 0:00:30.054057
Secondary circle #47(Theta=0.8225897198814499) plotted in 0:00:30.331511
Secondary circle #48(Theta=0.8400916288150978) plotted in 0:00:31.485802
Secondary circle #49(Theta=0.8575935377487457) plotted in 0:00:32.684741
Secondary circle #50(Theta=0.8750954466823936) plotted in 0:00:33.616829
Secondary circle #51(Theta=0.8925973556160414) plotted in 0:00:34.467434
Secondary circle #52(Theta=0.9100992645496893) plotted in 0:00:34.701639
Secondary circle #53(Theta=0.9276011734833371) plotted in 0:00:35.328472
Secondary circle #54(Theta=0.945103082416985) plotted in 0:00:35.895306
Secondary circle #55(Theta=0.9626049913506329) plotted in 0:00:35.995629
Secondary circle #56(Theta=0.9801069002842808) plotted in 0:00:36.049598
Secondary circle #57(Theta=0.9976088092179286) plotted in 0:00:36.364255
Secondary circle #58(Theta=1.0151107181515766) plotted in 0:00:37.305495
Secondary circle #59(Theta=1.0326126270852243) plotted in 0:00:38.331148
Secondary circle #60(Theta=1.0501145360188722) plotted in 0:00:39.428820
Secondary circle #61(Theta=1.06761644495252) plotted in 0:00:39.955416
Secondary circle #62(Theta=1.085118353886168) plotted in 0:00:40.486661
Secondary circle #63(Theta=1.1026202628198158) plotted in 0:00:41.210054
Secondary circle #64(Theta=1.1201221717534637) plotted in 0:00:41.782836
Secondary circle #65(Theta=1.1376240806871116) plotted in 0:00:42.002347
Secondary circle #66(Theta=1.1551259896207595) plotted in 0:00:42.051050
Secondary circle #67(Theta=1.1726278985544074) plotted in 0:00:42.106717
Secondary circle #68(Theta=1.1901298074880553) plotted in 0:00:43.954910
graphics.GraphicsError: Can't draw to closed window 

'Cause I closed the window to deliberately crash it out.

Notice how the time to plot each circle just keeps rising? This is a consistent result after multiple trial runs. The exact numbers in the “plotted in” field vary a bit from run to run, but they show the same pattern across runs: Every time a circle is plotted, it takes longer, until I finally give up and kill it. The longest I’ve let it run (I’m not sure how long that was - several hours, anyway) got it to somewhere in the neighborhood of circle #175 +/-5. At that point, it was taking 2:46 (Yep, you read that correctly -nearly 3 full minutes, based on watching the clock and waiting for the next dot to appear) to calculate and plot EACH DOT of each circle!

Can anybody point me at where this steadily increasing time-per-dot phenomenon comes from, and/or suggest a way to avoid it? Obviously, at the start of the run, the times are reasonable, but it doesn’t take too long to get to the point where it’s pretty obvious that it’s choking on SOMETHING, but I’m too rookie in Python to figure out where/why/how to fix it. It’s not like I’m doing something that leads to runaway recursion, or something like that.

2

You are somewhat misusing the graphis.py library. You are not supposed to constantly create and draw GraphicsObject, the all continue to live on as long as the corresponding canvas is alive. So you are peremantly increasing the number of objects attached to the canvas, all of which need to be refreshed on each point.draw() call, which obviously takes more and more time. A better library for your usage pattern is pygame. Alternatoively, as a somewhat fix you can disable the autoflush of the GraphWin instance, but then you will have to call win.flush() at some point.

3

Additionally, you are iterating over ThetaList to call PlotCircle but then iterating over ThetaList again a second time inside PlotCircle, which seems at least a little suspect at a glance.

for Theta in ThetaList:
    PlotCircle(atSecondaryCircleCenter, radius2, "green", True)

but then also

def PlotCircle(atCenter=(0,0), withRadius=5, inColor="black", Draw=True):
    for Theta in ThetaList:
4

That part makes sense–at each point on the large circle, they are plotting a smaller circle (by plotting all the points).

I suspect the slowdown is because of exploding memory usage (the growing number of objects to store and draw), I’d watch it with htop for a bit to see what it’s doing.

5

In addition to what others have said, in PlotCircle you’re calculating returnedPoint and then deciding whether to use the result. You’re not going to use it, why calculate it?

Also, you calculate the angles once, and then calculate their cosines and sines repeatedly.

6

OK but drawing 129600 “dots” arranged into the outlines of 360 smaller circles is going to be somewhat less efficient than just drawing 360 actual small circles directly—is that an intentional choice or requirement?

7

I have no idea, I’ve never used it before.

There’s two classes of issue going on in this script–major and minor. There are a lot of little “this could be done more efficiently” issues that might come up in a review for a PR, but they are not the issues that caused the exponential growth in runtime. If the exponential issue was solved, this script takes a few minutes. At that point, it’s worth fixing the rest.

8

Will look into this “pygame” you speak of, thanks.

The points that get calculated (not the actual image produced from them, though those will be wonky enough to be immediately obvious if I’m doing the underlying math wrong) are what’s important in this case - I’m using the Zelle library primarily to show them so that I know that my math is correct - A graphical presentation that’s more or less equivalent to doing a bunch of print() statements to show the values, but easier to spot “oops” cases than scrolling through line after line after line of X,Y pairs.

Meanwhile, if you (or whoever else) know of a dead-simple, bare-bones graphics package that works along the lines of

import NameOfPackage as g

TargetWindow = g.MakeWindow(XDimension, YDimension, BackgroundColor)
TargetWindow.PlotPoint(X, Y, Color)

I’d ABSOLUTELY LOVE to hear about it…

(Yeah, I know - I’m dreaming - Or at least, based on what I’ve found so far, I MUST be dreaming. Such a package appears to be made of 100% pure Unobtanium.)

9

Yep, and I took steps (or so I believe) to avoid problems due to re-use of ThetaList - It gets constructed exactly once, then treated as read-only. It doesn’t (or at least shouldn’t, by my understanding) matter who, or how many whos, want to iterate over it at any given time, since nobody is (supposed to be, at least…) doing anything but reading from it.

The part you seem to be taking issue with is part of a loop iterating over ThetaList - It first determines "Where is the center of the new circle for supposed to be? At X=Radius * cos(Theta), Y=Radius * sin(Theta). It then uses that X,Y pair as the center, and actually plots the circle, also by iterating over ThetaList. I can 't see any way (unless something I don’t know about is sneaking in and messing with it) ThetaList being used by both steps has any “downside”. If you can, PLEASE point it out, and be specific - I’m NO KIND of Python expert, so I’ll agree that there may be some sort of “side-effect” I’m not aware of.

10

Pygame is a decent library, but I assume graphics.py is also ok. My first thought when looking at your code is: Why are you laboriously plotting your circles with “PlotCircle”? Doesn’t graphics.py have a built-in for drawing circles? (Yes, it has - is this not usable?)

Second thought - why are you recalculating the points of the little green circles so many times? Those circles, if I understand correctly, are all copies of each other, so you only need to calculate those points once and then copy over the points to the right locations…

(Actually, looking at Pypi for graphics.py, I now have my doubts about that package. No documentation at all. No readme. No reference to github. Those are all red flags to me.)

11

Exactly.

I’d be strongly inclined to agree with you. It’s really the only thing that makes any sense, and I’m suspecting that using a graphics lib other than Zelle may very well cause the problem to vanish. (I’ve never considered Zelle to be anything but a “minimal functionality” item - and that’s exactly what I want - although not at the cost of having it bog down like what I’ve got happening.)

12

I think @MegaIng identified the issue–you are adding more and more objects and drawing them over and over again. It would be better to create fewer objects (e.g. by defining circles rather than lots of points), but it sounds like the individual points are the reason for this exercise. It would also be better to draw only at the end, after all the objects have been placed, rather than re-drawing the window after every point. In other words, draw(theWindow) only at the very end.

I don’t know which of those is the bigger problem.

13

But you are exactly wrong with this perception. graphics.py does not give you primitive drawing tools, it gives you high level objects that you can move around afterwards and that are tracted idnvidually. What you want is something closer to pygame, which does just give you pygame.draw.circle which just updates the pixels in the passed in surfaces.

p5.py might also do what you want with a slightly simpler interface than pygame.

14

It does have such functionality, but I’m not interested in the circles at all, except as indicators that my math is/isn’t correct - I’m interested in the points they’re made up of, because those points are (eventually, after some more programmatic massaging) going to be exported for use as the input data for a CAD program to use in drawing the profile of a complex shape. Complex enough that the one hand-drawn version of it I came up with took me more than 15 hours of work, only to realize that one of the inputs should have been a bit larger than it was, so the time had been completely wasted. The tiniest change in any of the half-dozen or so input parameters that controls it means throwing away the entire existing drawing (Well, there’s a center point that can be salvaged, but everything else instantly turns into trash) and starting over from scratch because of how they interact - Changing one parameter by half a millimeter can make multi-millimeter changes to several other things, in a shape that needs to be accurate to 0.005mm (or better) if it’s to be anything other than a labor-intensive piece of junk that might do anything from “absolutely nothing”, to a full-on “CRUD” - Catastrophic Rapid Unplanned Disassembly - once it goes from being a drawing to a physical object.

15

Not sure what the general context is, but you could also use matplotlib to do these kind of drawings:

import matplotlib.pyplot as plt
import numpy as np

def plot_circle(radius, num_points):
    fig, ax = plt.subplots()

    # Big circle
    circle = plt.Circle((0, 0), radius, fill=False, edgecolor='b', linestyle='dashed')
    ax.add_patch(circle)

    theta = np.linspace(0, 2*np.pi, num_points)
    x = radius * np.cos(theta)
    y = radius * np.sin(theta)

    # Little green circles
    ax.scatter(x, y, color='white', s=800, alpha=0.4, edgecolors='green')
    
    ax.set_aspect('equal', adjustable='box')
    plt.show()

radius = 10
num_points = 100
plot_circle(radius, num_points)
16

And if you need to actually plot the points, storing them all in a couple big lists of x and y values, and plotting with ax.scatter, would be much more performant.

edit: I missed that your code actually is doing the scatter, for the little circles. The same things works for the big circle.

17

Flexibility - At one point, I was ALWAYS going to use it, but then ran into a condition where using it becomes optional, so added the “optionality”.

I don’t follow? I calculate the angles (assuming you’re speaking of ThetaList) once, to be used repeatedly - Each time the cosines/sines are calculated, they’re used for something else, usually, but not always, with a different radius being used in the calculation.

18

cos(thetalist[0]) is the same every time but you compute it again for each small circle. You could calculate the cosine for each angle and the sin for each angle once, and store them. This is a micro-optimization though–it’s not going to speed things up noticeably.

19

For each theta in theta list you are recalculating all the sin and cos of all the points in the little circles. That’s a waste of time, since you only need to calculate the points of one little circle (at an arbitrary position) once, and then you can re-use those points (after a simple linear shift) for all those little circles.

20

Actually @hansgeunsmeyer is being sneakier, he’s plotting 100 big white circles with green edges, instead of thousands of tiny dots. If the goal really is to have the precise coordinates for CAD, then I would plot them individually as dots:

import numpy as np
import matplotlib.pyplot as plt

def plot_circle(big_radius, num_big_points, lil_radius, num_lil_points):
    fig, ax = plt.subplots()

    # big circle
    big_theta = np.linspace(0, 2*np.pi, num_big_points)
    
    x = big_radius * np.cos(big_theta)
    y = big_radius * np.sin(big_theta)

    ax.scatter(x, y, color="black", s=1)

    # lil circle
    lil_theta = np.linspace(0, 2*np.pi, num_lil_points)
    
    # use numpy broadcasting to make all pairs of big_x + lil_x, big_y + lil_y
    lil_x = (x[:,None] + (lil_radius * np.cos(lil_theta))).flatten()
    lil_y = (y[:,None] + (lil_radius * np.sin(lil_theta))).flatten()

    ax.scatter(lil_x, lil_y, color="green", s=0.1)
    
    ax.set_aspect('equal', adjustable='box')
    plt.show()

big_radius = 50
num_big_points = 100
lil_radius = 5
num_lil_points = 1000
plot_circle(radius, num_points, lil_radius, num_lil_points)

image