Exec() with 'return' keyword

Python Help
41

Therefore:

  • Separating data from code is inherently beneficial, even though it adds complexity to your code.
  • High level languages, with easy-to-use data structures which can stand conveniently independent of your application code, are the easiest way to build this kind of separation.
  • HERESY ALERT: The classic model of classes and objects (encapsulating data and code within an atomic unit) is inherently bad. END HERESY, SAFE TO CONTINUE READING.

For my Twitch channel bot, which supports full hot-reloading, the basic model is: a core event loop that dispatches to “this named function in the current version of this module”, allowing modules to be reloaded at will. Every function/method call is given a parameter which contains its most important state, stored in a plain data structure (eg a dictionary - not a custom class), and there’s room for globally shared state too. Every function is built to regularly return control to the main event loop.

There IS extra complexity to doing this. You have to think about your code and figure out how to divide it up so that it can return control to the main loop frequently (sometimes that’s trivially easy, sometimes not so much), and you have to plan your data structures and then maintain backward compatibility as you update the code. But the payoff is spectacular: full reloading capability at module granularity, with open files (especially sockets) being retained from one version of the code to another.

42

Found hy suitable for my tweaks. Decide to do some serious modification.

43

Hi James,

To achieve this kind of thing I think you need to set a trace and watch for exception events like ‘bdb.trace_dispatch’. It’s usually 2x-3x slower, which I suspect is unacceptable for data scientists.

44

Adding a breakpoint under “trace_dispatch” where exceptions are detected acts similar to post-mortem debugging, though it may provide more information. It does not allow continuing execution to next line with reloaded function and preserved context. Also it will capture every exception, even if it is wrapped inside try...except....

I think it may require some bytecode hacks when taking this bdb approach.

Talking about “continue execution”, it is not just about not exiting the current process, but also about not re-executing statements which don’t raise exceptions. You need to continue execution exactly at the line where the exception is raised.

It’s not always necessary to follow the “continue at exactly the line where things go wrong” discipline, even my “reloading” decorator cannot ensure that. But it does require to set boundaries between where every line of code must be wrapped inside “try-except” and where code are free to raise exceptions. A common boundary could be between unfinished code and production ready code, or between my code and others code.

Test code:

Save these two files under same directory, run python3 your_code.py and do whatever you can to fix all exceptions during runtime, also following rules mentioned above to minimize code re-execution.

(hint: you can only change these files during its execution, not before, not after)

# inside alien.py
raise Exception("you cannot import alien library, what you want to do?")

def alienFunction():
    raise Exception("you should not pause at this line, because this is alien")
    print("you can only reload this function after modification after you captured this exception in your code")
    return "val alien"

# inside your_code.py

import bdb
bdb.set_trace()

raise Exception("you cannot even continue from the beginning!")

def function_0():
    raise Exception("exc 0")
    print("can you retrieve val_0?")
    return "val 0"

def function_1():
    try:
        raise Exception("you may not capture this exception")
        print("but you cannot execute this line either")
    except:
        print("exception is raised, but captured, by source code, nut by debugger!")
    raise Exception("you may not retrieve val_1")
    return "val 1"

val_0 = function_0()
print("do you have val_0?", val_0)
val_1 = function_1()
print("do you have val_1?", val_1)
import alien
val_alien = alien.alienFunction()
print("do you have val_alien?", val_alien)

Modification in bdb.py:

# inside trace_dispatch function:
...
        if event == 'return':
            return self.dispatch_return(frame, arg)
        if event == 'exception':
            breakpoint() # do fun things, but just like post-mortem debugging, for every exception instead of uncaught exception only
            return self.dispatch_exception(frame, arg)
        if event == 'c_call':
            return self.trace_dispatch
...
45

So… why not just make those boundaries into different functions? Because that works already, without all this messy magic.

46

“Messy” isn’t the proper word for this topic. “Seamless” is. I want to make these features “built-in” so all you have to do is to write normal code then execute, with all the features described above by transforming the code automatically. I modify the hy compiler and present you this. You act normal and write hy code, but whenever you feel like to turn off some safety features, you can play with its command line arguments.