Reducing repetition when using import

Python Help
1

current scenario -

from a.b.c import d
from a.b.e import f
from a.b.g import h

the a.b part keeps repeating

  1. one way to reduce this repetition is,
dct = {'c': 'd', 'e': 'f', 'g': 'h'}
[exec("from a.b.{key} import {value}") for key, value in dct.items()]

but it requires using exec, I am not sure if this is a recommended way to achieve the same

expected scenario -

  1. if there is someway to achieve this without using exec then inform me
  2. if not, then I think there should be some way to reduce this repetition
2

There are certainly better ways to import things than using exec, I’d check out importlib.

There are times when a dynamic import may be required, and something like importlib is usually the way to go for such cases. With that said, if we are talking about normal importing, I would just stick with the usual repetitive from ... import ... approach.

More compact code isn’t necessarily “better”. I personally find the second example more convoluted. The second example also isn’t more efficient, it is just more compact. I don’t think you really gain anything from reducing the repetition except less repetition. On the other hand, you lose readability, you incur a higher maintenance burden, and you import in a less efficient manner.

1 Like
3

I want to know should exec be completely avoided, or is it only if it asks for user input that it should be avoided, I dont see what could be the issue with,

exec("import fastapi")

or the op exec example.

there could be issue with,

exec('input()')

or any command which requires user input.

4

There is not a fundamental issue per se. I think some people would have strong feelings about the approach, but I guess those could be argued as subjective.

5

It should be completely avoided if you do not have extremely good control over the content to be executed. So it is not just user input but also API input, configuration files etc. Practically any data which could come from outside of the application should not make a part of code to be executed or you have to sanitize them extremely thoroughly.

…but that is not all. There are other problems with exec() (and similar):

  • broken linting
  • broken type checking
  • boken syntax highlighting
  • complications in debugging and error handling

I think you got the idea.

I am not sure where you see problems with that. Problematic is when the external data make part of the code to be executed.

6

So what? Why do you care? This is harmless repetition. It is not just harmless, it is good repetition – it repeats the part of the code which has to repeat. Avoiding that natural repetition requires the most unnatural, complicated and inefficient code.

If you find that in practice this is painful, that’s probably a sign that your package is badly designed, with too many levels, and you should refactor it to something like:

from a import d, f, h

Yeah, that’s awful code. It is worse than the original in every way except one very minor advantage:

  • Save one line of code (2 lines instead of 3).

And even that advantage is only because you have artificially short names: “a”, “b”, “c” etc instead of realistic meaningful names. With more realistic names, you possibly will end up exceeding the recommended 79 char line length and will split one or both of the lines.

Compared to that, you haven’t reduced the repetition, you have just disguised it. In addition, the second version:

  • Is harder to read and understand.
  • Is less natural and straight-forward code.
  • Is less efficient and slower code.
  • It wastefully creates a list filled with None that has to garbage collected.
  • It defeats code analysis tools like linters.
  • It defeats code completion and syntax highlighting.
  • It is longer to type: 105 characters compared to 57.
  • It is more fragile code: because it does more work, there are more things to go wrong.
  • And it is harder to maintain if there are future changes.

E.g. if you refactor module a.b.d into a.x.y.z your shortcut doesn’t work and has to be completely changed. You can’t insert logging or print between two of the imports.

2 Likes
7

For beginners to Python: never use exec.

For intermediate and expert users: almost never use exec.

There is never a good reason to write a call to exec like that, where the code is a fixed string literal.

It might be useful, sometimes, to build some source code dynamically, at runtime, and then exec it. But if you know what the source code is ahead of time, when you are writing the program, you should just write that code and not wrap it into a string literal and exec it.

The overhead of exec is considerable: it has to parse and byte-code compile the source code, at runtime. So it cannot take advantage of pre-compiled .pyc files.

Even for importing modules, which is an expensive operation, this overhead is considerable:

[steve ~]$ python3.10 -m timeit "import this"
The Zen of Python, by Tim Peters
...
2000000 loops, best of 5: 104 nsec per loop
[steve ~]$ python3.10 -m timeit "exec('import this')"
The Zen of Python, by Tim Peters
...
50000 loops, best of 5: 7.46 usec per loop
3 Likes
8

Just as general note: PEP 8 - Style Guide for Python Code - Imports