Method to refresh os.environ

I’m now lost. Many people spoke up, but I’m lost in the number of propositions. To help me understand what the majority wants, I propose a vote. The first option of the poll is to remove the method. Sorry, you can only select a single name, but you can write a message with details (if you consider that previous messages didn’t include your point of view).

Vote for os.environ.refresh() method name
  • None! Remove the method!
  • invalidate_cache()
  • recreate_from_process()
  • refresh()
  • refresh_cache()
  • reload()
  • reload_cache()
  • reload_from_process()
  • synchronize()
  • synchronize_with_process()
0 voters

Method documentation:

The os.environ.refresh() method updates os.environ with changes to the environment made by os.putenv(), by os.unsetenv(), or made outside Python in the same process.

1 Like

Wasn’t it agreed this should happen at the Core Development category, voted by core devs? I don’t know the precudure but I got that from Steve’s message.

Also I don’t really understand including the method documentation in the poll. That is something that would be adjusted to the name so obviously it’d fit refresh() most.

1 Like

I copied the documentation to remind what we are talking about: explain what the method does.

Still seems a bit unnecessary to me as I’d expect anyone voting to have actually read the discussion before, but it’s okay since it doesn’t really matter that much.

It’ll be best to take a proposal there, so being able to say “ideas liked … best” is a good plan. (Doesn’t mean that this vote will carry any weight when it gets there, but showing up with 20 options isn’t going to help at all.)

To be clear, the thread safety issues that @yurivict was worried about aren’t about race conditions between multiple python threads, but between a python thread and foreign threads (that may call the underlying setenv/putenv C functions).

If somebody tries to call os.environ.refresh() in python, the interpreter will eventually access the global extern char **environ variable. If at the same moment some other non-python thread (e.g., a TCL thread or a C++ thread or a Java thread or …) happens to call setenv/putenv, that is undefined behaviour. Both the setenv/putenv and the extern char **environ reads might crash or return pointers to invalid memory or “summon nose demons”.

There is no general, robust way to guard against this, since that would require all of the different programming languages, standard libraries and implementations to agree on a single locking mechanism around extern char **environ and all of the downstream APIs.

Here is an article with more information about this issue:
https://www.evanjones.ca/setenv-is-not-thread-safe.html

And while there is no general way to avoid this issue, the documentation for os.environ.refresh should probably explicitly mention this problem. After all, if the developer is trying to call os.environ.refresh(), it is extremely likely that “foreign” execution contexts are expected to be modifying the current extern char **environ, so the user should be made aware, that it is only safe to refresh() if they can ensure that such modifications don’t happen concurrently with the refresh() call.

Edit: removed mentions of getenv since that’s a bit of a red herring

1 Like

Results of the poll: 25 persons (56%) prefer the method name “refresh()” and it’s the majority. Only 5 persons (11%) would prefer to see the method removed.

If someone knows how to explain this problem in a few words, please open a pull request to document the issue.

Note that voting results here are non-binding, as not only core developers can vote. This is just to convince the skeptics here. But if any core developer has serious objections, they can open this question in the Core Development section (as usually happens when core developers have differences and cannot resolve them among themselves).

1 Like

I’ll make the post (link will come here when posted)


2 Likes