Embedding: how do I extend sys.path?

Python Help
1

Hi,
I want to embed a Python interpreter into a game, extending sys.path so that I can use the game-specific files from the game’s directories, instead of installing them into the system’s (or the user’s) Python environment.

My current (admittedly somewhat hacky) code looks like this:

void _py(std::string run_this)
    {
        // TODO use a subinterpreter with py3.12+
        PyStatus status;

        std::wstring prog_name = L"openttd::Python";
        std::wstring run_name = multiByteToWide(run_this);

        PyConfig cfg;
        PyConfig_InitIsolatedConfig(&cfg);
        cfg.bytes_warning=1;
        cfg.faulthandler=1;
        cfg.use_environment=1;
        cfg.user_site_directory=1;
        cfg.module_search_paths_set = 0;

        std::string gd(GLOBAL_DATA_DIR);
        gd.append(PATHSEP "python");
        std::wstring wgd = multiByteToWide(gd);
        PyWideStringList_Append(&cfg.module_search_paths, wgd.c_str());

        std::string pd(PERSONAL_DIR);
        pd.append(PATHSEP "python");
        std::wstring wpd = multiByteToWide(pd);
        PyWideStringList_Append(&cfg.module_search_paths, wpd.c_str());

        status = PyConfig_SetString(&cfg, &cfg.program_name, prog_name.c_str());
        if (PyStatus_Exception(status))
            goto exc;
        status = PyConfig_SetString(&cfg, &cfg.run_filename, run_name.c_str());
        if (PyStatus_Exception(status))
            goto exc;

        cfg.quiet=1;
        Py_InitializeFromConfig(&cfg);
        Py_RunMain();
        Py_FinalizeEx();
    exc:
        PyConfig_Clear(&cfg);

        stopped = true;
    }

Now if cfg.module_search_paths_setis zero, the supplied module paths are ignored. If it’s one, the paths are used but the system paths are ignored. If I change the order and append my path elements after calling Py_InitializeFromConfig that has no effect.

How do I extend the default path?

2

Can you ship a sitecustomize.py or usercustomize.py, that just contain sys.path.insert(0, '...')?

Or include a .pth file (a “path configuration file”) somewhere effective?

3

If I wanted to (and/or could) install something in one of the system’s “official” Python’s directories I could just install the game-specific module itself, instead of a .pth file. For the same reason modifying a file in /etc/python* is not that good an idea.

4

If I wanted to (and/or could) install something in one of the system’s “official” Python’s directories I could just install the game-specific module itself, instead of a .pth file. For the same reason modifying a file in /etc/python* is not that good an idea.

Obviously.

If none of my suggestions work with embedded Python builds, that’s fine. I just thought it was easier and cleaner to use existing features of Python’s machinery, than writing C.

5

For one of my semi-ancient Python-based utilities I have this at the
top of the entry point script of the application (before any import
statements):

# added so distributors can consistently specify a private module location
private_module_path = "/usr/share/mypackage"
if private_module_path:
    import sys
    sys.path.insert(1, private_module_path)
6

You can try checking out pocketpy. It’s a small implementation of python meant to act as an alternative to Lua for game scripting

7

Unfortunately pocketpy lacks a bunch of features I need (threading and/or async among them).

I can work around this issue with a three-liner (pass the intended paths in via argv) until the problem is fixed.

8

Sorry if you’ve considered this already and I’m missing something obvious. Can you use setenv and put your module paths in environment variable PYTHONPATH?