FR: Allow private runtime config to enable extending without breaking the `PyConfig` ABI

Core Development
20

Ok, and now something completely different: PyInitConfig API.

I propose a new API made of only 11 functions to configure the Python initialization. It is based on previously discussed ideas:

  • Opaque structure
  • Allocate memory on the heap
  • NEW! Use a string to identify a PyConfig member.

IMO using a string to identify a PyConfig member is convenient than PyTypeObject integer slot such as Py_tp_del (53). But the most important thing is that the list of members is not part of the API, so members can be add and removed without changing the API. Moreover, the type of a member can change in a future Python version.

Add PyInitConfig functions:

  • PyInitConfig_Python_New() – caller must call PyInitConfig_Free() once done
  • PyInitConfig_Isolated_New() – caller must call PyInitConfig_Free() once done
  • PyInitConfig_Free(config)
  • PyInitConfig_SetInt(config, key, value) – value is a int64_t
  • PyInitConfig_SetStr(config, key, const char* value) – bytes string
  • PyInitConfig_SetStrList(config, key, length, items) – bytes strings (ex: argv)
  • PyInitConfig_SetWStr(config, key, value) – wide string
  • PyInitConfig_SetWStrList(config, key, length, items) – wide strings (ex: xoptions)
  • char* PyInitConfig_GetErrorMsg(config) – caller must call free() once done

Add also functions using it:

  • Py_InitializeFromInitConfig(config)
  • Py_ExitWithInitConfig(config)

See the PR for the exact API.

Long example showing usage of most APIs:

static int test_initconfig_api(void)
{
    PyInitConfig *config = PyInitConfig_Python_New();
    if (config == NULL) {
        printf("Init allocation error\n");
        return 1;
    }

    if (PyInitConfig_SetInt(config, "dev_mode", 1) < 0) {
        goto error;
    }

    // Set a list of wide strings (argv)
    wchar_t *argv[] = {PROGRAM_NAME, L"-c", L"pass"};
    if (PyInitConfig_SetWStrList(config, "argv",
                                 Py_ARRAY_LENGTH(argv), argv) < 0) {
        goto error;
    }

    if (PyInitConfig_SetInt(config, "hash_seed", 10) < 0) {
        goto error;
    }

    // Set a wide string (program_name)
    if (PyInitConfig_SetWStr(config, "program_name", PROGRAM_NAME) < 0) {
        goto error;
    }

    // Set a bytes string (pycache_prefix)
    if (PyInitConfig_SetStr(config, "pycache_prefix",
                            "conf_pycache_prefix") < 0) {
        goto error;
    }

    // Set a list of bytes strings (xoptions)
    char* xoptions[] = {"faulthandler"};
    if (PyInitConfig_SetStrList(config, "xoptions",
                                Py_ARRAY_LENGTH(xoptions), xoptions) < 0) {
        goto error;
    }


    if (Py_InitializeFromInitConfig(config) < 0) {
        Py_ExitWithInitConfig(config);
    }
    PyInitConfig_Free(config);

    dump_config();
    Py_Finalize();
    return 0;

error:
    printf("Init failed:\n");
    Py_ExitWithInitConfig(config);
}
1 Like