Use `PC/pyconfig.h.in` instead of hard coding `PC/pyconfig.h`

Ideas
1

This idea product a new way to generate PC/pyconfig.h:

Now you have this c.in file.

#ifndef Py_CONFIG_H
#define Py_CONFIG_H

// ...
#undef HAVE_XXX
#undef HAVE_YYY
#undef SIZEOF_ZZZ

#endif

Then write a configure.toml:

[analyse.bool]
macro = "HAVE_XXX"
header = ["example1.h"]
code = """
int main() {
    xxx();
    return 0;
}
"""
store_value = "fail or not"

[analyse.bool]
macro = "HAVE_YYY"
header = ["example2.h"]
code = """
int main() {
    yyy();
    return 0;
}
"""
store_value = "fail or not"

[analyse.number]
macro = "SIZEOF_ZZZ"
header = ["stdio.h"]
code = """
int main() {
    printf("%d\n", sizeof(zzz));
    return 0;
}
"""
store_value = "output"

The configure.bat will analyse this toml file.

There will be three modes: bool, number and string

For bool, it will only store 1 or just keep the comment of /*#undef XXX*/.
For number, it will store the result of something (recommend “output”).
For string, the final macro value will be enclosed by double quotes (recommend “output”).

2

What tool processes the configure.toml?

Don’t you have a bootstrapping problem?
Don’t you require a working python to build a new python?

3

See build requirements on windows.

1a. Optionally install Python 3.10 or later. If not installed, get_externals.bat (via build.bat) will download and use Python via NuGet.

After this change the min-version changes to 3.11.

4

Instead of explaining a possible solution, can you start by describing the problem you’re trying to solve?

6 Likes
5

The question is that hard coding PC/pyconfig.h make maintaining difficult on windows branch. If it is possible to write better implemention, on windows it is usually be ignored due to the configure question.

After this change, it will be more maintainable on windows.

7

Can you elaborate why you consider that this specific header file PC/pyconfig.h is difficult to maintain? In the last 3 years, the file was modified 15 times (git log --after=2023-01-01 PC/pyconfig.h |grep ^commit -c): that’s a low number of changes, only 5 changes per year in average.

The file has 776 lines, whereas Lib/typing.py is closer to 4,000 lines and Objects/unicodeobject.c has more than 15,500 lines (knowing that the Unicode file was already splitted in 3 other sub-files!). For example, I consider that Objects/unicodeobject.c is difficult to maintain because it’s big and it has a long history (many changes).

8

The less changes is caused by that some questions wasn’t discovered.

For example, I found this question:

It seems to be that mode_t has been defined. But …

Also seems that it is wrong.

On my computer, in configure, it was found that it is not defined.

Moreover, GETPGRP_HAVE_ARGS is a typo of GETPGRP_HAVE_ARG.

So, “a low number of changes” doesn’t mean that it is easy to maintain. On another side, it may also be “so hard that nobody know how to edit”.

9

Does Python fail to build?

10

In fact, the building is successful. But it doesn’t mean that the code is healthy.

From _gen_toml.py, I see some literal (in original PC/pyconfig.h):

    # dead autoconf boilerplate not referenced by CPython (kept for shape parity)
    ("const", "false", "", "Define to empty if the keyword does not work. Always available."),
    ("_POSIX_1_SOURCE", "false", "", "Define if you need POSIX.1 features. Never on Windows."),
    ("_POSIX_SOURCE", "false", "", "Define if you need POSIX features. Never on Windows."),
    ("_ALL_SOURCE", "false", "!defined(_ALL_SOURCE)", "Define on AIX 3. Never on Windows."),
    ("DIRENT", "false", "", "Define if you have dirent.h. (legacy autoconf macro)"),
    ("NDIR", "false", "", "Define if you have ndir.h. (legacy autoconf macro)"),
    ("SYSDIR", "false", "", "Define if you have sys/dir.h. (legacy autoconf macro)"),
    ("SYSNDIR", "false", "", "Define if you have sys/ndir.h. (legacy autoconf macro)"),
    ("TIME_WITH_SYS_TIME", "false", "", "Define if both <time.h> and <sys/time.h> may be included. (unused)"),
    ("TM_IN_SYS_TIME", "false", "", "Define if struct tm is in <sys/time.h>. (unused)"),
    ("SYS_SELECT_WITH_SYS_TIME", "false", "", "Define if <sys/select.h> needs <sys/time.h>. (unused)"),
    ("VOID_CLOSEDIR", "false", "", "Define if closedir returns void. (unused)"),
    ("GETPGRP_HAVE_ARGS", "false", "", "Define if getpgrp takes an argument. (unused; code uses GETPGRP_HAVE_ARG)"),

It means that some dead macros haven’t been cleaned on windows.

What’s truly terrifying upon closer reflection is: are there any apis which can work on both windows and unix theoretically can be used only on unix due to the hard code finally?

11

If the build works, then presumably mode_t doesn’t need to be defined in pyconfig.h, which means that the config is correct. Where’s the problem to be solved?

I don’t know. Are there any? Find some. Then maybe there’d be something we can actually discuss.

12

The problem is: the comment misleads the users.

If there is no need to define, delete the comment, or write it matching the reality.

In fact, from the implementing for this idea, I can sure that the problem is not just these.

13

Finish. Now I’m sure that implementation has no problem in build after I review and modify the code.

14

Why not just fix the problems you have identifed with the pyconfig.h?
I’m not sure what being dynamic is helping fix.

I always found that Win32 builds are very stable in features.

The Windows features you can sense from the #define’s in the Windows .h files.

1 Like
15

The first step is to ensure that this dynamic is OK.

In the next step, I will check the unix macros. In fact, some changes for configure for macros often skip windows, so the list of the macros on windows may not be complete. Several days I just found this problem. Now I am check whether there are also some similar problems.

16

Your goal is to set the posix feature macros that are emulated on Windows?

17

More clearly, I want to find whether there are some functions that can be used on both windows and unix but now the configure macros only exist on unix. It will take me much time.

18

In my experience with cross platform code the Windows emulated posix API is not as robust as the Win32 APIs that implement the same features.

As Python already uses the Windows API I’m not sure what you expect from finding emulated posix functions.

19

Just a investigation. Maybe it have.

From my history posts I found that PC/pyconfig.h is often ignored. Maybe some function is not unix only but was forgot to add to windows?

20

We need specific features to be able to comment.

My guess at what you mean is that python has an API that it exposes on unix systems but not on Windows because there is not a #define in the pyconfig.h? Surely someone would have reported such an ommision by now?