I’m revisiting meson-python support for including symbolic links in sdists. meson-python up to version 0.16.0 includes the target of symbolic links as a regular file member in the sdist archive, version 0.17.0 and later ignore symbolic links all together. There are however use cases where having symbolic links in the package makes development easier thus we would like to restore the ability of including symbolic links in a form or another.
PEP 721 revised the Source distribution specification to clarify support for symbolic links. The specification prescribes that sdist must be unpacked using the tarfile.data_filter() or following the directives in the Unpacking without the data filter section. However, the documentation for tarfile.data_filter() and that section disagree on the handling of symbolic links: tarfile.data_filter() refuses to extract links that link to absolute paths, or ones that link outside the destination, the sdist specification states:
The following files are invalid in an sdist archive. Upon encountering such an entry, tools SHOULD notify the user, MUST NOT unpack the entry, and MAY abort with a failure:
[…] Links (symbolic or hard) pointing outside the destination directory. […] The following are also invalid. Tools MAY treat them as above, but are NOT REQUIRED to do so: Files with a..component in the filename or link target.
Which I interpret as meaning that it is up to the consumer of the sdist to decide whether symbolic link targets containing a .. component are allowed or not, independently of whether the link target is still within the archive or not.
As what is supported in an sdist is the minimum common denominator supported by all the consumers, this wording makes sdists including symbolic links to a relative path in the sdist unsupported: the only kind of supported symlinks are the ones that point to a file in the same directory, not to a file in a parent or sibling directory.
Why are the restrictions in the Unpacking without the data filter section of the specification tighter than the ones in the Unpacking with the data filter section?
Is this to discourage and eventually remove support for symbolic links? Because of the limitation highlighted above, meson-python will need to include symbolic links as regular files in the sdist.