TLDR
macOS creates ._*
files for every file in certain file systems like exFAT. As a result, when creating a virtual environment in exFAT, a ._*.pth
file is generated, which contains binary-encoded content that prevents Python from functioning properly.
Computer information
macOS version: Ventura 13.2.1
Mac model: Mac mini 2023 with m2
Python version: 3.11.3
Issue
After attempting to create a virtual environment by running python3 -m venv venv
on an external exFAT-formatted disk, I executed the command pip install -r requirements.txt
. This resulted in the following error:
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap>", line 982, in exec_module
File "<frozen site>", line 616, in <module>
File "<frozen site>", line 599, in main
File "<frozen site>", line 531, in venv
File "<frozen site>", line 384, in addsitepackages
File "<frozen site>", line 226, in addsitedir
File "<frozen site>", line 179, in addpackage
File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte
At this point, the python
command also produced the same error. However, python -S
executed successfully.
Investigation
I ran the python -vvv
command and extracted the relevant log information below:
Processing global site-packages
Adding directory: '/path/to/project/venv/lib/python3.11/site-packages'
Processing .pth file: '/path/to/project/venv/lib/python3.11/site-packages/._distutils-precedence.pth'
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap>", line 982, in exec_module
File "<frozen site>", line 616, in <module>
File "<frozen site>", line 599, in main
File "<frozen site>", line 531, in venv
File "<frozen site>", line 384, in addsitepackages
File "<frozen site>", line 226, in addsitedir
File "<frozen site>", line 179, in addpackage
File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte
# destroy site
then I inspected the ._distutils-precedence.pth
file:
(venv) ā site-packages git:(master) ā cat ._distutils-precedence.pth
Mac OS X 2��ATTR��
ļæ½
com.apple.provenance$u�@�This resource fork intentionally left blank ��%
(venv) ā site-packages git:(master) ā file ._distutils-precedence.pth
._distutils-precedence.pth: AppleDouble encoded Macintosh file
(venv) ā site-packages git:(master) ā file -I ._distutils-precedence.pth
._distutils-precedence.pth: application/octet-stream; charset=binary
It is clear that this file was generated by macOS. After researching the issue, I found this explanation: macos - Why are dot underscore ._ files created, and how can I avoid them? - Ask Different (stackexchange.com).
I deleted the ._distutils-precedence.pth
file, and everything is working correctly now.
Conclusion
Iām not sure how to permanently resolve this issue. Deleting all ._*
files every time after creating a venv seems to be the best workaround for now. Is there anything I can do or Python can do to prevent from this?
Perhaps I should consider switching to Linux in the future.