Stop requiring `__file__` in Python packages

[indygreg]

The Problem

Many Python modules and scripts use __file__ to derive the filesystem path to the current file.

As documented at https://docs.python.org/3/reference/datamodel.html (search for __file__), __file__ is optional (the __file__ attribute may be missing for certain types of modules).

However, because Python has traditionally relied on filesystem-based imports and hasn't had a stable story around non-module resource handling, , __file__ is almost always defined and has been used to locate and load files next to Python source files for seemingly forever. This is arguably tolerable. But reliance on __file__ undermines tools - like PyOxidizer - which don't import Python modules from the filesystem. This in turn constrains the flexibility and utility of the larger Python ecosystem.

The Solution

Python code should be rewritten to not assume the existence of __file__. By doing so, Python code will be more compatible with more Python execution environments (such as PyOxidizer), and this benefits the overall Python ecosystem.

Instructions for writing portable Python code that doesn't rely on __file__ can be found at https://pyoxidizer.readthedocs.io/en/latest/packaging_pitfalls.html#reliance-on-file.

This Issue

This issue can serve as a focal point for tracking and coordinating Python packages and tools which currently rely on __file__ but shouldn't. If you file a GitHub issue against a project that relies on __file__, you can reference this issue by typing indygreg/PyOxidizer#69 and provide Python project maintainers with enough context to make informed decisions about the use of __file__ in their projects.

[QuantumChamploo]

I am having a similar issue, just when I am trying to import a package, for example numpy. So I am not sure how to change my code not to depend on file as the only line of code I have is "import numpy"

I have this issue when I use the repl mode, the eval mode, and when I try to run a script

[Alex-Mann]

Is there a recommended strategy for patching these modules locally to try and get them to work with PyOxidizer? Currently I am unable to find a way to build when I use a module that contains the __file__ variable.

I have set up a venv, downloaded all the packages that I am using locally, and have tried to monkey patch the __file__ variables where I am hitting errors, but it seems like pyoxidize run is using a cached version of the modules somewhere (unless they get pulled directly from pip each time) since the same error is still showing even though I have removed the __file__ variables.

Is it possible to somehow search and replace the __file__ variable when it is found? I've only just started using this module but it seems like many python packages are probably using it as well. Expecting them to accept pull requests / updates to deprecate this in the near future doesn't really seem all that feasible.

[Alex-Mann]

Ok, after doing some more digging around the repo, I found the solution offered for the black example. I didn't see this earlier so I didn't know it existed. The package I'm using is GitPython, so what I did was add:

[[embedded_python_config]]
sys_paths = ["$ORIGIN/lib"]

[[packaging_rule]]
type = "pip-install-simple"
package = "gitpython"
install_location = "app-relative:lib"

Now, the python application that I am developing lives in a virtualenv, so originally I was just using that virtualenv to dictate what packages needed to be rolled up. Is there an equivalent command for install_location = "app-relative:lib" and sys_paths = ["$ORIGIN/lib"] to get this to work with a virtualenv?

[n8henrie]

it seems like pyoxidize run is using a cached version of the modules somewhere

Try making a minor (e.g. whitespace) change to the toml file

[n8henrie]

Running a file with python -i and looking through its globals, I noted that __spec__.origin is the same as __file__, but when I made a test package that prints its value into a PyOxidizer executable, it turns into None. (At least the file runs.)