Work around Pipenv bug when using --system#1842
Merged
Merged
Conversation
After moving Pipenv to its own virtual environment in #1840 (to fix leaking Pipenv's dependencies into the app environment), apps that explicitly depend on a package that happen to also be a Pipenv dependency were seeing `ModuleNotFoundError` failures. For example: ``` ModuleNotFoundError: No module named 'certifi' ``` It appears that Pipenv's `--system` feature has a bug whereby it sees its own dependencies (installed in its own environment) as already installed (even though they aren't actually installed in the app environment), so incorrectly skips installing them. In general, it seems Pipenv's support for `--system` is pretty buggy: https://github.com/pypa/pipenv/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22--system%20flag%22 (for example, it doesn't work with `pipenv run` either) The classic Python buildpack doesn't currently use virtual environments (since it installs a dedicated Python install for the app, that's already separate from system Python), and switching to them would be a significant change that may cause other types of breakage (for example, to apps that hardcode the path to Python). As such for now, we need to continue to use `pipenv install --system` (as the buildpack has done for some time), but I have found the bug can be worked around if we set `VIRTUAL_ENV` to the Python install location, to trick Pipenv into thinking it's operating in a virtual environment. The Pipenv tests have been updated to confirm that an explicit `certifi` install now works. GUS-W-19118553.
9bee985 to
a8b2fdc
Compare
colincasey
approved these changes
Jul 23, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After moving Pipenv to its own virtual environment in #1840 (to fix leaking Pipenv's dependencies into the app environment), apps that explicitly depend on a package that happen to also be a Pipenv dependency were seeing
ModuleNotFoundErrorfailures.For example:
Or:
It appears that Pipenv's
--systemfeature has a bug whereby it sees its own dependencies (installed in its own environment) as already installed (even though they aren't actually installed in the app environment), so incorrectly skips installing them.In general, it seems Pipenv's support for
--systemis pretty buggy (for example, it doesn't work withpipenv runeither):https://github.com/pypa/pipenv/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22--system%20flag%22
The classic Python buildpack doesn't currently use virtual environments (since it installs a dedicated Python install for the app, that's already separate from system Python), and switching to them would be a significant change that may cause other types of breakage (for example, to apps that hardcode the path to Python).
As such for now, we need to continue to use
pipenv install --system(as the buildpack has done for some time), but I have found the bug can be worked around if we setVIRTUAL_ENVto the Python install location, to trick Pipenv into thinking it's operating in a virtual environment.The Pipenv tests have been updated to confirm that an explicit
certifiinstall now works.GUS-W-19118553.