Consider dropping Pydantic

[Secrus]

Will you consider dropping pydantic in favor of dataclasses or just plain Python objects? pydantic recently released v2 which has its core rewritten in Rust. That will cause major issues when trying to vendor pythonfinder in projects like pipenv.

[dvzrv]

FWIW, making it work with >= 2.0 is trivial: https://gitlab.archlinux.org/archlinux/packaging/packages/python-pythonfinder/-/blob/main/python-pythonfinder-2.0.4-pydantic2.patch

I'm currently doing rebuilds on Arch Linux for pydantic >= 2.0

[Secrus]

FWIW, making it work with >= 2.0 is trivial: https://gitlab.archlinux.org/archlinux/packaging/packages/python-pythonfinder/-/blob/main/python-pythonfinder-2.0.4-pydantic2.patch

I'm currently doing rebuilds on Arch Linux for pydantic >= 2.0

Yes, but in the case of vendoring dependencies and the Python distribution model, it works a bit differently. Linux packaging is different from Python packaging.

[matteius]

I am not in favor of removing pydantic, I've put quite a bit of time into replacing attr/attrs in pythonfinder, requirementslib and pipenv. We are trying to further overhaul requirementslib and want to make use of the validations with plans to backport changes that are made directly in vendor'd requirementslib back to requirementslib the package on some cadence or interval. We have no immediate plans to upgrade past pydantic 1.x and may revisit how the vendoring is done in pipenv before we would consider 2.x. Either way, changing vendoring or not, it will require serious considerations to go to pydantic 2 and there are more fundamental bugs in requirementslib that should be addressed first.

[Secrus]

I understand that, but still, I believe that pydantic should be dropped. Their migration to Rust want be taken back and one day they won't support v1 anymore. For now, at the very least, you could consider restricting pydantic version to <2 to make sure that it will continue working.

[matteius]

Is your objection to just pythonfinder using it because something other than pipenv is using pythonfinder or more broadly speaking? I think to get to pydantic 2, which has some benefits of being faster, we would possible need to not vendor pydantic but install it alongside pipenv. I've spent a bit of time looking more at how poetry installs itself for possible inspiration.

[Secrus]

well, I am part of Poetry core team and we are looking into getting pythonfinder or a similar library to use in our code for finding proper python executables. Recently we had an issue with jsonschema suddenly depending on a Rust-based library, which made me think about how many of our dependencies are using native extensions and whether we need all of them. Personally I am not against pydantic, but I see it as kinda more problematic than beneficial when it comes to libraries. It's fine when we are considering applications (like web apps built with pydantic and fastapi)

I've spent a bit of time looking more at how poetry installs itself for possible inspiration.

We are migrating to suggesting pipx as the default installation method, with our script as "second best".

[matteius]

@Secrus That is interesting, I would be open to a PR that converts pythonfinder to vanilla dataclasses; It would be cool to see poetry using pythonfinder if you head in that direction. I think as far as requirementslib goes, it will keep pydantic for the foreseeable future, at least until that library can be cleaned up but that likely wouldn't affect you.

Just a note on latest pythonfinder areas of improvement:

• when I dropped pep514tools to standardize the codepaths I didn't realize it was providing some kind of py command support for windows. I think it should be supported again but natively to pythonfinder without re-adding pep514tools`` Pythonfinder should support the windows py command pypa/pipenv#5779
• pythonfinder is a tad bit slower in Windows which is evidenced by the pipenv CI in the setup step -- for whatever reason the virtualenv creation and possibly finding python takes just marginally longer on windows -- I haven't quite figured out why or if it can be improved, it seems odd since the test CI is passing in the path to python so it shouldn't really be "finding" anything differently there.

We are migrating to suggesting pipx as the default installation method, with our script as "second best".

That is also interesting, admittedly I know how much about pipx but I wouldn't want to require another package manager that isn't the standard python package manager pip to install pipenv for example. What is the rationale for heading towards pipx?

[dvzrv]

Restricting this package to v1 only is problematic for downstream distributions, which now have to carry patches and continuously rework them (depending on upstream activity).

Could this not have been turned into a conditional import? 😢

[matteius]

Restricting this package to v1 only is problematic for downstream distributions

@dvzrv I am not sure what you mean seeing as pythonfinder won't run on pydantic 2 without code changes.

Aside -- I am in favor of removing pydantic to vanilla dataclasses, but I don't have the time to take it on myself at present.

[dvzrv]

pythonfinder won't run on pydantic 2 without code changes.

Yeah, the changes are fairly trivial though (see https://gitlab.archlinux.org/archlinux/packaging/packages/python-pythonfinder/-/blob/main/python-pythonfinder-2.0.5-pydantic2.patch) and could be turned into something that works across 1 and 2 for the time being.

I am in favor of removing pydantic to vanilla dataclasses, but I don't have the time to take it on myself at present.

Makes sense. I'll wait then :)

[oz123]

Late to join the party, but still.
I would also like to see a pydantic removed in favor of pure dataclasses. BUT, who is going to do this? It is a huge amount of work to do... If you are willing to put the effort, I think we should merge such a PR.

[Secrus]

@oz123 I am up to make the effort, just been busy with other projects lately.

[oz123]

I'd love to collaborate on that. I am also trying to do the same on plette.