Type hints

[MBradbury]

I have seen #755 and wanted to take a look at providing type hints also. I would appreciate a review of this, as there are some choices I made which need to be checked and others where a decision needs to be made.

Summary:

• Increases minimum version to Python 3.9, as this is when hinting list[T] was introduced (instead of using List[T])
• Added Lpt* type aliases to avoid excessive Unions across the code
• Overloads for dict, dicts and matrix
• Most of the examples type check successfully
• Removed LpElement
• Needed to move away from dicts to dataclasses to serialise MPS. Ability to convert to dicts is provided by Python, but a separate package (dacite) has been needed to load from json into the dataclasses.
• Removed the ability for `LpAffineExpression to inherit from an OrderedDict, always inherit from dict.

Choices to make:

1. What type should be used to store coefficients? Currently a mixture of int and float is allowed.
2. dicts and matrix cannot be fully typed with Python's type hints. I have added overloads for 3 dimensional inputs, how many is appropriate?
3. What is LpConstraintVar? Is it a variable or is it a constraint? There are several pieces of code in LpAffineExpression that assume that all the items it contains are either a LpVariable or a LpConstraintVar. Several options:
   • Could LpConstraintVar inherit from LpVariable?
   • Manage dict of LpConstraintVar separately?
   • Does LpConstraintVar support operators in a LpAffineExpression?
4. There seems to be a number of problems with FractionElasticSubProblem as parameters seemed to be in the incorrect order for the __init__.

There is also a test failure to investigate (PULP_CBC_CMDTest.test_dual_variables_reduced_costs), but I am having trouble finding the root cause.

[pchtsp]

Thanks, I will happy contribute to this initiative of adding types.

I have no problem moving to 3.9, although if it's only to avoid writing from typing import List then it's a bit too much, isn't it?
Moving to dataclasses seems a good idea. I'm not well versed on them so I cannot say much. I know pydantic is also used to type objects, not sure if relevant (I don't regularly used these libraries).
On the dacite question, I would try hard not to add new dependencies unless really justified. The package currently has none and I think that's quite nice.

Many of your questions are for @stumitchell to answer, I believe.
On the first two questions:

• What type should be used to store coefficients? Currently a mixture of int and float is allowed.
  • Without checking the "int" cases I would assume coefficients should be float (because that's what LP understands best).
• dicts and matrix cannot be fully typed with Python's type hints. I have added overloads for 3 dimensional inputs, how many is appropriate?
  • It's hard to impose anything... I myself make dicts of tuples or various size. But part of the charm of PuLP is that it's very flexible and people use it very differently

[christiansegercrantz]

Amazing work, thank you! I can also help with reviewing and changes if needed, although currently a bit busy.

Regarding python 3.8, it's already end-of-lifecycle, so I don't see a reason not to move away from it, people shouldn't be using it anyway.

[MBradbury]

For dacite, alternatively, pulp moves to pickle for disk serialisation which would eliminate the need for dacite, but then human-readability of the files would be lost.

The type hints do not prevent dicts / matrix being used with more dimensions, just means that IDEs won't provide accurate information on the returned type.

[pchtsp]

well, I just took python 3.8 out because gurobipy did and just makes testing more complicated.

[MBradbury]

I have been thinking about this and I think it would be best to split this up into different PRs:

1. The class hierarchy of LpElement, LpVariable, and LpConstraintVar
2. The loading and saving via dataclasses
3. Other general type hint work

[pchtsp]

as reference @frrad has been doing some typing/annotations work here: #819

[pchtsp]

hello! Should we close this PR? I think most of the hints are there, and the rest can be addressed as part of the regular changes to the repo. Thanks for all your help @MBradbury

[MBradbury]

Yes we should, just two things to get out of it before closing:

1. Need some kind of resolution to: Type utilities.value and utilities.valueOrDefault #825, it is not clear to me how to handle the failing tests due to the discussion in Type examples/ #819 with @frrad
2. Once that's addressed then I think the most valuable bit to extract is the typing of matrix and dicts

   pulp/pulp/pulp.py

   Line 285 in 7e59084

   @overload

   as these were non-trivial to reason about

Other issues still need to be looked at, but may be better as individual issues:

1. LpVariable/LpConstraintVar both stored in LpAffineExpression, but do not implement the same interface
2. FractionElasticSubProblem looks like it has parameters in the incorrect order
3. What type should coefficients be stored as (with TypeError when LpAffineExpressions are Specified Using Decimals #815 also adding Decimal to be considered)