Expand merging to Localities, Paleo Ctxt, Collecting Event

[CarolineDenis]

Fixes #4046

Checklist

• Self-review the PR after opening it to make sure the changes look good
  and self-explanatory (or properly documented)
• Add automated tests
• Add relevant issue to release milestone

Testing instructions

• perform same tests as agent merging but for new tables
• Verify that we can’t merge PC and CE records when they are configured to be embedded
• Test for Merging agents unchecks isCurrent checkbox in associated determinations #4969 (see instructions in that issue) -- has a unit test

Utils:
https://discourse.specifysoftware.org/t/shared-vs-embedded-records-configuration/1712/1

[CarolineDenis]

What would be the logical new code?
@maxpatiiuk , @realVinayak

MergeSubviewButton line 52, CompareSubView.tsx

    () =>
      relationshipIsToMany(relationship)
        ? (resource as SpecifyResource<Accession>).getDependentResource(
            relationship.name as 'accessionAgents'
          )?.models.length ?? 0
        : resource.get(relationship.name) === undefined
        ? 0
        : 1,
    [relationship, resource]
  );

In the code above, relationship is a 'zero-to-one'.
relationshipIsToMany(relationship) returns true because

export const relationshipIsToMany = (
  relationship: Relationship | undefined
): boolean =>
  relationship?.type.includes('-to-many') === true ||
  relationship?.type === 'zero-to-one';

models.length is now throwing an error for other tables

Why do we have zero-to-one being consider as a IsToMany?

[maxpatiiuk]

In the code above, relationship is a 'zero-to-one'.
relationshipIsToMany(relationship) returns true because

zero-to-one is a "hack" relationship type introduced at some point in sp6 to do a quick fix for some user request, and now we have to live with the consequences (that's why quick fixes aren't a good idea most of the time)
I must admit, while Ben explained it to me several times, I didn't fully understand what zero-to-one means or why it's necessary. If I remember correctly, zero-to-one means "one-to-one" from one direction, but "many-to-many" in the reverse duration, or something like that, which is why in relationshipIsToMany we say true for zero-to-one fields.

But, @realVinayak or @melton-jason might be able to provide a better description that me based on their knowledge of back-end, business rules, databases and sp6

If it becomes really necessary, and we can't infer what zero-to-one means from the back-end code, we could contact Ben to ask him to explain it one more time

---

edit:

my messages to Ben about this:

Ok, I implemented zero-to-one. Tell me if I did it correctly:
they are handled as -to-many's:

but if you addded a -to-many item, you can not add another one (there is no Add button)

Ben:

That'll work. thanks

---

See also helpful information in https://specifydev.slack.com/archives/CC6V12D3J/p1617992659051300

---

Max:

I just found a relationship type I haven't seen before: zero-to-one
How should I treat these? are those like one-to-one but one directional?

Ben:

Yes. I don't know why it is called that. It is kind of stupid. It is just a one-to-one.

[melton-jason]

might be able to provide a better description that me based on their knowledge of back-end, business rules, databases and sp6

#4606 (comment)

The assesment @maxpatiiuk provided is correct to my knowledge.

There are two zero-to-one relationships in Specify and both on the Locality table:
localityDetails and geoCoordDetails.

I don't think the backend of Specify 7 has any dedicated logic to handling zero-to-one relationships, besides marking the to-"many" side of the relationship as dependent on the to-one side.

specify7/specifyweb/specify/load_datamodel.py

Lines 333 to 336 in 906d739

	'Locality.geocoorddetails',
	'Locality.latlonpolygons',
	'Locality.localitycitations',
	'Locality.localitydetails',

At a database level, it is indistinguishable from a one-to-many relationship.
That is, the LocalityDetail and GeoCoordDetail tables contain a ForeignKey to the Locality table. There are no other constraints in place, so it is perfectly valid from the database's perspective for a Locality to have more than one LocalityDetails or GeocoordDetails.

However, for whatever reason the restriction was needed in Specify 6 to enforce the relationship as a one-to-one. (i.e. a maximum of one LocalityDetails/GeocoordDetails per Locality).
This restriction is enforced and managed entirely within the application (MySQL or Mariadb has no concept of the relationship type or the restriction that is desired).

In the Specify 6 code, the relationship was denoted ZeroOrOne, but used zero-to-one when writing the specify_datamodel.xml.

https://github.com/specify/specify6/blob/945d28f00288d2d64979848fb2fb63fda34bdd1b/src/edu/ku/brc/af/core/db/DBRelationshipInfo.java#L36

Although, when working with the ORM Specify6 uses (Hibernate), the columns still needed to be mapped as OneToMany (on Locality) and ManyToOne (on LocalityDetails/GeocoordDetails)

localityDetails relationship on Locality:
https://github.com/specify/specify6/blob/945d28f00288d2d64979848fb2fb63fda34bdd1b/src/edu/ku/brc/specify/datamodel/Locality.java#L1083-L1091

locality relationship on LocalityDetail:
https://github.com/specify/specify6/blob/945d28f00288d2d64979848fb2fb63fda34bdd1b/src/edu/ku/brc/specify/datamodel/LocalityDetail.java#L935-L940

It seems they were first added in specify/specify6@3126d02, but I can not find any documenttation or reasoning as to why this restriction was needed or desired in the first place.

The relationship can be approached conceptually as "A Locality can have zero or one associated LocalityDetail/GeocoordDetail records".

[maxpatiiuk]

Thanks for the detailed analysis @melton-jason!

The relationship can be approached conceptually as "A Locality can have zero or one associated LocalityDetail/GeocoordDetail records".

Interesting, because it seems like this could have been represented as an optional one-to-one, unless it has to be different for some reason specific to sp6 codebase

[maxpatiiuk]

@melton-jason would it make sense to use the schemaoverwrites files to make sp7 front-end treat this as a -to-one relationship?
or would this cause issues because back-end would still return it as if it's a nested resources (thus will be an array rather than a direct resource in the API response)?
or should front-end act as if it's a one-to-many, but then have a business rule that causes a crash when more than one is added (causes a crash, so that it becomes obvious where in the code we forgot to restrict adding more than 1 resource for a zero-to-one)

[melton-jason]

Currently, it would be easiest for the frontend to treat zero-to-one relationships as a dependent one-to-many relationships.
This is how the backend currently serializes and returns records with the relationship type: as an array containing one or no records.

While it would require modifications to both the frontend and backend, It might be easier to treat the relationship as a dependent many-to-one relationship, like the collectionObjectAttribute field on CollectionObject. Instead of returning an array with zero or one element, the value of the field would either be the serialized resource or null.

However, there is a small concern regarding the name of the relationships: localityDetails and geocoordDetails. The pluralized name implies there can be more than one and that the relationship is a to-many.
So perhaps the most ideal solution is to make the relationship a true one-to-many and remove the one record maximum restriction.
Of course, this would require compatibility with Specify 6 to be broken and assumes it makes sense to allow more than one LocalityDetail and GeocoordDetail record per Locality.

[realVinayak]

Currently, it would be easiest for the frontend to treat zero-to-one relationships as a dependent one-to-many relationships. This is how the backend currently serializes and returns records with the relationship type: as an array containing one or no records.

While it would require modifications to both the frontend and backend, It might be easier to treat the relationship as a dependent many-to-one relationship, like the collectionObjectAttribute field on CollectionObject. Instead of returning an array with zero or one element, the value of the field would either be the serialized resource or null.

However, there is a small concern regarding the name of the relationships: localityDetails and geocoordDetails. The pluralized name implies there can be more than one and that the relationship is a to-many. So perhaps the most ideal solution is to make the relationship a true one-to-many and remove the one record maximum restriction. Of course, this would require compatibility with Specify 6 to be broken and assumes it makes sense to allow more than one LocalityDetail and GeocoordDetail record per Locality.

the current pr patches that for this. see relevant changes

[maxpatiiuk]

small simplification, otherwise looking good

[emenslin]

In paleocontext even if the forms are exact copies the merge dialog appears like there are differences between the two. Not sure if this happens in other tables but I only noticed it with the pc table.

I cloned a pc record, made no changes, tried to merge and this happened

[emenslin]

Close button behavior looks better however collectors are still deleted

OH6hnVVK4l.mp4

I was also able to find the pc issue mentioned before in the agent dialog so it seems like it might be in every table

[alesan99]

• perform same tests as agent merging but for new tables
• Verify that we can’t merge PC and CE records when they are configured to be embedded
• Test for #4969 (see instructions in that issue) -- has a unit test

I also encountered a crash when removing all Locality Attachments when merging Localitites

chrome_HxfA730Ahg.mp4

Specify 7 Crash Report - 2024-06-05T14_05_15.914Z.txt

[alesan99]

I also encountered a crash when removing all Locality Attachments when merging Localitites

As I was writing up my issue it got fixed by the new changes.
After testing it again its working fine now 👍

[emenslin]

Testing instructions

• perform same tests as agent merging but for new tables
• Verify that we can’t merge PC and CE records when they are configured to be embedded
• Test for #4969 (see instructions in that issue) -- has a unit test

Collectors issue seems to be fixed, if deleted in the merge dialog they are not deleted from the individual record

The show conflicting fields issue is in edge so I will write up a separate issue for that.

Everything looks good but I would like another testing review just to make sure we caught everything since there's a lot to cover

[lexiclevenger]

Testing instructions

• perform same tests as agent merging but for new tables
• Verify that we can’t merge PC and CE records when they are configured to be embedded
• Test for #4969 (see instructions in that issue) -- has a unit test

---

Looks good! The requested changes also seem to be fixed.

[Areyes42]

Testing instructions

• perform same tests as agent merging but for new tables
• Verify that we can’t merge PC and CE records when they are configured to be embedded
• Test for Merging agents unchecks isCurrent checkbox in associated determinations #4969 (see instructions in that issue) -- has a unit test

---

I encountered similar issues as mentioned in the previous comments when I first tested this PR, but it appears everything has been resolved now. Great job! 👏

[pashiav]

Testing instructions

• perform same tests as agent merging but for new tables
• Verify that we can’t merge PC and CE records when they are configured to be embedded
• Test for #4969 (see instructions in that issue) -- has a unit test

---

Looks good! Working as expected.