Skip to content

Fix missing coordinate text fields for decimal coordinates#5368

Merged
grantfitzsimmons merged 7 commits into
productionfrom
issue-5365
Mar 11, 2025
Merged

Fix missing coordinate text fields for decimal coordinates#5368
grantfitzsimmons merged 7 commits into
productionfrom
issue-5365

Conversation

@grantfitzsimmons

@grantfitzsimmons grantfitzsimmons commented Nov 2, 2024

Copy link
Copy Markdown
Member

Warning

This PR affects database migrations. See migration testing instructions.

Fixes #5365

This essentially runs the following SQL statement:

# There are cases from Specify 6 where Longitude and Latitude are captured as decimal values but
# the accompanying text fields are not populated. In cases where the text fields are not populated,
# it appears the records do not have coordinates. This script will populate the text fields with the
# decimal values where the text fields are empty but the decimal values are not.

UPDATE locality
SET Lat1text = Latitude1
WHERE Lat1text IS NULL AND Latitude1 IS NOT NULL;

UPDATE locality 
SET Long1text = Longitude1 
WHERE Long1text IS NULL AND Longitude1 IS NOT NULL;

UPDATE locality
SET Lat2text = Latitude2
WHERE Lat2text IS NULL AND Latitude2 IS NOT NULL;

UPDATE locality 
SET Long2text = Longitude2 
WHERE Long2text IS NULL AND Longitude2 IS NOT NULL;

The migration is also fully reversible!

specify@feed99cd5b4e:/opt/specify7$ ve/bin/python manage.py migrate specify 0010_updateDelete_parentcojo
Operations to perform:
  Target specific migration: 0010_updateDelete_parentcojo, from specify
Running migrations:
  Rendering model states... DONE
  Unapplying specify.0012_coordinate_fields_fix... OK
  Unapplying specify.0011_cascading_tree_nodes... OK
specify@feed99cd5b4e:/opt/specify7$ 

Testing instructions

Some databases are known to have this issue due to migration artifacts and this comes up from time to time.

You can usually identify databases with this issue by running the following query on them:

SELECT *  FROM locality  WHERE Lat1text IS NULL AND Latitude1 IS NOT NULL;

If any records are returned, it's a great candidate for testing this migration.

Examples that I found on the test panel are available in this document.

On production before the migration

  • *Modify several locality records to remove values from the Lat1text and Long1text fields while preserving Latitude1 and Longitude1 values OR use one of the test databases mentioned in the earlier doc and find all records where Latitude1 exists but Lat1text does not (using Query Builder or other methods).
  • Verify that the LatLonUI plugin on Locality doesn't display any values.

On issue-5365 after the migration

  • Verify that all cases have been resolved after the migration has completed via Query Builder or SQL
  • Verify that the LatLonUI plugin on Locality now displays the correct values copied from the decimal coordinate fields

@grantfitzsimmons grantfitzsimmons added the Migration Prs that contain migration label Nov 2, 2024
@grantfitzsimmons grantfitzsimmons added this to the 7.9.x milestone Nov 2, 2024
Comment thread specifyweb/specify/migrations/0012_coordinate_fields_fix.py Outdated

@maxpatiiuk maxpatiiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@grantfitzsimmons grantfitzsimmons modified the milestones: 7.x, 7.10.1 Feb 4, 2025
Triggered by 2ca6acf on branch refs/heads/issue-5365
@grantfitzsimmons
grantfitzsimmons changed the base branch from production to issue-5131 February 4, 2025 20:39
@grantfitzsimmons
grantfitzsimmons marked this pull request as ready for review February 4, 2025 20:41
Base automatically changed from issue-5131 to production March 10, 2025 19:50
@grantfitzsimmons
grantfitzsimmons requested a review from a team March 10, 2025 19:52

@emenslin emenslin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Verify that all cases have been resolved after the migration has completed via Query Builder or SQL
  • Verify that the LatLonUI plugin on Locality now displays the correct values copied from the decimal coordinate fields

Looks good!

@emenslin
emenslin requested a review from a team March 11, 2025 13:08
@grantfitzsimmons
grantfitzsimmons merged commit 41c974d into production Mar 11, 2025
@grantfitzsimmons
grantfitzsimmons deleted the issue-5365 branch March 11, 2025 14:04
@github-project-automation github-project-automation Bot moved this from 📋Back Log to ✅Done in General Tester Board Mar 11, 2025
@specifysoftware

Copy link
Copy Markdown

This pull request has been mentioned on Specify Community Forum. There might be relevant details there:

https://discourse.specifysoftware.org/t/specify-7-10-2-release-announcement/2455/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Migration Prs that contain migration

Projects

Status: ✅Done

Development

Successfully merging this pull request may close these issues.

Latitude and Longitude coordinates without matching Lat1text and Long1text coordinate fields appear empty

6 participants