Fix fallout from SimpleGraphic upgrade with wider Unicode support#7586
Closed
zao wants to merge 5 commits intoPathOfBuildingCommunity:devfrom
Closed
Fix fallout from SimpleGraphic upgrade with wider Unicode support#7586zao wants to merge 5 commits intoPathOfBuildingCommunity:devfrom
zao wants to merge 5 commits intoPathOfBuildingCommunity:devfrom
Conversation
As the runtime is going to support Unicode installation locations and build directories, some UTF-8 text is going to reach the Lua side of the project. This includes the script path, the user path, any paths yielded from file searches and also imported character names from accounts. Care needs to be taken in many places where string operations are performed as no longer does a byte necessarily correspond to a single character and anything that truncates, reverses or otherwise slices strings could need an audit. This change fixes cursor movement in `EditControl`s with the arrow keys as those historically used string matching and byte offsets. It also ensures that the use of arbitrary Unicode codepoints as decimal and thousands separators works correctly as the previous code used unaware reversing and slicing.
Historical UI code uses empty text controls with invalid font sizes as anchors for other UI as that used to be benign with older font technology. While the runtime should be hardened against such use, this replaces the controls with more appropriate base types to better communicate intent.
The updater is a fixed piece of older code that uses a Lua runtime that only handles paths that are representable in the user's text codepage. As the software may be installed in a location that cannot be expressed in that way, to mitigate the problem we turn all the paths in the update op-files into relative paths. That way as long as we never use exotic codepoints in our own paths it should be able to apply them cleanly and restart Path of Building afterward with a relative path. The updater executable can ironically enough not be updated at all with the related type of runtime hacks we introduced in SimpleGraphic as the updater deadlocks in updating itself. We have to work around its shortcomings in how we produce the op-files and possibly the update application script that runs under that limited runtime.
Upon removing support for several file formats like GIF and BLP from the SimpleGraphic runtime, we noticed that there were some assets that had incorrect file extensions and loaded only thanks to file format detection ignoring extensions. As the actual file format loader for GIF was removed, these stealth GIFs are now losslessly converted to PNG.
As all code to use bitmaps fonts is gone from the runtime, this introduces a basic set of main and fallback fonts to render text once again, this time with the ability to show Unicode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of the problem being solved:
In PathOfBuildingCommunity/PathOfBuilding-SimpleGraphic#59 there were significant foundational improvements for users installing Path of Building into paths with characters that cannot be expressed in their codepage.
There is also a companion PR for the Launcher in PathOfBuildingCommunity/PathOfBuilding-Launcher#6.
In short:
GetScriptPath/GetUserPathyields paths in UTF-8,[U+1234]"tofu" in a smaller font size,lua-utf8is exposed to the Lua side.This means that Lua code that touches things like the full build path or the script path may need to adapt to process UTF-8 correctly with string operations.
In order to help with that, the SimpleGraphic update also features luautf8, a Lua extension that provides UTF-8 analogues of many string operations like
gsub,match, etc.This change leverages that extension via
local utf8 = require('lua-utf8')to correctly move the text caret with the cursor keys in edit controls, and also showcases its use to handle (hypothetical) Unicode decimal and thousands separators.The runtime also lost the vestigial ability to process GIF and BLP image formats, some existing assets were stealth GIFs with PNG extensions and have here been converted lossless to PNG.
The update check logic has been adapted to generate relative paths in the op-files to make its limited interpreter work correctly in exotic install locations.
Steps taken to verify a working solution:
Install location:
Build path:
Build name:
Numeric separators:
Settings.xml,Link to a build that showcases this PR:
n/a
After screenshot: