GH-46589: [C++] Fix utf8_is_digit to support full Unicode digit range

[iabhi4]

Rationale for this change

pyarrow.compute.utf8_is_digit did not recognize valid Unicode digit characters (e.g., superscripts like '³'), diverging from the behavior of Python's built-in str.isdigit()
This caused inconsistencies in downstream libraries like pandas when using PyArrow-backed StringDtype.

What changes are included in this PR?

Updated IsDigitCharacterUnicode implementation to cover a broader range of Unicode digits by replacing category check with one that aligns with Python’s str.isdigit() semantics.

Added tests in scalar_string_test.cc to validate correct digit detection across diverse Unicode digit inputs.

Are these changes tested?

Yes. New unit tests were added and pass successfully, verifying behavior on various Unicode digit characters.

Are there any user-facing changes?

Yes, users relying on pc.utf8_is_digit() will now get correct results for a wider range of Unicode digit characters, improving correctness and parity with Python semantics

• GitHub Issue: [Python] utf8_is_digit in PyArrow doesn't fully match Python's str.isdigit() (e.g., fails for '³') #46589

[github-actions]

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

• Other pull requests
• Contribution Guidelines - Contributing Overview

[kou]

@pitrou Could you take a look at this?

NOTE: utf8_is_digit() was introduced by #7656.

[pitrou]

Thanks a lot for the update @iabhi4 ! This LGTM, I'll just wait for CI.

[pitrou]

I pushed a quick fix for the C++ lint failures, will merge.

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 0 benchmarking runs that have been run so far on merge-commit dc0f5a9.

None of the specified runs were found on the Conbench server.

The full Conbench report has more details.