Skip to content

ENH: sys.monitoring compliance#327

Merged
Erotemic merged 11 commits into
pyutils:mainfrom
TTsangSC:monitoring-compliance
Mar 24, 2025
Merged

ENH: sys.monitoring compliance#327
Erotemic merged 11 commits into
pyutils:mainfrom
TTsangSC:monitoring-compliance

Conversation

@TTsangSC
Copy link
Copy Markdown
Collaborator

@TTsangSC TTsangSC commented Mar 22, 2025

Background

This came up during the discussion around #326. Since Python 3.12, profiling tools (like cProfile.Profile) are supposed to register themselves with sys.monitoring.use_tool_id(sys.monitoring.PROFILER_ID, ...) when in use, and sys.monitoring.free_tool_id(sys.monitoring.PROFILER_ID) when done.

This seemed like a trivial addition, but doing so naïvely caused tests/test_complex_case.py::test_varied_complex_invocations() to fail, presumably due to complications from the test script's use of concurrency. Trying to debug it was difficult, particularly since the singular test was essentially 11 in a trenchcoat.

The PR

Hence, this (draft) PR:

  • Refactors tests/test_complex_case.py::test_varied_complex_invocations() into a parametrized test, which allows the 11 subtests to be tested separately.
  • Adds to the Cython code of line_profiler/_line_profiler.pyx::LineProfiler.{en,dis}able() the sys.monitoring hooks; when calling use_tool_id(), the profiler is registered under the 'line_profiler' handle.

Obviously, the PR is in draft status since the test is still failing.

EDIT 22 Mar

The test has been fixed and the PR has been un-drafted with the following changes:

  • Added new test tests/test_line_profiler.py::test_sys_monitoring() to ensure that LineProfiler is interacting with sys.monitoring in the expected manner.
  • Added CHANGELOG.rst entry.

Caveats

Notably, as opposed to cProfile.Profile, profile.Profile doesn't register itself with sys.monitoring, and only uses the sys.setprofile() hook (which is intended for profilers written in pure Python). So maybe it isn't strictly necessary for us to go through the trouble...?

tests/test_complex_case.py::test_varied_complex_invocations()
    Broke down test into parametrized subtests for easier identification
    of problematic cases
line_profiler/_line_profiler.pyx::LineProfiler
    enable(), disable()
        Added `sys.monitoring.use_tool_id()` and
        `sys.monitoring.free_tool_id()` calls to comply with the
        standards post-Python-3.12
line_profiler/_line_profiler.pyx::LineProfiler.{en,dis}able()
    - Now only interacting with `sys.monitoring` on the main thread to
      avoid compilcations in multithreaded environments
    - Now checking for `sys.monitoring` availability at Cython
      compilation time, so that the internal methods interacting with
      `sys.monitoring` can be substituted with no-ops for Python < 3.12
tests/test_line_profiler.py
    test_init()
        Fixed flake (extra space)
    test_show_func_column_formatting()
        New test to ensure that `line_profiler.LineProfiler` is properly
        registered to `sys.monitoring` when in use
@TTsangSC TTsangSC changed the title DRAFT: ENH: sys.monitoring compliance ENH: sys.monitoring compliance Mar 22, 2025
@TTsangSC TTsangSC marked this pull request as ready for review March 22, 2025 14:13
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.32%. Comparing base (94dff94) to head (b80e87d).
Report is 13 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #327   +/-   ##
=======================================
  Coverage   61.32%   61.32%           
=======================================
  Files          11       11           
  Lines         848      848           
  Branches      186      186           
=======================================
  Hits          520      520           
  Misses        274      274           
  Partials       54       54           

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ee94e6...b80e87d. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@Erotemic Erotemic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Almost everything here is a nitpick. Feel free to discuss or mark them as resolved.

The only thing that I'm going to insist on is going over the test_complex_case case.

Comment thread tests/test_line_profiler.py Outdated
Comment thread tests/test_line_profiler.py Outdated
Comment thread tests/test_line_profiler.py Outdated
Comment thread tests/test_complex_case.py Outdated
Comment thread line_profiler/_line_profiler.pyx Outdated
Comment thread line_profiler/_line_profiler.pyx Outdated
- Stripped blank line between imports
- Renamed `h()` -> `get_profiling_tool_name()`
- `test_sys_monitoring()`:
  - Added error messages to assertions to be more self-documenting
  - Added check that no profiling tool is registered before the profiled
    version of `get_profiling_tool_name()` is called
- Dropped these functions and replaced them with C implementations:
 - `is_main_thread()`
 - `LineProfiler._sys_monitoring_register()`
 - `LineProfiler._sys_monitoring_deregister()`
- The switch between the post-Python-3.12 implementations and the no-op
  dummies now happens at Cython compile time, without polluting the
  namespace with the potentially-unused `_no_op()` method
- Added comments to `LineProfiler.enable()` and `.disable()`
Rolled refactoring of `~~::test_varied_complex_invocations()` into a
parametrized `pytest` test;
see discussion at
pyutils#327 (comment)
@Erotemic
Copy link
Copy Markdown
Member

I'm going to change my mind about the C implementation of the threading check after looking at its complexity. It opens way too many potential issues, and I want to land the feature. I doubt having the thread / monitoring check is going to be in the critical path, but in case it is let's make note of this patch in case we want to come back to it. But for now let's just revert to the pure-python calls.

My main concern is that we might be missing explicit error checks that need to be done, or GIL management. E.g. it looks like there are missing NULL checks on returns of PyObject_CallMethod calls, and maybe the result needs to have its reference count decreased? I'd need to think a lot more about it to be sure that it's all safe code.

Apologies for the flip/flop.

@TTsangSC
Copy link
Copy Markdown
Collaborator Author

TTsangSC commented Mar 24, 2025

missing explicit error checks

Yikes, I did miss the NULL check and the Py_DECREF() call in _sys_monitoring_deregister(). Guess that I was too focused on making those right for _is_main_thread(), but forgot to circle around to do the same for the other functions. But yeah the function is probably not hot enough to warrant the verbosity and drop in maintainability in going to C.

(EDIT: just realized that since the return values of the C calls are either a NULL pointer or a pointer to None (which is immortal), the refcounts don't really matter. But it's fixed nonetheless for good form. Also, interestingly, _lsprof.c doesn't raise an explicit error should the C call to sys.monitoring.free_tool_id() return a NULL pointer, as it does for use_tool_id()...)

Apologies for the flip/flop.

Don't worry about it, it was a fun exercise and I learned quite a bit about the C API and Cython.

but in case it is let's make note of this patch in case we want to come back to it. But for now let's just revert to the pure-python calls.

In the interest of keeping records, I guess I will fix _sys_monitoring_deregister() in a patch commit, before moving back to Python in the next commit then.

TTsangSC and others added 3 commits March 24, 2025 03:02
line_profiler/_line_profiler.pyx
    _sys_monitoring_register()
        Fixed bug where the return value of
        `sys.monitoring.use_tool_id()` (`None`) did not have its
        refcount decremented
    _sys_monitoring_deregister()
        Fixed bugs with `sys.monitoring.free_tool_id()`:
        - Where if it errored out (shouldn't happen), a Python error
          wasn't raised
        - Where if it didn't, the return value (`None`) didn't have its
          refcount decremented
line_profiler/_line_profiler.pyx::_sys_monitoring_[de]register()
    Replaced C implementations with pure Python for maintainability
@Erotemic Erotemic merged commit fc7c6b6 into pyutils:main Mar 24, 2025
@Erotemic
Copy link
Copy Markdown
Member

Thanks for the PR!

Now that this is merged, please rebase the next one on the latest main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants