Skip to content

gh-131591: Implement PEP 768 #131937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 75 commits into from
Apr 3, 2025
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
6f6b4cd
gh-131591: Implement PEP 768
pablogsal Mar 22, 2025
9b86022
Add tests and use io.open_code
pablogsal Mar 23, 2025
af84100
Some fixes
pablogsal Mar 23, 2025
19ef7ae
Prepare for windows support
pablogsal Mar 24, 2025
444453c
Update windows files
pablogsal Mar 24, 2025
1d3ad3c
Fix syntax error
ivonastojanovic Mar 25, 2025
eeec1f6
Merge pull request #97 from ivonastojanovic/fix_windows_files
pablogsal Mar 25, 2025
fd993e3
Locate PyRuntime on Windows
ivonastojanovic Mar 27, 2025
96798c3
Read process memory on Windows
ivonastojanovic Mar 27, 2025
075ca65
Write to process memory on Windows
ivonastojanovic Mar 27, 2025
45e73c5
Prevent attaching to a process with a different architecture
ivonastojanovic Mar 28, 2025
ed2f325
Revert "Prevent attaching to a process with a different architecture"
godlygeek Mar 28, 2025
6076548
Merge pull request #98 from ivonastojanovic/external_debugger_windows…
godlygeek Mar 28, 2025
a9d3ea9
Remove unused struct member
godlygeek Mar 28, 2025
e235e62
Set an explicit size for a struct member
godlygeek Mar 28, 2025
d51dda0
Revert an accidental whitespace change
godlygeek Mar 28, 2025
38a4d51
Factor running a debugger script into a helper function
godlygeek Mar 28, 2025
a98898d
Harden remote debugging integration
godlygeek Mar 28, 2025
997b557
Use _fdopen to open the debugger script on Windows
godlygeek Mar 28, 2025
f6dec59
Ensure the debugger script is always closed
godlygeek Mar 28, 2025
d273c5b
Fix incorrect docstring
godlygeek Mar 29, 2025
c9a2146
Document restrictions for remote_exec
godlygeek Mar 29, 2025
4af1744
Simplify handling of Windows paths
godlygeek Mar 29, 2025
5c0b8b9
Make remote_exec accept `bytes` paths
godlygeek Mar 29, 2025
c8779cd
Refactor to avoid duplicate error handling
godlygeek Mar 26, 2025
6889042
Check for debug offset compatibility before using the offsets
godlygeek Mar 26, 2025
7f7aa8b
Give a some variables shorter names
godlygeek Mar 26, 2025
fbecfdb
Have read_memory/write_memory return 0 on success
godlygeek Mar 27, 2025
fa98f64
Improve an error message
godlygeek Mar 27, 2025
5b4cb00
Remove a debugging printf
godlygeek Mar 29, 2025
0dd7797
Fix a remote read to use the correct type
godlygeek Mar 29, 2025
b8a0503
Improve the error message when we can't find the requested thread
godlygeek Mar 29, 2025
9344d1d
Remove a useless check
godlygeek Mar 29, 2025
9368d38
Only accept a flag of 1 to mean remote debugging is enabled
godlygeek Mar 29, 2025
166f4d6
Merge remote-tracking branch 'upstream/main'
pablogsal Mar 31, 2025
d253966
Lint
pablogsal Mar 31, 2025
8e04fdd
simplify socket handling
pablogsal Mar 31, 2025
0c2b275
Add NEWS entry
pablogsal Mar 31, 2025
80856d3
Add more docs
pablogsal Mar 31, 2025
f01d8d3
Small fixes
pablogsal Mar 31, 2025
cca28c5
Small fixes
pablogsal Mar 31, 2025
727a02f
windows fix
pablogsal Mar 31, 2025
6ec528c
fix windows warnings
pablogsal Mar 31, 2025
f7e3963
Go back to previous test socket handling
pablogsal Mar 31, 2025
40bda7c
Fix docs lint
pablogsal Mar 31, 2025
780561b
Add defined(__FreeBSD__)
pablogsal Mar 31, 2025
2ab5d70
Small fixes
pablogsal Mar 31, 2025
3a5f004
windows fix
pablogsal Mar 31, 2025
6346063
Update 3.14.rst
pablogsal Mar 31, 2025
b442b1c
Update sys.rst
pablogsal Mar 31, 2025
3a4ed23
Address doc review
pablogsal Mar 31, 2025
8514d6e
Fix WASI
pablogsal Mar 31, 2025
72d3ece
whooopsy
pablogsal Mar 31, 2025
9130bb8
Fixes
pablogsal Mar 31, 2025
373d4c4
Update Doc/whatsnew/3.14.rst
pablogsal Mar 31, 2025
bccc8a8
Update Doc/using/configure.rst
pablogsal Mar 31, 2025
90ab65f
Fixes
pablogsal Mar 31, 2025
ffd340f
Address code review
pablogsal Mar 31, 2025
247e753
Apply suggestions from code review
pablogsal Mar 31, 2025
3c4c41c
Update Doc/library/sys.rst
pablogsal Mar 31, 2025
18c2e64
lint
pablogsal Mar 31, 2025
139b234
address code review from Steve
pablogsal Apr 1, 2025
4306a73
Merge remote-tracking branch 'upstream/main' into pep-768
pablogsal Apr 1, 2025
35003a8
Merge remote-tracking branch 'upstream/main' into pep-768
pablogsal Apr 1, 2025
80fab75
Use PyUnicode_FSDecoder instead of os.fsdecode
godlygeek Apr 1, 2025
0905105
Clean up #includes in remote_debugging.c
godlygeek Apr 1, 2025
8db68e3
Mark (read|write)_memory unreachable on platforms w/o remote debugging
godlygeek Apr 1, 2025
25cc486
Add error code to exception when failing to write memory on macOS
godlygeek Apr 1, 2025
0557de0
Fix the size for one of our remote reads
godlygeek Apr 1, 2025
6dd7530
Move reading eval_breaker closer to where it's written
godlygeek Apr 1, 2025
7c340e1
Merge remote-tracking branch 'upstream/main' into pep-768
pablogsal Apr 2, 2025
9096375
Handle arbitrarily long paths in /proc/<PID>/maps
godlygeek Apr 2, 2025
6516356
Ensure we munmap before closing the fd
godlygeek Apr 2, 2025
e6cb8fb
Merge branch 'main' into pep-768
pablogsal Apr 2, 2025
8b3ffa9
Update Doc/whatsnew/3.14.rst
pablogsal Apr 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions Doc/using/cmdline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -604,7 +604,11 @@ Miscellaneous options
.. versionadded:: 3.13

* ``-X disable_remote_debug`` disables the remote debugging support as described
in :pep:`768`. This option is only available on some platforms and will do nothing
in :pep:`768`. This includes both the functionality to schedule code for
execution in another process and the functionality to receive code for
execution in the current process.

This option is only available on some platforms and will do nothing
if is not supported on the current system. See also
:envvar:`PYTHON_DISABLE_REMOTE_DEBUG` and :pep:`768`.

Expand Down Expand Up @@ -1170,7 +1174,9 @@ conflict.
.. envvar:: PYTHON_DISABLE_REMOTE_DEBUG

If this variable is set to a non-empty string, it disables the remote
debugging feature described in :pep:`768`.
debugging feature described in :pep:`768`. This includes both the functionality
to schedule code for execution in another process and the functionality to
receive code for execution in the current process.

See also the :option:`-X disable_remote_debug` command-line option.

Expand Down
4 changes: 3 additions & 1 deletion Doc/using/configure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -665,7 +665,9 @@ also be used to improve performance.
Deactivate remote debugging support described in :pep:`768` (enabled by default).
When this flag is provided the code that allows the interpreter to schedule the
execution of a Python file in a separate process as described in :pep:`768` is
not compiled.
not compiled. This includes both the functionality to schedule code to be executed
and the functionality to receive code to be executed.


.. versionadded:: next

Expand Down
46 changes: 28 additions & 18 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ PEP 768: Safe external debugger interface for CPython

:pep:`768` introduces a zero-overhead debugging interface that allows debuggers and profilers
to safely attach to running Python processes. This is a significant enhancement to Python's
debugging capabilities, bringing them in line with other major programming languages.
debugging capabilities allowing debuggers to forego unsafe alternatives.

The new interface provides safe execution points for attaching debugger code without modifying
the interpreter's normal execution path or adding runtime overhead. This enables tools to
Expand All @@ -124,23 +124,7 @@ Here's a simple example that inspects object types in a running Python process:
# Create a temporary script
with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as f:
script_path = f.name
f.write("""
import gc
import collections

# Collect all objects managed by the garbage collector
gc.collect()

# Count objects by type
type_counts = collections.Counter(type(obj).__name__
for obj in gc.get_objects())

# Print the most common types
print("Most common object types in process:")
for type_name, count in type_counts.most_common(10):
print(f" {type_name}: {count}")
""")

f.write("import my_debugger; my_debugger.connect(HOST_PROCESS)")
try:
# Execute in process with PID 1234
print("Behold! An offering:")
Expand All @@ -163,6 +147,32 @@ See :pep:`768` for more details.

(Contributed by Pablo Galindo Salgado, Matt Wozniski, and Ivona Stojanovic in :gh:`131591`.)

.. _whatsnew314-pep758:

PEP 758 – Allow except and except* expressions without parentheses
------------------------------------------------------------------

The :keyword:`except` and :keyword:`except* <except_star>` expressions now allow
parentheses to be omitted when there are multiple exception types and the ``as`` clause is not used.
For example the following expressions are now valid:

.. code-block:: python

try:
release_new_sleep_token_album()
except AlbumNotFound, SongsTooGoodToBeReleased:
print("Sorry, no new album this year.")

# The same applies to except* (for exception groups):
try:
release_new_sleep_token_album()
except* AlbumNotFound, SongsTooGoodToBeReleased:
print("Sorry, no new album this year.")

Check :pep:`758` for more details.

(Contributed by Pablo Galindo and Brett Cannon in :gh:`131831`.)


.. _whatsnew314-pep649:

Expand Down
1 change: 0 additions & 1 deletion Include/internal/pycore_global_objects_fini_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion Include/internal/pycore_global_strings.h
Original file line number Diff line number Diff line change
Expand Up @@ -458,7 +458,6 @@ struct _Py_global_strings {
STRUCT_FOR_ID(fromlist)
STRUCT_FOR_ID(fromtimestamp)
STRUCT_FOR_ID(fromutc)
STRUCT_FOR_ID(fsdecode)
STRUCT_FOR_ID(fset)
STRUCT_FOR_ID(func)
STRUCT_FOR_ID(future)
Expand Down
1 change: 0 additions & 1 deletion Include/internal/pycore_runtime_init_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 0 additions & 4 deletions Include/internal/pycore_unicodeobject_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 27 additions & 0 deletions Lib/test/test_sys.py
Original file line number Diff line number Diff line change
Expand Up @@ -1965,6 +1965,7 @@ def _supports_remote_attaching():
"Test only runs on Linux, Windows and MacOS")
@unittest.skipIf(sys.platform == "linux" and not _supports_remote_attaching(),
"Test only runs on Linux with process_vm_readv support")
@test.support.cpython_only
class TestRemoteExec(unittest.TestCase):
def tearDown(self):
test.support.reap_children()
Expand Down Expand Up @@ -2152,5 +2153,31 @@ def test_remote_exec_invalid_script_path(self):
with self.assertRaises(OSError):
sys.remote_exec(os.getpid(), "invalid_script_path")

def test_remote_exec_in_process_without_debug_fails_envvar(self):
"""Test remote exec in a process without remote debugging enabled"""
script = os_helper.TESTFN + '_remote.py'
self.addCleanup(os_helper.unlink, script)
with open(script, 'w') as f:
f.write('print("Remote script executed successfully!")')
env = os.environ.copy()
env['PYTHON_DISABLE_REMOTE_DEBUG'] = '1'

_, out, err = assert_python_failure('-c', f'import os, sys; sys.remote_exec(os.getpid(), "{script}")', **env)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Random thought - should we special-case (or disallow) calling remote_exec on ourselves? It seems like it could be useful, but there's probably overheads involved that could be skipped. Alternatively, just saying "it's not for this" and blocking it also seems fine to me, though it'd complicate the tests.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah, it actually requires no work whatsoever on our part, which is very cool! And we use it all the time for testing and developing since having to attach debuggers to two or even 3 processes is much much harder.

I am happy to keep it unless you feel strongly

self.assertIn(b"Remote debugging is not enabled", err)
self.assertEqual(out, b"")

def test_remote_exec_in_process_without_debug_fails_xoption(self):
"""Test remote exec in a process without remote debugging enabled"""
script = os_helper.TESTFN + '_remote.py'
self.addCleanup(os_helper.unlink, script)
with open(script, 'w') as f:
f.write('print("Remote script executed successfully!")')

_, out, err = assert_python_failure('-Xdisable-remote-debug', '-c', f'import os, sys; sys.remote_exec(os.getpid(), "{script}")')
self.assertIn(b"Remote debugging is not enabled", err)
self.assertEqual(out, b"")



if __name__ == "__main__":
unittest.main()
Loading
Loading