Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exception ignored in atexit callback / RuntimeError: can't create new thread at interpreter shutdown #271

Open
thmo opened this issue Apr 14, 2024 · 3 comments
Open

Exception ignored in atexit callback / RuntimeError: can't create new thread at interpreter shutdown #271

thmo opened this issue Apr 14, 2024 · 3 comments
Assignees
@RonnyPfannschmidt
Labels
bug

Comments

@thmo
Copy link

thmo commented Apr 14, 2024

Trying to update the Fedora RPM to 2.1.1, I see this:

+ cd execnet-2.1.1
+ PYTEST_SELECT='not test_popen_io[gevent-sys.executable]'
+ PYTEST_SELECT+=' and not [gevent-socket]'
+ PYTEST_SELECT+=' and not [eventlet-socket]'
+ PYTEST_SELECT+=' and not [python2.7]'
++ pwd
+ PYTHONPATH=.../execnet-2.1.1/src
+ py.test-3.12 -r s -k 'not test_popen_io[gevent-sys.executable] and not [gevent-socket] and not [eventlet-socket] and not [python2.7]' testing --timeout=30
============================= test session starts ==============================
platform linux -- Python 3.12.2, pytest-7.3.2, pluggy-1.2.0
rootdir: .../execnet-2.1.1
configfile: tox.ini
plugins: timeout-2.1.0, hypothesis-6.82.0, forked-1.6.0, anyio-3.7.0, cov-4.0.0
timeout: 30.0s
timeout method: signal
timeout func_only: False
collected 1095 items / 103 deselected / 992 selected

testing/test_basics.py ...............s.s.s.......                       [  2%]
testing/test_channel.py ..s...s...s...s...s...s...s...s...s...s...s...s. [  7%]
..s...s...s...s...s...s...s...s...s...s...s...s...s...s...s...s...s...s. [ 14%]
                                                                         [ 14%]
testing/test_gateway.py ..s...s...s...s...s...s.XXsX..s...s...s...s...s. [ 19%]
..s...s...s...s...s...s...s...s....ssss                                  [ 23%]
testing/test_multi.py ..s.xX                                             [ 24%]
testing/test_termination.py .s.xs                                        [ 24%]
testing/test_threadpool.py ......x.......                                [ 26%]
testing/test_basics.py .s.s.......                                       [ 27%]
testing/test_channel.py ..s...s...s...s...s...s...s...s...s...s...s...s. [ 32%]
..s...s...s...s...s...s...s...s...s...s...s...s...s...s...s...s...s...s. [ 39%]
                                                                         [ 39%]
testing/test_gateway.py ..s...s...s...s...s...s.XXsX..s...s...s...s...s. [ 44%]
..s...s...s...s...s...s...s...s....ss..                                  [ 48%]
testing/test_multi.py ..s.xX                                             [ 48%]
testing/test_termination.py .s.xs                                        [ 49%]
testing/test_threadpool.py ......x.......                                [ 50%]
testing/test_basics.py .s.s.......                                       [ 51%]
testing/test_channel.py .s..s..s..s..s..s..s..s..s..s..s..s..s..s..s..s. [ 56%]
.s..s..s..s..s..s..s..s..s..s..s..s..s..s.                               [ 60%]
testing/test_gateway.py .s..s..s..s..s..s.XsX.s..s..s..s..s..s..s..s..s. [ 65%]
.s..s..s..s....ssss                                                      [ 67%]
testing/test_multi.py .s.xx                                              [ 68%]
testing/test_termination.py .sxxs                                        [ 68%]
testing/test_threadpool.py ......x......s                                [ 69%]
testing/test_basics.py .ss.......                                        [ 70%]
testing/test_channel.py .s..s..s..s..s..s..s..s..s..s..s..s..s..s..s..s. [ 75%]
.s..s..s..s..s..s..s..s..s..s..s..s..s..s.                               [ 80%]
testing/test_gateway.py .s..s..s..s..s..s.XsX.s..s..s..s..s..s..s..s..s. [ 84%]
.s..s..s..s....ssss                                                      [ 86%]
testing/test_multi.py .s.xx                                              [ 87%]
testing/test_termination.py .sxxs                                        [ 87%]
testing/test_threadpool.py ......x......s                                [ 89%]
testing/test_basics.py .s.s............s..                               [ 91%]
testing/test_compatibility_regressions.py .                              [ 91%]
testing/test_gateway.py ........X.X......x                               [ 93%]
testing/test_multi.py ..............                                     [ 94%]
testing/test_rsync.py ...........                                        [ 95%]
testing/test_serializer.py ................                              [ 97%]
testing/test_termination.py ..s                                          [ 97%]
testing/test_xspec.py ...............X....ssss.                          [100%]

=========================== short test summary info ============================
SKIPPED [1] testing/test_basics.py:70: no pypy3 found
SKIPPED [4] testing/test_basics.py:179: no pypy3 found
SKIPPED [4] testing/test_basics.py:187: no pypy3 found
SKIPPED [4] testing/test_channel.py:19: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:24: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:28: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:36: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:42: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:49: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:62: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:68: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:73: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:78: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:86: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:94: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:104: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:119: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:132: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:157: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:174: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:192: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:210: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:249: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:256: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:273: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:287: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:310: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:325: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:333: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:347: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:363: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:371: no 'gspecs' option found
SKIPPED [4] testing/test_channel.py:384: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:35: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:41: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:44: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:49: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:53: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:69: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:78: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:100: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:113: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:146: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:186: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:197: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:201: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:205: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:209: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:214: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:220: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:231: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:248: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:265: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:398: no 'gspecs' option found
SKIPPED [4] testing/test_gateway.py:401: no 'gspecs' option found
SKIPPED [3] testing/test_gateway.py:571: can only run with main_thread_only
SKIPPED [3] testing/test_gateway.py:610: can only run with main_thread_only
SKIPPED [4] testing/test_multi.py:22: no 'gspecs' option found
SKIPPED [4] testing/test_termination.py:24: no pypy3 found
SKIPPED [4] testing/test_termination.py:111: no pypy3 found
SKIPPED [2] testing/test_threadpool.py:196: can only run with threading
SKIPPED [1] testing/test_basics.py:217: no pypy3 found
SKIPPED [1] testing/test_basics.py:233: no pypy3 found
SKIPPED [1] testing/test_basics.py:433: no pypy3 found
SKIPPED [1] testing/test_termination.py:84: no pypy3 found
SKIPPED [1] testing/test_xspec.py:198: no 'gspecs' option found
SKIPPED [1] testing/test_xspec.py:206: Vagrant binary not in PATH
SKIPPED [1] testing/test_xspec.py:227: no 'gspecs' option found
SKIPPED [1] testing/test_xspec.py:238: no 'gspecs' option found
= 715 passed, 245 skipped, 103 deselected, 17 xfailed, 15 xpassed in 61.38s (0:01:01) =
Exception ignored in atexit callback: <bound method Group._cleanup_atexit of <Group []>>
Traceback (most recent call last):
  File ".../execnet-2.1.1/src/execnet/multi.py", line 208, in _cleanup_atexit
    self.terminate(timeout=1.0)
  File ".../execnet-2.1.1/src/execnet/multi.py", line 237, in terminate
    safe_terminate(
  File ".../execnet-2.1.1/src/execnet/multi.py", line 345, in safe_terminate
    reply = workerpool.spawn(termkill, termfunc, killfunc)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../execnet-2.1.1/src/execnet/gateway_base.py", line 452, in spawn
    self.execmodel.start(self._perform_spawn, (reply,))
  File ".../execnet-2.1.1/src/execnet/gateway_base.py", line 155, in start
    _thread.start_new_thread(func, args)
RuntimeError: can't create new thread at interpreter shutdown
Exception ignored in atexit callback: <bound method Group._cleanup_atexit of <Group []>>
Traceback (most recent call last):
  File ".../execnet-2.1.1/src/execnet/multi.py", line 208, in _cleanup_atexit
    self.terminate(timeout=1.0)
  File ".../execnet-2.1.1/src/execnet/multi.py", line 237, in terminate
    safe_terminate(
  File ".../execnet-2.1.1/src/execnet/multi.py", line 345, in safe_terminate
    reply = workerpool.spawn(termkill, termfunc, killfunc)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../execnet-2.1.1/src/execnet/gateway_base.py", line 452, in spawn
    self.execmodel.start(self._perform_spawn, (reply,))
  File ".../execnet-2.1.1/src/execnet/gateway_base.py", line 155, in start
    _thread.start_new_thread(func, args)
RuntimeError: can't create new thread at interpreter shutdown
Exception ignored in atexit callback: <bound method Group._cleanup_atexit of <Group []>>
Traceback (most recent call last):
  File ".../execnet-2.1.1/src/execnet/multi.py", line 208, in _cleanup_atexit
    self.terminate(timeout=1.0)
  File ".../execnet-2.1.1/src/execnet/multi.py", line 237, in terminate
    safe_terminate(
  File ".../execnet-2.1.1/src/execnet/multi.py", line 345, in safe_terminate
    reply = workerpool.spawn(termkill, termfunc, killfunc)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../execnet-2.1.1/src/execnet/gateway_base.py", line 452, in spawn
    self.execmodel.start(self._perform_spawn, (reply,))
  File ".../execnet-2.1.1/src/execnet/gateway_base.py", line 155, in start
    _thread.start_new_thread(func, args)
RuntimeError: can't create new thread at interpreter shutdown

Not sure what exactly causes it.
@RonnyPfannschmidt
Copy link
Member

i wont be able to deep debug before next week

does this only happen on 3.12?

i suspect a new stafety is in place, and i think we need to switch to a better error for the details

@thmo
Copy link
Author

thmo commented Apr 14, 2024

Not seen on Fedora 38 (Python 3.11.8, pytest-7.2.2, pluggy-1.0.0, timeout-2.1.0).

@RonnyPfannschmidt
Copy link
Member

as per python/cpython#104690 its new in python3.12

the issue description indicates it was wrong to run such code in atexit to begin with

we need a bufix

@RonnyPfannschmidt RonnyPfannschmidt self-assigned this Apr 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@RonnyPfannschmidt RonnyPfannschmidt

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@RonnyPfannschmidt @thmo