Dynamically calling async fixture causes a runtime error saying "This event loop is already running"

[ykuzma1]

There seems to be a bug with how request.getfixturevalue(argname) interacts with pytest-asyncio. Calling the function leads to a runtime error saying the event loop is already running. If you change it from being dynamically called to being fixed in the function definition, it works as expected.

Platform Info:

platform win32 -- Python 3.7.2, pytest-4.2.0, py-1.7.0, pluggy-0.8.1
plugins: asyncio-0.11.0.dev0

Test fixture that can be used in both dynamic/fixed cases:

@pytest.fixture
async def async_fixture():
    yield 'Hi from async_fixture()!'

Successful fixed function argument fixture test:

@pytest.mark.asyncio
async def test_async_fixture_fixed(async_fixture):
    assert async_fixture == 'Hi from async_fixture()!'

Failed dynamic fixture test:

@pytest.mark.asyncio
async def test_async_fixture_dynamic(request):
    async_fixture = request.getfixturevalue('async_fixture')
    assert async_fixture == 'Hi from async_fixture()!'

Failed test trace-back:

================================== FAILURES ===================================
_________________________ test_async_fixture_dynamic __________________________

request = <FixtureRequest for <Function test_async_fixture_dynamic>>

    @pytest.mark.asyncio
    async def test_async_fixture_dynamic(request):
>       async_fixture = request.getfixturevalue('async_fixture')

tests\test_app_factory.py:52: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv\lib\site-packages\_pytest\fixtures.py:478: in getfixturevalue
    return self._get_active_fixturedef(argname).cached_result[0]
venv\lib\site-packages\_pytest\fixtures.py:501: in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
venv\lib\site-packages\_pytest\fixtures.py:586: in _compute_fixture_value
    fixturedef.execute(request=subrequest)
venv\lib\site-packages\_pytest\fixtures.py:881: in execute
    return hook.pytest_fixture_setup(fixturedef=self, request=request)
venv\lib\site-packages\pluggy\hooks.py:284: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
venv\lib\site-packages\pluggy\manager.py:68: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
venv\lib\site-packages\pluggy\manager.py:62: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
venv\lib\site-packages\_pytest\fixtures.py:923: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
venv\lib\site-packages\_pytest\fixtures.py:782: in call_fixture_func
    res = fixturefunc(**kwargs)
..\pytest-asyncio\pytest_asyncio\plugin.py:97: in wrapper
    return loop.run_until_complete(setup())
C:\Users\ykuzm\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py:571: in run_until_complete
    self.run_forever()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_WindowsSelectorEventLoop running=False closed=False debug=False>

    def run_forever(self):
        """Run until stop() is called."""
        self._check_closed()
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

[ykuzma1]

I'd be happy to put in a pull request. I mostly want to use this space for ideas, because I've tried everything I can think of. Ideas welcome! I'll put my couple attempts below.

[ykuzma1]

My thinking is that it fails because plugin.py calls loop.run_until_complete(setup()) during the middle of async tests being run on the event loop already. So calling run_until_complete tries adding the async fixture onto the already running loop - causing the error. It sounds like iPython had the same issue. So I tried some workarounds they suggested to replace that line:

Causes infinite loop:

return asyncio.run_coroutine_threadsafe(setup(), loop).result()

Causes asyncio.base_futures.InvalidStateError: Result is not ready. error:

return asyncio.ensure_future(setup()).result()

Another infinite loop:

from concurrent.futures import ThreadPoolExecutor
loop = asyncio.new_event_loop()
ThreadPoolExecutor().submit(loop.run_forever)
return asyncio.run_coroutine_threadsafe(setup(), loop).result()

[ykuzma1]

Funny enough I got one workaround to work while I was going back through the iPython thread:

from concurrent.futures import ThreadPoolExecutor
pool = ThreadPoolExecutor(1)
loop = asyncio.new_event_loop()
pool.submit(asyncio.set_event_loop, loop).result()
return pool.submit(loop.run_until_complete, setup()).result()

Will start working on a proper pull request.

[senciucserban]

Any news? 😢

[brianmaissy]

any updates on this?

[dazza-codes]

Sorry this doesn't help solve this for pytest-asyncio, but I struggled with this for a while, including trying custom fixtures with module scope and nest_asyncio, but eventually a solution was to drop all the pytest-asyncio decorators and async def tests to go back to vanilla non-async test functions with a non-async entry point that runs the rest of the async coroutines. The only trick to executing the integration tests for the async coroutines was to create a main like entry point that all the tests pass through and that function gets a new event loop every time, e.g.

def run_aync_main(*args, **kwargs):
    main_loop = asyncio.new_event_loop()
    try:
        main_loop.run_until_complete(any_async_entry_point(*args, *kwargs))
    finally:
        main_loop.stop()
        main_loop.close()

Any non-async function that calls a run_until_complete does not need to be run by pytest-asyncio decorators and it should not conflict across tests when it gets a new loop. Other unit tests on async coroutines with regular await statements work OK with pytest-asyncio.

[PidgeyBE]

In our case it worked by simply making the dynamic fixture sync:

def test_async_fixture_dynamic(request, event_loop):
    async_fixture = request.getfixturevalue('async_fixture')
    assert async_fixture == 'Hi from async_fixture()!'

[roganov]

Got this error when I tried to upgrade to 0.14.0 from 0.10.0. Rolled back for now.

[dorindivo1]

Got this error in 0.15.1 also, any update ?

[ReznikovRoman]

got this error in 0.18.3

[seifertm]

As of v0.18.3 this error could be caused by an unexpected interaction with other pytest plugins that manipulate the event loop.

@ReznikovRoman Can you provide a reproducible example?

[vadim-su]

I got the same issue today.

Code

@fixture
async def client():
    async with httpx.AsyncClient() as http_client:
        yield HavenClient(
            http_client,
            TEST_APIKEY,
        )


@fixture
async def client_without_apikey():
    async with httpx.AsyncClient() as http_client:
        yield HavenClient(
            http_client,
        )

@pytest.mark.parametrize('client_fixture,expectation', [
    ('client', do_not_raise()),
    ('client_without_apikey', pytest.raises(errors.ApikeyNotSetError)),
])
async def test_get_user_settings(client_fixture, expectation, request: pytest.FixtureRequest):
    client: HavenClient = request.getfixturevalue(client_fixture)
    with expectation:
        settings = await client.get_user_settings()
        assert settings

    @pytest.mark.parametrize('client_fixture,expectation', [
        ('client', do_not_raise()),
        ('client_without_apikey', pytest.raises(errors.ApikeyNotSetError)),
    ])
    async def test_get_user_settings(client_fixture, expectation, request: pytest.FixtureRequest):
>       client: HavenClient = request.getfixturevalue(client_fixture)

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def _check_running(self):
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

/usr/local/lib/python3.10/asyncio/base_events.py:582: RuntimeError

Full output

pytest -k test_get_user_settings                     
======================================================================================== test session starts ========================================================================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
rootdir: /workspaces/haven, configfile: pyproject.toml, testpaths: tests
plugins: anyio-3.6.1, asyncio-0.18.3
asyncio: mode=auto
collected 11 items / 9 deselected / 2 selected                                                                                                                                                      

tests/test_client.py FF                                                                                                                                                                       [100%]

============================================================================================= FAILURES ==============================================================================================
____________________________________________________________________________ test_get_user_settings[client-expectation0] ____________________________________________________________________________

client_fixture = 'client', expectation = <contextlib.suppress object at 0x7f33c81ee9b0>, request = <FixtureRequest for <Function test_get_user_settings[client-expectation0]>>

    @pytest.mark.parametrize('client_fixture,expectation', [
        ('client', do_not_raise()),
        ('client_without_apikey', pytest.raises(errors.ApikeyNotSetError)),
    ])
    async def test_get_user_settings(client_fixture, expectation, request: pytest.FixtureRequest):
>       client: HavenClient = request.getfixturevalue(client_fixture)

tests/test_client.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:554: in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:573: in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:659: in _compute_fixture_value
    fixturedef.execute(request=subrequest)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1057: in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_hooks.py:265: in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_manager.py:80: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1111: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:890: in call_fixture_func
    fixture_result = fixturefunc(**kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pytest_asyncio/plugin.py:293: in _asyncgen_fixture_wrapper
    result = event_loop.run_until_complete(setup())
/usr/local/lib/python3.10/asyncio/base_events.py:622: in run_until_complete
    self._check_running()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def _check_running(self):
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

/usr/local/lib/python3.10/asyncio/base_events.py:582: RuntimeError
____________________________________________________________________ test_get_user_settings[client_without_apikey-expectation1] _____________________________________________________________________

client_fixture = 'client_without_apikey', expectation = <_pytest.python_api.RaisesContext object at 0x7f33c81ee950>
request = <FixtureRequest for <Function test_get_user_settings[client_without_apikey-expectation1]>>

    @pytest.mark.parametrize('client_fixture,expectation', [
        ('client', do_not_raise()),
        ('client_without_apikey', pytest.raises(errors.ApikeyNotSetError)),
    ])
    async def test_get_user_settings(client_fixture, expectation, request: pytest.FixtureRequest):
>       client: HavenClient = request.getfixturevalue(client_fixture)

tests/test_client.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:554: in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:573: in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:659: in _compute_fixture_value
    fixturedef.execute(request=subrequest)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1057: in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_hooks.py:265: in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_manager.py:80: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1111: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:890: in call_fixture_func
    fixture_result = fixturefunc(**kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pytest_asyncio/plugin.py:293: in _asyncgen_fixture_wrapper
    result = event_loop.run_until_complete(setup())
/usr/local/lib/python3.10/asyncio/base_events.py:622: in run_until_complete
    self._check_running()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def _check_running(self):
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

/usr/local/lib/python3.10/asyncio/base_events.py:582: RuntimeError
====================================================================================== short test summary info ======================================================================================
FAILED tests/test_client.py::test_get_user_settings[client-expectation0] - RuntimeError: This event loop is already running
FAILED tests/test_client.py::test_get_user_settings[client_without_apikey-expectation1] - RuntimeError: This event loop is already running
================================================================================== 2 failed, 9 deselected in 0.38s ==================================================================================
sys:1: RuntimeWarning: coroutine '_wrap_asyncgen.<locals>._asyncgen_fixture_wrapper.<locals>.setup' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

I'll try to get a clean example little bit later.

[vadim-su]

I made a simple example

plugins: anyio-3.6.1, asyncio-0.18.3
asyncio: mode=auto

test_test.py

import pytest


@pytest.fixture
async def test_value():
    return 'test'


async def test_value_is_test(request: pytest.FixtureRequest):
    tv = request.getfixturevalue('test_value')
    assert tv == 'test'

Run output

pytest -k tests/test_test.py 
======================================================================================== test session starts ========================================================================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
rootdir: /workspaces/haven, configfile: pyproject.toml, testpaths: tests
plugins: anyio-3.6.1, asyncio-0.18.3
asyncio: mode=auto
collected 12 items / 11 deselected / 1 selected                                                                                                                                                     

tests/test_test.py F                                                                                                                                                                          [100%]

============================================================================================= FAILURES ==============================================================================================
________________________________________________________________________________________ test_value_is_test _________________________________________________________________________________________

request = <FixtureRequest for <Function test_value_is_test>>

    async def test_value_is_test(request: pytest.FixtureRequest):
>       tv = request.getfixturevalue('test_value')

tests/test_test.py:10: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:554: in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:573: in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:659: in _compute_fixture_value
    fixturedef.execute(request=subrequest)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1057: in execute
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_hooks.py:265: in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pluggy/_manager.py:80: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:1111: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/_pytest/fixtures.py:890: in call_fixture_func
    fixture_result = fixturefunc(**kwargs)
/home/vscode/.local/share/pdm/venvs/haven-eEiRBr5u-3.10/lib/python3.10/site-packages/pytest_asyncio/plugin.py:309: in _async_fixture_wrapper
    return event_loop.run_until_complete(setup())
/usr/local/lib/python3.10/asyncio/base_events.py:622: in run_until_complete
    self._check_running()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def _check_running(self):
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

/usr/local/lib/python3.10/asyncio/base_events.py:582: RuntimeError
====================================================================================== short test summary info ======================================================================================
FAILED tests/test_test.py::test_value_is_test - RuntimeError: This event loop is already running
================================================================================= 1 failed, 11 deselected in 0.25s ==================================================================================
sys:1: RuntimeWarning: coroutine '_wrap_async.<locals>._async_fixture_wrapper.<locals>.setup' was never awaited

@seifertm

[seifertm]

Thanks for the example @suharnikov. I managed to reproduce the error.

When a fixture is requested dynamically, it is looked up in pytest's fixture cache first. If it cannot be found in the cache pytest evaluates the fixture function. Since the async fixture coroutine has a synchronous wrapper around it that calls loop.run_until_complete that wrapper will fail to execute, because the event loop from the async test function's wrapper is already running.

[seifertm]

A fix would require that the fixture wrapper can decide dynamically whether it is run asynchronously in an event loop or synchronously. However, the fixture wrapper itself needs to be synchronous, because that's what pytest expects. If an event loop is already running, the fixture wrapper needs to submit a task to the event loop and block execution until that task has finished.

I'm not aware of a way to await a task from a synchronous function. With the current state of pytest-asyncio, I don't see how this bug can be solved. Suggestions are welcome.

#235 would probably solve this issue.

[block2busted]

I solved this problem by adding nest_asyncio.apply() on the top on contest.py:

import nest_asyncio
nest_asyncio.apply()