Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable reuse of current_app #90

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

alexdong
Copy link

The current approach crashes the worker processes because the sqlalchemy connection isn't initiated properly.

The sympton is that the first job was processed beautifully but subsequent jobs failed with the following stack trace:

Exception during reset or similar
Traceback (most recent call last):
File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 739, in _r
ollback_impl
self.engine.dialect.do_rollback(self.connection)
File "app/lib/python3.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 22
41, in do_rollback
dbapi_connection.rollback()
MySQLdb._exceptions.OperationalError: (2006, 'MySQL server has gone away')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "app/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 680, in _fin
alize_fairy
fairy._reset(pool)
File "app/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 865, in _res
et
self._reset_agent.rollback()
File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1705, in r
ollback
self._do_rollback()
File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1743, in _
do_rollback
self.connection._rollback_impl()
File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 741, in _r
ollback_impl
self._handle_dbapi_exception(e, None, None, None, None)
File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1466, in _
handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "app/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 399, in ra
ise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "app/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 153, in re
raise
raise value.with_traceback(tb)
File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 739, in _r
ollback_impl
self.engine.dialect.do_rollback(self.connection)
File "app/lib/python3.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 22
41, in do_rollback
dbapi_connection.rollback()
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2006, 'MySQL server has gone a
way')
(Background on this error at: http://sqlalche.me/e/e3q8)

The FlaskJob.load_app has the good intention to reuse the current_app but IMHO it's making too many assumptions of how the Flask app is structured.

Since RQ is mainly used in backend processes where performance is not the highest priority, I'd like to suggest we call create_app every time a job is performed. This provides the maximum guarantee on the correctness of the appcontext.

The current approach crashes the worker because the sqlalchemy connection isn't initiated properly.
matt-takumi added a commit to TakumiHQ/Flask-RQ2 that referenced this pull request Feb 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant