Cannot run migration from Django on Docker to MariaDB installed on host - sql

I created a Django project running on Docker, and I successfully connected from the container to the host MariaDB server as an URL host.docker.internal:3306 with the command sudo docker run -p 8000:8000 --add-host=host.docker.internal:host-gateway -d kms1212/jsis:latest. But due to several problems(I can't remember why I did it), I removed the original database, created a new database, and modified settings.py. Then I ran the modified Docker image that is built from Dockerfile:
FROM ubuntu:22.04
ENV PYTHONUNBUFFERED 1
RUN apt-get -y update
RUN apt-get -y install build-essential python-is-python3 pip mysql-client libmysqlclient-dev
RUN mkdir /usr/local/workdir
WORKDIR /usr/local/workdir
COPY requirements.txt .
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
COPY jsis .
EXPOSE 8000
CMD ["bash", "-c", "python manage.py makemigrations && python manage.py showmigrations && python manage.py migrate && gunicorn --bind 0.0.0.0:8000 -k uvicorn.workers.UvicornWorker jsis.asgi:application"]
I dropped and recreated the database and checked that no old constraints and tables were left, but despite my effort, I got some error messages from the container log like this eventually:
No changes detected
admin
[ ] 0001_initial
[ ] 0002_logentry_remove_auto_add
[ ] 0003_logentry_add_action_flag_choices
auth
[ ] 0001_initial
[ ] 0002_alter_permission_name_max_length
[ ] 0003_alter_user_email_max_length
[ ] 0004_alter_user_username_opts
[ ] 0005_alter_user_last_login_null
[ ] 0006_require_contenttypes_0002
[ ] 0007_alter_validators_add_error_messages
[ ] 0008_alter_user_username_max_length
[ ] 0009_alter_user_last_name_max_length
[ ] 0010_alter_group_name_max_length
[ ] 0011_update_proxy_permissions
[ ] 0012_alter_user_first_name_max_length
contenttypes
[ ] 0001_initial
[ ] 0002_remove_content_type_name
sessions
[ ] 0001_initial
sites
[ ] 0001_initial
[ ] 0002_alter_domain_unique
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions, sites
Running migrations:
Applying contenttypes.0001_initial... OK
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/mysql/base.py", line 75, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python3.10/dist-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/usr/local/lib/python3.10/dist-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/usr/local/lib/python3.10/dist-packages/MySQLdb/connections.py", line 254, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (1005, 'Can\'t create table `jsisdb`.`django_admin_log` (errno: 150 "Foreign key constraint is incorrectly formed")')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/workdir/manage.py", line 22, in <module>
main()
File "/usr/local/workdir/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 402, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 448, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 96, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/commands/migrate.py", line 349, in handle
post_migrate_state = executor.migrate(
File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 135, in migrate
state = self._migrate_all_forwards(
File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
state = self.apply_migration(
File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 249, in apply_migration
with self.connection.schema_editor(
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/schema.py", line 164, in __exit__
self.execute(sql)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/schema.py", line 199, in execute
cursor.execute(sql, params)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 103, in execute
return super().execute(sql, params)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
with self.db.wrap_database_errors:
File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/mysql/base.py", line 75, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python3.10/dist-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/usr/local/lib/python3.10/dist-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/usr/local/lib/python3.10/dist-packages/MySQLdb/connections.py", line 254, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1005, 'Can\'t create table `jsisdb`.`django_admin_log` (errno: 150 "Foreign key constraint is incorrectly formed")')
(+ Edit)And this is model code:
class UserPermissions(models.IntegerChoices):
USER = 0
HQ = 1
TEACHER = 2
OTHER = 3
class UserAccountManager(BaseUserManager):
def create_user(self, username, email, visiblename, agreement_status, password=None):
if not email:
raise ValueError('Users must have an email address')
if not username:
raise ValueError('Users must have a username')
if not visiblename:
raise ValueError('Users must have a visible name')
user = self.model(
username=username,
email=self.normalize_email(email),
visiblename=visiblename,
agreement_status=agreement_status,
)
user.set_password(password)
user.save(using=self._db)
return user
def create_superuser(self, username, email, visiblename, agreement_status, password):
user = self.create_user(
username=username,
email=self.normalize_email(email),
visiblename=visiblename,
agreement_status=agreement_status,
password=password,
)
user.is_admin = True
user.permission = UserPermissions.OTHER
user.save(using=self._db)
return user
class UserAccount(AbstractBaseUser):
userid = models.AutoField(primary_key=True)
username = models.CharField(max_length=20, unique=True)
visiblename = models.CharField(max_length=40, unique=True)
email = models.EmailField(max_length=255, unique=True)
permission = models.IntegerField(default=UserPermissions.USER, choices=UserPermissions.choices)
agreement_status = models.JSONField()
date_joined = models.DateTimeField(auto_now_add=True)
date_lastlogin = models.DateTimeField(auto_now=True)
is_active = models.BooleanField(default=True)
is_blocked = models.BooleanField(default=False)
is_blocked_anon = models.BooleanField(default=False)
block_reason = models.TextField(default="", blank=True)
is_admin = models.BooleanField(default=False)
USERNAME_FIELD = 'username'
EMAIL_FIELD = 'email'
REQUIRED_FIELDS = ['email', 'visiblename']
objects = UserAccountManager()
def __str__(self):
return self.username
def has_perm(self, perm, obj=None):
return self.is_admin
def has_module_perms(self, app_label):
return True
So, what can I do about this problem? Should I reset all the MariaDB databases? Or are there some better solutions?

Related

Running into uvloop issues with Database queries from Rasa-X?

I'm trying to make a simple query to my amazon neptune database, from Rasa-x.
Here is the code from my actions.py:
class ActionQueryDietary(Action):
def name(self) -> Text:
return "action_query_dietary"
def run(self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
available = False
restaurant = "XXXX"
dietaryQuestion=tracker.get_slot('dietaryQuestion')
g, remoteConn = kb.openConnection()
dietary = kb.getDietary(g, restaurant, dietaryQuestion)
if(dietary=="Yes"):
available = True
if(available==True):
dispatcher.utter_message(text="According to our knowledge base, {} is on the menu").format(dietaryQuestion)
else:
dispatcher.utter_message(text="Sorry, according to our knowledge base, we don't have this option on the menu")
kb.closeConnection(remoteConn)
return []
and here is the code from knowledgebase.py:
def getDietary(g, restaurant, dietary):
properties = queryRestaurantProperties(g, restaurant)
result = properties[dietary]
print(result)
return result
but any query to the knowledgebase results in this error:
2020-10-22T18:01:22.347345241Z File "/opt/venv/lib/python3.7/site-packages/sanic/app.py", line 973, in handle_request
2020-10-22T18:01:22.347351643Z response = await response
2020-10-22T18:01:22.347357446Z File "/app/rasa_sdk/endpoint.py", line 102, in webhook
2020-10-22T18:01:22.347363522Z result = await executor.run(action_call)
2020-10-22T18:01:22.347369398Z File "/app/rasa_sdk/executor.py", line 392, in run
2020-10-22T18:01:22.347375473Z events = action(dispatcher, tracker, domain)
2020-10-22T18:01:22.347381348Z File "/app/actions/actions.py", line 33, in run
2020-10-22T18:01:22.347387063Z dietary = kb.getDietary(g, restaurant, dietaryQuestion)
2020-10-22T18:01:22.347392835Z File "/app/actions/knowledgebase.py", line 117, in getDietary
2020-10-22T18:01:22.347399112Z properties = queryRestaurantProperties(g, restaurant)
2020-10-22T18:01:22.347405111Z File "/app/actions/knowledgebase.py", line 86, in queryRestaurantProperties
2020-10-22T18:01:22.347411869Z result = g.V().has('name', restaurant).valueMap().next()
2020-10-22T18:01:22.347418048Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/process/traversal.py", line 89, in next
2020-10-22T18:01:22.347424293Z return self.__next__()
2020-10-22T18:01:22.347430069Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/process/traversal.py", line 48, in __next__
2020-10-22T18:01:22.347436333Z self.traversal_strategies.apply_strategies(self)
2020-10-22T18:01:22.347441940Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/process/traversal.py", line 573, in apply_strategies
2020-10-22T18:01:22.347447667Z traversal_strategy.apply(traversal)
2020-10-22T18:01:22.347453031Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/driver/remote_connection.py", line 149, in apply
2020-10-22T18:01:22.347459352Z remote_traversal = self.remote_connection.submit(traversal.bytecode)
2020-10-22T18:01:22.347465069Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/driver/driver_remote_connection.py", line 55, in submit
2020-10-22T18:01:22.347486749Z result_set = self._client.submit(bytecode)
2020-10-22T18:01:22.347493788Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/driver/client.py", line 127, in submit
2020-10-22T18:01:22.347499424Z return self.submitAsync(message, bindings=bindings).result()
2020-10-22T18:01:22.347505093Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/driver/client.py", line 146, in submitAsync
2020-10-22T18:01:22.347511092Z return conn.write(message)
2020-10-22T18:01:22.347516610Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/driver/connection.py", line 55, in write
2020-10-22T18:01:22.347522673Z self.connect()
2020-10-22T18:01:22.347529987Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/driver/connection.py", line 45, in connect
2020-10-22T18:01:22.347536097Z self._transport.connect(self._url, self._headers)
2020-10-22T18:01:22.347542222Z File "/opt/venv/lib/python3.7/site-packages/gremlin_python/driver/tornado/transport.py", line 36, in connect
2020-10-22T18:01:22.347547822Z lambda: websocket.websocket_connect(url))
2020-10-22T18:01:22.347553477Z File "/opt/venv/lib/python3.7/site-packages/tornado/ioloop.py", line 571, in run_sync
2020-10-22T18:01:22.347559295Z self.start()
2020-10-22T18:01:22.347564864Z File "/opt/venv/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 132, in start
2020-10-22T18:01:22.347570978Z self.asyncio_loop.run_forever()
2020-10-22T18:01:22.347576693Z File "uvloop/loop.pyx", line 1351, in uvloop.loop.Loop.run_forever
2020-10-22T18:01:22.347582342Z File "uvloop/loop.pyx", line 484, in uvloop.loop.Loop._run
2020-10-22T18:01:22.347588222Z RuntimeError: Cannot run the event loop while another loop is running
I tried using nest_asyncio.apply, but that resulted in this error:
ValueError: Can't patch loop of type <class 'uvloop.Loop'>
which according to the docs is just a rule.
So I don't really know how to proceed?
Adding my comment above as an answer. In some cases it is necessary to downlevel the version of Tornado being used. There are some issues that you can sometimes run into if the event loop is already running when someone else tries to create it. For now, down leveling to Tornado 4.5.1 with Gremlin Python should resolve any issues.

`flush` fails when initilized from object

I'm currently running into a permanent while using the flush (or commit as it depends on it) method from SQLAlchemy session under Flask Alchemy
The flush part always failed with sqlalchemy.exc.ResourceClosedError: This transaction is closed (full stack error bellow). While running directly an insert call from engine works as well as retrieving data using the query builder.
Also, deleting an item is working correctly (through session.delete(model) and session.commit())
Here is the code failing:
roles_put = Blueprint('roles_put', __name__)
#roles_put.route('<role_id>', methods=['PUT'])
def role_update(role_id):
role = Role.query.get(role_id)
if not role:
role = Role.query.filter_by(name=role_id).first()
if not role:
raise IDNotFoundError()
print(role)
role.set_data(
request.form,
[
'name', 'manage_user', 'manage_video', 'manage_comment', 'manage_avatar', 'manage_channel', 'manage_reward',
'manage_role', 'manage_top', 'manage_calendar', 'manage_setting', 'validate_video', 'moderate_comment',
]
)
MainAPI.db.session.add(role)
MainAPI.db.session.flush()
MainAPI.db.session.commit()
# if not role.save():
# raise UpdateError()
return jsonify(role.serialize())
SQLAlchemy initialize through:
app = Flask('Name')
def init_db(app, config):
"""
Init SQLAlchemy DB
"""
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://%s:%s#%s/%s' % (
config.get('database', 'user'),
config.get('database', 'password'),
config.get('database', 'host'),
config.get('database', 'database')
)
print(app.config['SQLALCHEMY_DATABASE_URI'])
MainApi.db = SQLAlchemy(app)
Role Model:
class RoleModel(MainApi.db.Model):
__tablename__ = 'roles'
LOCKED_ROLE_NAMES = ['guest', 'admin', 'logged', 'public']
id = MainApi.db.Column(MainApi.db.Integer, primary_key=True, unique=True, autoincrement=True)
name = MainApi.db.Column(MainApi.db.String(40), nullable=False, unique=True)
# manage rights
manage_user = MainApi.db.Column(MainApi.db.Boolean, nullable=False, default=False)
moderate_comment = MainApi.db.Column(MainApi.db.Boolean, nullable=False, default=False)
created_at = MainApi.db.Column(MainApi.db.DateTime, nullable=False, default=datetime.utcnow)
last_updated_at = MainApi.db.Column(MainApi.db.DateTime, nullable=True)
created_by = MainApi.db.Column(
MainApi.db.Integer,
MainApi.db.ForeignKey(
'users.id', ondelete='RESTRICT', onupdate='CASCADE'
),
nullable=True
)
last_updated_by = MainApi.db.Column(
MainApi.db.Integer,
MainApi.db.ForeignKey(
'users.id', ondelete='CASCADE', onupdate='CASCADE'
), nullable=True
)
users = MainApi.db.relationship(
'UserModel', foreign_keys='UserModel.role_id',
back_populates='role'
)
#staticmethod
def is_allowed(action, role):
"""
Check if user having role can make action
:param action: action name
:type action: str
:param role: user role
:type role: int|str|RoleModel
:return:
"""
if isinstance(role, str):
role = RoleModel.query.filter_by(name=role).first()
elif isinstance(role, int):
role = RoleModel.query.get(role)
if not isinstance(role, RoleModel):
raise Exception
return getattr(role, action.strip().replace(' ', '_'))
#staticmethod
def get_role_id(role):
"""
Check if user having role can make action
:param role: user role
:type role: int|str|RoleModel
:return: role id
:rtype: int
"""
if isinstance(role, str):
role = RoleModel.query.filter_by(name=role).first()
elif isinstance(role, int):
role = RoleModel.query.get(role)
if not isinstance(role, RoleModel):
raise Exception
return role.id
def serialize(self):
users = [u.serialize() for u in self.users] if self.users else []
return {
'id': self.id,
'name': self.name,
'manage_user': self.manage_user,
'moderate_comment': self.moderate_comment,
'created_at': self.created_at,
'last_updated_at': self.last_updated_at,
'created_by': self.created_by,
'last_updated_by': self.last_updated_by,
'users': users,
}
#event.listens_for(RoleModel.name, 'set', propagate=True)
def before_set_name(_target, value, old, _initiator):
print(_initiator)
print(request.url)
if request and 'roles/init' not in request.url:
if old in RoleModel.LOCKED_ROLE_NAMES or value in RoleModel.LOCKED_ROLE_NAMES:
raise UnauthorizedError()
#event.listens_for(RoleModel, 'before_insert', propagate=True)
def receive_before_insert(_mapper, _connection, target):
user = Registry.registered('current-user-id')
target.created_at = datetime.utcnow()
if user:
target.created_by = user
#event.listens_for(RoleModel, 'before_update', propagate=True)
def receive_before_update(_mapper, _connection, target):
user = Registry.registered('current-user-id')
target.updated_at = datetime.utcnow()
if user:
target.updated_by = user
Full error stack:
INFO:werkzeug: * Running on http://localhost:8080/ (Press CTRL+C to quit)
<RoleModel 5>
<sqlalchemy.orm.attributes.Event object at 0x7f5baedf8788>
http://localhost:8080/MainApi/roles/5
ERROR:flask.app:Exception on /MainApi/roles/5 [PUT]
Traceback (most recent call last):
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1177, in _execute_context
conn = self._revalidate_connection()
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 469, in _revalidate_connection
raise exc.ResourceClosedError("This Connection is closed")
sqlalchemy.exc.ResourceClosedError: This Connection is closed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2556, in _flush
flush_context.execute()
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/unitofwork.py", line 589, in execute
uow,
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 236, in save_obj
update,
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 978, in _emit_update_statements
statement, multiparams
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
distilled_params,
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
e, util.text_type(statement), parameters, None, None
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1177, in _execute_context
conn = self._revalidate_connection()
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 469, in _revalidate_connection
raise exc.ResourceClosedError("This Connection is closed")
sqlalchemy.exc.StatementError: (sqlalchemy.exc.ResourceClosedError) This Connection is closed
[SQL: UPDATE roles SET name=%s, manage_video=%s WHERE roles.id = %s]
[parameters: [{'name': 'admin2', 'manage_video': '0', 'roles_id': 5}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/flask/app.py", line 2311, in wsgi_app
response = self.full_dispatch_request()
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/flask/app.py", line 1834, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/flask/app.py", line 1737, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/flask/_compat.py", line 36, in reraise
raise value
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/flask/app.py", line 1832, in full_dispatch_request
rv = self.dispatch_request()
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/flask/app.py", line 1818, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/titouan/main_api/python-api/main_api/routes/roles/put.py", line 28, in role_update
MainApi.db.session.flush()
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/scoping.py", line 162, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2458, in flush
self._flush(objects)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2596, in _flush
transaction.rollback(_capture_exception=True)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 79, in __exit__
compat.reraise(type_, value, traceback)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 129, in reraise
raise value
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2596, in _flush
transaction.rollback(_capture_exception=True)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 509, in rollback
self._assert_active(prepared_ok=True, rollback_ok=True)
File "/home/titouan/main_api/python-api/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 303, in _assert_active
raise sa_exc.ResourceClosedError(closed_msg)
sqlalchemy.exc.ResourceClosedError: This transaction is closed
Version:
Python 3.7.3 (VENV)
MySQL 8 (docker)
Flask-SQLAlchemy==2.4.0 / Flask-SQLAlchemy==2.3.2 (tried both)
Flask==1.0.3
SQLAlchemy==1.3.4
Thanks to anyone having a hint here.
Finally solved it.
The issue was in the models before_insert and before_update callbacks.
Trying to retrieve the current-user-id from Flask.g seems to have a strange impact on the session. It's certainly linked to the way my Registry class implementation works.

Get odoo 10 environment outside the http request

there is a need to get a 'bus.bus' model after the my module post_load. But I get an exception:
Traceback (most recent call last):
File "/path/to/file.py", line 19, in __call__
self.event_bus.sendone('channel', 'message')
File "/opt/odoo10/addons/bus/models/bus.py", line 70, in sendone
self.sendmany([[channel, message]])
File "/opt/odoo10/addons/bus/models/bus.py", line 54, in sendmany
self.sudo().create(values)
File "/opt/odoo10/odoo/models.py", line 4870, in sudo
return self.with_env(self.env(user=user))
File "/opt/odoo10/odoo/api.py", line 781, in __call__
return Environment(cr, uid, context)
File "/opt/odoo10/odoo/api.py", line 726, in __new__
env, envs = None, cls.envs
File "/opt/odoo10/odoo/tools/func.py", line 111, in __get__
return self.fget.__get__(None, owner)()
File "/opt/odoo10/odoo/api.py", line 699, in envs
return cls._local.environments
File "/opt/odoo10/venv/local/lib/python2.7/site-packages/werkzeug/local.py", line 72, in __getattr__
raise AttributeError(name)
AttributeError: environments
I am try get environment like this:
registry = RegistryManager.get(db_name)
cursor = registry.cursor()
return Environment(cursor, None, context={})
So how I can get correct environment outside the http request?
Try to get Environment using following.
from odoo import api, fields, models, _
env = api.Environment(cr, SUPERUSER_ID, {})
return env

<StdImageFieldFile: None> is not JSON serializable

I'm using django-stdimage for uploading and resizing images, and it works well.
I'm having a problem, though, with django-allauth; when I try to login with the social account, and there already is a normal account with the same e-mail address, I'm getting the following error:
TypeError at /accounts/facebook/login/callback/
is not JSON serializable
This is the full traceback:
Environment:
Request Method: GET
Request URL: http://localhost:8000/accounts/facebook/login/callback/?code=AQCf7MjgfOAsqf0sS0gup0hqLKyZClQvkGKyWtkORNBru_ITaRNHKgxwaH5RaCSARIb9U1ZgnqhWm3OQAfKW1r5nbVRkKr4fcLWtXdGL85-LYIyuF-NftkJpIhdIMR-VTMF8XXbKescZhxz0hDP_eKl1tKL6uPqWKc8NliWWHh9kOYSS69rAzNRUjZhgx6Zul9sAkV9nRoDo-JunhDRtvOV3crnpr9zAU6jsPDChcJ5dgcRPQ39EoOhrDE16-ia6WF1lFMz_fw1Pgjvo-2jduNG-c9TPyY23A205wm3d1PItoXH2U4GU8j1u5iAg1OIJuvDh-2viQA1disQoM_Du3vUldbX4Plun-yNay2kzNepOyw&state=0J6Ydn3lDKi0
Django Version: 1.8
Python Version: 2.7.6
Installed Applications:
('django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.sites',
'blog',
'custom_user',
'django_markdown',
'storages',
'parsley',
'stdimage',
'stdimage_serializer',
'rest_framework',
'allauth',
'allauth.account',
'allauth.socialaccount',
'allauth.socialaccount.providers.facebook')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware')
Traceback:
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
132. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/allauth/socialaccount/providers/oauth2/views.py" in view
62. return self.dispatch(request, *args, **kwargs)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/allauth/socialaccount/providers/oauth2/views.py" in dispatch
135. return complete_social_login(request, login)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/allauth/socialaccount/helpers.py" in complete_social_login
145. return _complete_social_login(request, sociallogin)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/allauth/socialaccount/helpers.py" in _complete_social_login
161. ret = _process_signup(request, sociallogin)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/allauth/socialaccount/helpers.py" in _process_signup
26. request.session['socialaccount_sociallogin'] = sociallogin.serialize()
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/allauth/socialaccount/models.py" in serialize
198. user=serialize_instance(self.user),
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/allauth/utils.py" in serialize_instance
194. return json.loads(json.dumps(data, cls=DjangoJSONEncoder))
File "/usr/lib/python2.7/json/__init__.py" in dumps
250. sort_keys=sort_keys, **kw).encode(obj)
File "/usr/lib/python2.7/json/encoder.py" in encode
207. chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py" in iterencode
270. return _iterencode(o, 0)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/django/core/serializers/json.py" in default
112. return super(DjangoJSONEncoder, self).default(o)
File "/usr/lib/python2.7/json/encoder.py" in default
184. raise TypeError(repr(o) + " is not JSON serializable")
Exception Type: TypeError at /accounts/facebook/login/callback/
Exception Value: <StdImageFieldFile: None> is not JSON serializable
I can't understand if it's a django-allauth problem or something else.
This is models.py:
class CustomUser(AbstractBaseUser, PermissionsMixin):
first_name = models.CharField(max_length=254, blank=True)
second_name = models.CharField(max_length=254, blank=True)
email = models.EmailField(blank=True, unique=True)
date_joined = models.DateTimeField(_('date joined'), default=datetime.now())
#avatar = models.ImageField('profile picture', upload_to=upload_avatar_to, null=True, blank=True)
avatar = StdImageField(upload_to=upload_avatar_to, null=True, blank=True,
variations={
'thumbnail': {'width': 250, 'height': 250, "crop": True}
})
is_active = models.BooleanField(default=False)
is_admin = models.BooleanField(default=False)
is_staff = models.BooleanField(default=False)
USERNAME_FIELD = 'email'
REQUIRED_FIELDS = ['first_name', 'second_name']
objects = CustomUserManager()
class Meta:
verbose_name = _('user')
verbose_name_plural = _('users')
def save(self, *args, **kwargs):
super(CustomUser, self).save(*args, **kwargs)
def get_absolute_url(self):
return "/users/%s" % urlquote(self.email)
def get_full_name(self):
"""
Returns the first_name plus the last_name, with a space in between.
"""
return self.email
def __str__(self):
return self.email
def get_short_name(self):
"""
Returns the first name for the user.
"""
return self.first_name
def email_user(self, subject, message, from_email=None):
"""
Sends an email to this user.
"""
send_email(subject, message, from_email, [self.email])
I've also tried to use django-stdimage-serializer, but when I try to set up the models file with it, and fire up makemigrations, that's what I get:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute()
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 312, in execute
django.setup()
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models(all_models)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/django/apps/config.py", line 198, in import_models
self.models_module = import_module(models_module_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/home/stefano/projects/blog-project/blog/models.py", line 6, in <module>
from custom_user.models import CustomUserManager, CustomUser
File "/home/stefano/projects/blog-project/custom_user/models.py", line 52, in <module>
class CustomUser(AbstractBaseUser, PermissionsMixin):
File "/home/stefano/projects/blog-project/custom_user/models.py", line 60, in CustomUser
'thumbnail': {'width': 250, 'height': 250, "crop": True}
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/rest_framework/fields.py", line 1405, in __init__
super(ImageField, self).__init__(*args, **kwargs)
File "/home/stefano/projects/blog-project/blogprojectenv/local/lib/python2.7/site-packages/rest_framework/fields.py", line 1359, in __init__
super(FileField, self).__init__(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'upload_to'
What could I do?
If anyone is interested, this answer is exactly what I was looking for.

SchedulingError: Couldn't apply scheduled task downloader: argument must be an int, or have a fileno() method

i have added djcelery and kombu.transport.django to django apps.
settings.py:
import djcelery
djcelery.setup_loader()
from .celery import app as celery_app
BROKER_URL = "redis://abc:abc#localhost:6379/0"
CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"
CELERY_TIMEZONE = "Europe/London"
CELERY_ENABLE_UTC = True
# store AsyncResult in redis
CELERY_RESULT_BACKEND = "redis"
REDIS_HOST = "localhost"
REDIS_PORT = 6379
REDIS_DB = 0
REDIS_VHOST = 0
REDIS_USER="xyz"
REDIS_PASSWORD="xyz"
REDIS_CONNECT_RETRY = True
celery.py
from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings
from . import celeryconfig
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')
app = Celery('apps')
# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
#app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))
celery supervisor configuration:
[program:celeryd]
command=/home/vagrant/.virtualenvs/project/bin/python /vagrant/project/manage.py celeryd -B -E -c 1 --settings=config.settings
directory=/vagrant/project
user=vagrant
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/celeryd.log
stderr_logfile=/var/log/supervisor/celeryd_err.log
when a task is scheduled using django-celery interface which supposed to download some files, following errors are thrown:
Traceback (most recent call last):
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/beat.py", line 203, in maybe_due
result = self.apply_async(entry, publisher=publisher)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/beat.py", line 259, in apply_async
entry, exc=exc)), sys.exc_info()[2])
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/beat.py", line 251, in apply_async
**entry.options)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/app/task.py", line 555, in apply_async
**dict(self._get_exec_options(), **options)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/app/base.py", line 323, in send_task
reply_to=reply_to or self.oid, **options
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/app/amqp.py", line 302, in publish_task
**kwargs
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/messaging.py", line 168, in publish
routing_key, mandatory, immediate, exchange, declare)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/connection.py", line 440, in _ensured
return fun(*args, **kwargs)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/messaging.py", line 180, in _publish
[maybe_declare(entity) for entity in declare]
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/messaging.py", line 111, in maybe_declare
return maybe_declare(entity, self.channel, retry, **retry_policy)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/common.py", line 99, in maybe_declare
return _maybe_declare(entity)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/common.py", line 110, in _maybe_declare
entity.declare()
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/entity.py", line 505, in declare
self.queue_declare(nowait, passive=False)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/entity.py", line 531, in queue_declare
nowait=nowait)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 447, in queue_declare
return queue_declare_ok_t(queue, self._size(queue), 0)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 651, in _size
sizes = cmds.execute()
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/redis/client.py", line 2157, in execute
conn.disconnect()
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 800, in disconnect
channel._on_connection_disconnect(self)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 461, in _on_connection_disconnect
self.connection.cycle._on_connection_disconnect(connection)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 259, in _on_connection_disconnect
self.poller.unregister(connection._sock)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/utils/eventio.py", line 85, in unregister
self._epoll.unregister(fd)
SchedulingError: Couldn't apply scheduled task downloader: argument must be an int, or have a fileno() method.
and i'm not getting any clue where from these errors are coming...
any suggestions will be greatly appreciated.
Update 1:
i use django-celery admin interface to schedule a task and task is defined inside an app registered in INSTALLED_APP as follows:
#shared_task
def downloader():
# execute static/shell_scripts/feed_downloader.sh
import subprocess
PROJECT_PATH = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
subprocess.call(PROJECT_PATH+"/static/shell_scripts/feed_downloader.sh", shell=True)
the feed_downloader.sh file calls a django management command which if i execute independetly works fine like:
python manage.py feed_downloader
if i add a print statement just before line no. 85 to check fd parameter in file
/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/utils/eventio.py
then following messages get printed:
see this <socket._socketobject object at 0x131b750>
see this None
see this <socket._socketobject object at 0x131bc90>
see this None