I'm trying to figure out how to write a hook to query the database before inserting a row from the ORM. I hope to achieve something similar to this:
class Table(Base):
id = Column(Integer, primary_key=True)
value = Column(Integer, nullable=False)
def before_insert_hook(self, session):
"""Some arbitrary queries and code. For example:"""
if self.value is None:
self.value = session.query(func.avg(Table.value))\
.filter(Table.value > 100).scalar()
I've been reading up in the SQLAlchemy docs about ORM events and such, but I can't figure out how to use them to achieve this.
Looks like you want ORM Events:
from sqlalchemy import event
class Table(Base):
...
#event.listens_for(Table, 'before_insert')
def do_stuff(mapper, connect, target):
# target is an instance of Table
target.value = ...
Related
After reading the following link https://github.com/googleapis/python-bigquery-sqlalchemy I managed to query a table stored on Google BigQuery by using SLQAlchemy. Now I would like to create a SQLAlchemy Users model on Google BigQuery so that I can use Flask-login features (e.g. UserMixin) in order to validate whether a user is authenticated, active, etc. but in this case my database is stored on BigQuery instead of a traditional SQL database (mainly due to billing costs as I find Google Cloud SQL way more expensive than Google BigQuery).
This is my code:
from sqlalchemy.engine import create_engine
from flask_sqlalchemy import SQLAlchemy
engine = create_engine('bigquery://my_project',credentials_path='my_credentials.json')
db = SQLAlchemy()
class Users(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(15), unique=True, nullable = False)
email = db.Column(db.String(50), unique=True)
Users.metadata.create_all(engine)
Unfortunately I'm getting the following error:
DatabaseError: (google.cloud.bigquery.dbapi.exceptions.DatabaseError) 400 Table "users" must be qualified with a dataset (e.g. dataset.table).
I tried to modify the engine variable as follows:
engine = create_engine('bigquery://my_project.my_dataset',credentials_path='my_credentials.json')
but then I get the following error (I'm intrigued by the None part):
ValueError: table_id must be a fully-qualified ID in standard SQL format, e.g., "project.dataset.table_id", got my_project.my_dataset.None.users
Does anyone know how can I create a SQLAlchemy model on Google BigQuery?
Can you try this:
from sqlalchemy.engine import create_engine
from flask_sqlalchemy import SQLAlchemy
from pybigquery.api import ApiClient
from flask import Flask
db = SQLAlchemy()
#ToDo:Change project name and dataset name.
engine = create_engine('bigquery://my-project/my-dataset')
db = SQLAlchemy()
class Users(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(15), unique=True, nullable = False)
email = db.Column(db.String(50), unique=True)
Users.metadata.create_all(engine)
I solved the error by adding the dataset name.
engine = create_engine('bigquery://my-project/my-dataset')
So I'm just trying to make sense of the output of the SQLAlchemy ORM methods after creating a model, committing some entries and running queries. Most queries are fine...I'm getting back a list but for some it just returns an object (see below). I know this sounds obvious but is this normal behavior? I'm specifically referring to the filter_by query as you can see below...
#sample_app.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app=Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI']='...'
db = SQLAlchemy(app)
class Person(db.Model):
__tablename__='persons'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(), nullable=False)
def __repr__(self):
return f'<Person Id: {self.id}, name: {self.name}>'
db.create_all()
#Run some basic commands in interactive mode with model already populated
python3
from sample_app import db,Person
#add a bunch of persons
person1=Person(name='Amy')
person2=...
db.session.add(person1)
db.session.commit()
...
#Run queries
Person.query.all() #returns all persons as a list
Person.query.first() #returns first item in the list
Person.query.filter_by(name='Amy')
#returns <flask_sqlalchemy.Basequery object at 0xsadfjasdfsd>
So why am I not getting the same type of output for the third query for 'Amy'? is that normal behavior for the filter_by method?
Thanks
You didn’t execute the query in the last example. The all method brings back all object selected by the query, first is the first. You’ve specified a filter in the last example, but you didn’t execute a method which processes the query and returns a result [set].
If there are more than one Amy’s, you get all the matches with all() or the first with first(). If you had a filter which should yield a unique record, you could also use .one()
I'm trying to query for an object in my DB, and get data from another object that shares a Foreign Key relationship.
For example, if I have these models:
class Book(models.Model):
language = models.ForeignKey('Language')
...
class Language(models.Model):
name = models.CharField(max_length=255, unique=True)
I want to query these models and get a QuerySet of books, then return the books via an API.
In raw SQL I would do something akin to:
SELECT book, language.name
FROM book
JOIN ....
Is there any way to accomplish this with the Django ORM?
If you are using django rest framework for the API, You can do this by using serializer Method field
class BookSerializer(serializers.ModelSerializer):
language = serializers.SerializerMethodField()
class Meta:
model = Book
fields = ['id', 'language']
def get_language(self,obj)
return obj.language.name
Another way to do this is adding a property field to Book Model
#property
def language(self):
return self.language.name
Now if you do book_obj.language ,You can get the name in language model
I am attempting to model a friendship using SQLAlchemy ORM. The relationship that I am trying to model is symmetric. Similar to Facebook, if user a is to add user b, user b must approve that friendship request. My current model is as follows.
class User(db.Model):
__tablename__ = 'User'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(35), unique=False)
username = db.Column(db.String(25), index=True, unique=True)
password = db.Column(db.String(35), unique=False)
email = db.Column(db.String(35), unique=True)
phone_number = db.Column(db.String(22))
# define relationships
requester = db.relationship('Relationship', foreign_keys='Relationship.requesting_user', backref='requester')
receiver = db.relationship('Relationship', foreign_keys='Relationship.receiving_user', backref='received')
def __repr__(self):
return '<User %r>' % (self.username)
class Relationship(db.Model):
__tablename__ = 'Relationship'
id = db.Column(db.Integer, primary_key=True)
requesting_user = db.Column(db.Integer, db.ForeignKey('User.id'))
receiving_user = db.Column(db.Integer, db.ForeignKey("User.id"))
status = db.Column(db.Integer)
__table_args__ = (db.UniqueConstraint('receiving_user', 'requesting_user', name='_receiving_user_uc'), )
The model works, however, I don't think that it is properly modeled. Is it even required that I use a status? I'm assuming it can be modeled so that each friend relationship gets its own entry. Currently, a user can initiate a friend request with another user. When the other user approves the request, the status changes to accepted. I have looked a little into association tables but am not too sure how they would play into a model like this. Any advice on my current model and how it can be improved would be greatly appreciated.
Among other things, you may want to learn about association proxies. An association proxy tells SQLAlchemy that you have a many-to-many relationship mediated by an intermediate table which may contain additional data. In your case, each User can send multiple requests and also receive multiple requests and Relationship is the mediating table which contains the status column as additional data.
Here is a variant of your code which stays relatively close to what you wrote:
from sqlalchemy.ext.associationproxy import association_proxy
class User(db.Model):
__tablename__ = 'User'
# The above is not necessary. If omitted, __tablename__ will be
# automatically inferred to be 'user', which is fine.
# (It is necessary if you have a __table_args__, though.)
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(35), unique=False)
# and so forth
requested_rels = db.relationship(
'Relationship',
foreign_keys='Relationship.requesting_user_id',
backref='requesting_user'
)
received_rels = db.relationship(
'Relationship',
foreign_keys='Relationship.receiving_user_id',
backref='receiving_user'
)
aspiring_friends = association_proxy('received_rels', 'requesting_user')
desired_friends = association_proxy('requested_rels', 'receiving_user')
def __repr__(self):
# and so forth
class Relationship(db.Model):
# __tablename__ removed, becomes 'relationship'
# __table_args__ removed, see below
requesting_user_id = db.Column(db.Integer, db.ForeignKey('User.id'), primary_key=True)
receiving_user_id = db.Column(db.Integer, db.ForeignKey('User.id'), primary_key=True)
# Marking both columns above as primary_key creates a compound primary
# key, which at the same time saves you the effort of defining the
# UNIQUE constraint in __table_args__
status = db.Column(db.Integer)
# Implicit one-to-many relations: requesting_user, receiving_user.
# Normally it would be more convenient to define those relations on
# this side, but since you have two outgoing relationships with the
# same table (User), you chose wisely to define them there.
(Note how I ordered the lines slightly differently and how I used the _id suffix for foreign key columns while reserving the same name without the suffix for the corresponding db.relationships. I would suggest that you adopt this style, too.)
Now you have a clean way to access incoming and outgoing friendship requests as well as the corresponding users directly from your User model. However, this is still less than ideal because you need to write the following code in order to get all confirmed friends of a user:
def get_friends(user):
requested_friends = (
db.session.query(Relationship.receiving_user)
.filter(Relationship.requesting_user == user)
.filter(Relationship.status == CONFIRMED)
)
received_friends = (
db.session.query(Relationship.requesting_user)
.filter(Relationship.receiving_user == user)
.filter(Relationship.status == CONFIRMED)
)
return requested_friends.union(received_friends).all()
(I did not test this; you might need to also join with User in both queries in order for the union to work.)
To make things worse, the model name Relationship as well as the names of several members within the models don't seem to convey very well what they actually mean.
You can improve matters by removing Relationship.status and renaming Relationship to FriendshipRequest. Then, add a second User-to-User association model called Friendship and add a corresponding second set of db.Relationships with backrefs and association_proxys to User. When somebody sends a friendship request, you file a record to FriendshipRequest. If the request is accepted, you remove the record and replace it with a new record in Friendship. This way, instead of using a status code, the status of a friendship is encoded by the table in which you store a pair of users. The Friendship model may look like this:
class Friendship(db.Model):
user1_id = db.Column(db.Integer, db.ForeignKey('User.id'), primary_key=True)
user2_id = db.Column(db.Integer, db.ForeignKey('User.id'), primary_key=True)
# Implicit one-to-many relations: user1, user2
# (defined as backrefs in User.)
(Corresponding db.relationships and association_proxys in User are left as an exercise to the reader.)
This approach saves you half of the filtering operations when you need the confirmed friends of a user. Still, you need to make a union of two queries because your user can be either user1 or user2 in each instance of Friendship. This is inherently difficult because we are dealing with a reflexive symmetric relationship. I think it is possible to invent still more elegant ways to do it, but I think that would be complicated enough to warrant a new question here on Stack Overflow.
I have the following 2 tables
In models.py
class Foo(models.Model):
uuid = models.CharField(_('UUID'), primary_key=True, default=uuid4)
and
class FooExt(models.Model):
uuid = models.ForeignKey(Foo, verbose_name=_('UUID'), primary_key=True)
time = models.DateTimeField(_('Create DateTime'), auto_now_add=True)
Basically, I have Foo and FooExt. I want a one-to-one relation between FooExt. That's why I set FooExt's primary key to be foreign key into Foo (not sure if this is the right thing to do).
Now I add an entry into Foo. Does an entry for FooExt automatically get created? Or do I need to manually add an entry to both Foo and FooExt?
Is there anything I can do to get the "automatic" add feature? Conceptually, these 2 tables describe the same thing, but I just don't want to pollute Foo with extra information. So it'd be great if an add to Foo automatically creates a corresponding FooExt.
If you want an OneToOne relation, then use models.OneToOneField instead of models.ForeignKey. with foreign keys you will need add unique=True in you ForeignKey:
class Foo(models.Model):
uuid = models.CharField(_('UUID'), primary_key=True, default=uuid4)
class FooExt(models.Model):
uuid = models.OneToOneField(Foo, verbose_name=_('UUID'), primary_key=True)
time = models.DateTimeField(_('Create DateTime'), auto_now_add=True)
No, an entry for FooExt don't get created when you create a Foo instance, you need to manually add an entry to both Foo and FooExt. think in Places and Restaurants, many places can be restaurants, but no all the places are restaurants.
if you like an automatic add feature inside Foo that create a FooExt instance, then you can overload the save method inside Foo that create and save FooExt instance too, something like this:
class Foo(models.Model):
....
....
def save(self, *args, **kwargs):
super(Foo, self).save(*args, **kwargs)
foo_ext = FooExt()
foo_ext.uuid = self
foo_ext.save()
Looks like there was mistake in Yonsy Solis answer in save method(corrected), try this:
class Foo(models.Model):
....
....
def save(self, *args, **kwargs):
super(Foo, self).save(*args, **kwargs)
foo_ext = FooExt()
foo_ext.uuid = self
foo_ext.save()
remark: i cant comment yet, so i decide to create answer
Have a look at the AutoOneToOneField in django-annoying
https://github.com/skorokithakis/django-annoying
or answer to this question: Can Django automatically create a related one-to-one model?