how we can optimize sql request to db to avoid extra information and decrease the time response in nested models? - sql

We work with SQLalchemy and postgre sql to write api.We have 2 nested models: user and company.
class UserBaseModel(BaseModel):
first_name: str
last_name: str
email: str
company:CompanyModel
class CompanyCreateModel(BaseModel):
name: str
description: Optional[str] = None
def get_all_companies(db: Session, name_like: Optional[str]) -> List[Company]:
query = db.query(Company)
if name_like:
query = query.filter(Company.name.ilike(f"%{name_like}%"))
return query.all()
and with this query.all() in response it gives back whole the class of company inside the user but i only need company_id.
I also want to know if it the pydantic model which triggers the sql requests or it's just a filter on results after sending the request to the database.

Related

Queries with inner joins in database layer or filtering on ids generated in application layer?

I have a table that describes a document with title, content, created_at, etc and a struct that describes columns to filter that table
struct Partial {
handle: String,
title: String,
content: String,
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
}
which; after some transformations, I pass to the database layer and depending on who the user is, return a set of rows.
For example, if user is the author, user can search any columns, otherwise they can search only publicly visible documents.
A user can also 'share' a document with another user so if there a share from the requester to the document owner, the user should also be able to search in those documents as well.
There are two ways to go about that (as far as I know)
Use inner joins and filter on columns
INNER JOIN author on document.id = author.document_id
INNER JOIN visibility on document.id = visibility.document_id
INNER JOIN share on document.id = share.document_id
...
...
...
WHERE document.handle = $handle -- from `Partial` struct
WHERE document.title = $title -- from `Partial` struct
WHERE document.content = $content -- from `Partial` struct
WHERE author.user_id = $user_id -- from user info passed to db
WHERE visibility.scope = 'PUBLIC' -- from user info passed to db
WHERE share.shared_to = $user_id -- from user info passed to db
WHERE group.member = $user_id -- from user info passed to db
...
...
...
This requires passing user information ($user_id for this example but this also sometimes includes additional info like if the user is blocked or the account is non-public, belongs to a group, etc. Around 12 properties in total) to the database layer.
generate a list of ids in application layer and pass it on to the database layer.
For example,
let public_document_ids = visibility.find(Find { visibility: Visibility::Public }).map(|visibility_row| visibility_row.document_id).collect();
let authored_document_ids = author.find(Find { document_author: user_id }).map(|authored_row| authored_row.document_id).collect();
let shared_document_ids = share.find(Find { shared_to: user_id }).map(|share_row| share_row.document_id).collect();
and then pass sum of all of these ids to database
WHERE document.handle = $handle -- from `Partial` struct
WHERE document.title = $title -- from `Partial` struct
WHERE document.content = $content -- from `Partial` struct
WHERE document.id IN $ids -- from `ids` passed to db
...
...
...
To me it seems that the first approach of using inner joins and filtering on columns in the database can
potentially be more efficient,
and a lot more easier to get right
but the second approach, generating a list of ids in the application layer, allows
a lot more flexibility and
also lets me use different (distributed) databases to store different tables (perhaps even different database drivers)
It also reduces the amount of user information passed to the database layer
but it also requires loading all the data into the application layer before filtering the results
and implementing pagination will be lot more complex
So my question - is there a way to know which approach I should go with? Perhaps there's something I haven't considered yet.

Typeorm query for entity using another entity in where clause in version 0.3.X

In my system we have users and addresses.
I want to query the addresses table for all the addresses belonging to a user.
Within the version 0.2.X I was able to write the query like
user: UserEntity = blah
AddressEntity.findOne({ user: user }))
Whereas now I need to do
user: UserEntity = blah
AddressEntity.findOneBy({ user: {id : user.id }))
Is there any workaround to support the shorthand notation again?

Flask-Admin User Read/Write Settings Per Row

Say I have User and Subuser tables, where a User can have many Subusers. How can I only allow the logged in user (current_user) to view only the Subusers who are a part of the User? I.e. Subuser.user_id == current_user.id. I get I can do this by a filter, but this must be forced and not optional.
I'm using the SQLAlchemy backend if that helps!
It's pretty simple to override the queries that are produced by the backend in Flask Admin - you can do some quite complex things. Remember to override all of the below - especially get_one().
class BaseModelView(ModelView):
def is_accessible(self):
# if not authenticated we can't see this
if not login.current_user.is_authenticated():
return False
return True
def get_query(self):
return query.filter(self.model.parent_user_id.any(id=login.current_user.id))
def get_count_query(self):
# faster count query!
query = self.session.query(func.count('*')).select_from(self.model).filter(self.model.location_id == login.current_user.id)
return query
def get_one(self, id):
"""
Return a single model by its id.
:param id:
Model id
"""
result = self.session.query(self.model).get(iterdecode(id))
# if the users location does not exist and she is only allowed to edit locations they control
if login.current_user.id != result.parent_user_id:
app.logger.warn('You are not allowed to edit entries for this user.')
abort(404)
return result

Django Query or SQL

I have these models:
class Person(models.Model):
user = models.OneToOneField(User)
class Post(models.Model):
author = models.ForeignKey(Person, null=False)
text = models.TextField(null=False, blank=False)
and want a Queryset at least with the below fields,
author.user.username
text
I have read select_related() queries but when I try to use that with this view can't get username field
posts = Post.objects.select_related('person__user')[:10]
can I use Django query or have to use SQL raw ?
Thanks for any help
You can serialize like this:
import json
from django.core.serializers.json import DjangoJSONEncoder
json_data = json.dumps(list(Post.objects.values('author__user__username', 'text')[:10]), cls=DjangoJSONEncoder)
select_related should be called with the field names, not the type.
posts = Post.objects.select_related('author__user')[:10]
for post in posts:
print(post.person.user.username)
print(post.text)
All the select_related does is ensure that the foreign fields can be accessed without extra queries (select_related constructs joins to the relevant tables).

TastyPie: Consulting and updating fields from other models within a resource

I have a Profile model with a OneToOne relationship with a User. For such Profile model I use the following class, which uses two additional fields coming directly from the User:
class ProfileResource(ModelResource):
username = fields.CharField(attribute='user__username')
email = fields.CharField(attribute='user__email')
class Meta:
# Base data
queryset = Profile.objects.all()
# Allowed methods
list_allowed_methods = ['get']
detail_allowed_methods = ['get', 'put', 'patch']
# Security setup (FEEBLE)
authorization = Authorization()
authentication = BasicAuthentication()
Such resource works fine when consulting profiles. It retrieves the username and email perfectly, and is capable of filtering and ordering by such parameters.
However, I cannot manage to update such fields on the User model in a, say, elegant fashion. All I have come up with is this:
def obj_update(self, bundle, skip_errors=False, **kwargs):
bundle = super().obj_update(bundle, skip_errors=skip_errors, **kwargs)
if 'username' in bundle.data: bundle.obj.user.username = bundle.data['username']
if 'email' in bundle.data: bundle.obj.user.email = bundle.data['email']
bundle.obj.user.save()
return bundle
Which works fine but doesn't seem the best of solutions.
Does anybody know a better way to work out this kind of field-only relationship between a resource and a related model?