Django: How to add counts of non-related object? - sql

I have two indirectly related tables - Posts and Follower_to_followee
models.py:
class Post(models.Model):
auth_user = models.ForeignKey(User, null=True, blank=True, verbose_name='Author', help_text="Author")
title = models.CharField(blank=True, max_length=255, help_text="Post Title")
post_content = models.TextField (help_text="Post Content")
class Follower_to_followee(models.Model):
follower = models.ForeignKey(User, related_name='user_followers', null=True, blank=True, help_text="Follower")
followee = models.ForeignKey(User, related_name='user_followees', null=True, blank=True, help_text="Followee")
The folowee is indirectly related to post auth_user (post author) in posts. It is, though, directly related to Django user table and user table is directly related to post table.
How can I select all followees for a specific follower and include post counts for each followee in the result of the query without involving the user table? Actually, at this point I am not even clear how to do that involving the user table. Please help.

It's possible to write query generating single SQL, try something like
qs = User.objects.filter(user_followees__follower=specific_follower).annotate(
post_count=models.Count('post'))
for u in qs:
print u, u.post_count
Check the second part of https://stackoverflow.com/a/13293460/165603 (things work similarly except the extra M2M manager)
When being used inside User.objects.filter, both user_followees__follower=foo and user_followers__followee=foo would cause joining of the table of the Follower_to_followee model and a where condition checking for follower=foo or followee=foo
(Note that user_followees__followee=foo or user_followerers__follower=foo works differently from above, Django ORM simplifies them smartly and would generate something like User.objects.filter(pk=foo.pk)).

I'm not entirely sure I understand the question, but here is a simple solution. Note that this could be written more succinctly, but I broke it up so you can see each step.
How can I select all followees for a specific follower?
# First grab all the follower_to_followee entries for a given
# follower called: the_follower
follows = Follower_to_followee.objects.filter(follower=the_follower)
followee_counts = []
# Next, let's iterate through those objects and pick out
# the followees and their posts
for follow in follows:
followee = follow.followee
# post for each followee
followee_posts = Post.objects.filter(auth_user=followee).count()
# Count number of posts in the queryset
count = followee_posts.count()
# Add followee/post_counts to our list of followee_counts
followee_counts.append((followee, count))
# followee_counts is now a list of followee/post_count tuples

For get post counts you can use this:
#get follower
follower = User.objects.get(username='username_of_fallower')
#get all followees for a specific follower
for element in Follower_to_followee.objects.filter(follower=follower):
element.followee.post_set.all().count()

views.py
def view_name(request):
followers = Follower_to_followee.objects.filter(user=request.user)
.......
html
{{user}}<br/>
My followers:<br/>
{% follower in followers %}
<p>{{follower}} - {{follower.user.follower_to_followee_set.count}}</p>
{% endfor %}

Related

Django models ManyToManyField always adding item to list

I'implemented two models, Card and Dish. I used many-to-many relationship because Dish can be in many cards. Not sure if I did that right because any Dish that I add is automatically added to every Card.
Here is the code:
class Dish(models.Model):
name = models.CharField(max_length=255, unique=True)
description = models.TextField(max_length=1000)
price = models.DecimalField(max_digits=5, decimal_places=2)
preparation_time = models.IntegerField()
date_added = models.DateField(auto_now_add=True)
update_date = models.DateField(auto_now=True)
vegan = models.BooleanField(default=False)
def __str__(self):
return self.name
class Card(models.Model):
name = models.CharField(max_length=255, unique=True)
description = models.TextField(max_length=1000)
date_added = models.DateField(auto_now_add=True)
update_date = models.DateField(auto_now=True)
dishes = models.ManyToManyField(Dish, blank=True)
def __str__(self):
return self.name
An the problem in admin panel looks like this:
When I create a dish it is alway added to every card.
Any help please I'm just begining to learn sql and django ORM
Dishes being in that admin field doesn't mean they're actually selected, that's just the Multi-Select field. Only the ones that are highlighted are actually in the field. and you do +Click to toggle if they're selected or not
I guess that was the best way to show a Many-to-Many field, tho it might be confusing to use imo.. That's why I always just edit them in the shell, especially when there gets 100+, 200+ items.

Django Making 1000 Duplicate Queries

Model:
class Comment(MPTTModel):
submitter = models.ForeignKey(User, blank=True, null=True)
post = models.ForeignKey(Post, related_name="post_comments")
parent = TreeForeignKey('self', blank=True, null=True, related_name="children")
text = models.CharField("Text", max_length=1000)
rank = models.FloatField(default=0.0)
pub_date = models.DateTimeField(auto_now_add=True)
Iterating through nodes has the same effect (>1000 queries).
I had similar issue with MPTT models. It was solved with select_related
(also for parent's foreign keys).
So, depending on your needs, proper queryset can looks like:
Comment.objects.select_related('post', 'submitter', 'parent', 'parent__submitter', 'parent__post')
Also, if you need comment's children in your loop as well, it can be optimized like that:
queryset.prefetch_related('children')
Or even like that:
queryset.prefetch_related(
Prefetch(
'children',
queryset=Comment.objects.select_related('post', 'etc.'),
to_attr='children_with_posts'
)
)
... and depending on tree depth, you can use that:
queryset.select_related('parent', 'parent__parent', 'parent__parent__parent')
# you got the idea:)
Duplicated queries happens because all objects from iteration hits the data base when you refer a related object.
Try using select_related in your view method.
Probably using django prefetch related or select related will resolve that, but if not work, sorry you will need a raw query.
Have you ever read about optimizing Django queries? Here is a simple tutorial that's explain a lot of things: https://docs.djangoproject.com/en/3.1/topics/db/optimization/

Django Query or SQL

I have these models:
class Person(models.Model):
user = models.OneToOneField(User)
class Post(models.Model):
author = models.ForeignKey(Person, null=False)
text = models.TextField(null=False, blank=False)
and want a Queryset at least with the below fields,
author.user.username
text
I have read select_related() queries but when I try to use that with this view can't get username field
posts = Post.objects.select_related('person__user')[:10]
can I use Django query or have to use SQL raw ?
Thanks for any help
You can serialize like this:
import json
from django.core.serializers.json import DjangoJSONEncoder
json_data = json.dumps(list(Post.objects.values('author__user__username', 'text')[:10]), cls=DjangoJSONEncoder)
select_related should be called with the field names, not the type.
posts = Post.objects.select_related('author__user')[:10]
for post in posts:
print(post.person.user.username)
print(post.text)
All the select_related does is ensure that the foreign fields can be accessed without extra queries (select_related constructs joins to the relevant tables).

Rails combine two objects into one result hash

I have the following models in Rails: Users and *Friendship_Requests* friendship_requests contains user ids in the fields "from_user" and "to_user". I want to get a list of requests linked to a specific user from the requests model (which contains a message field also). I would also like the result to return the requesting users data that is held in the Users model.
In SQL I would use the following:
SELECT users.*, friendships_requests.*
FROM friendships_requests
JOIN users ON friendships_requests.from_user = users.id
Any ideas appreciated, thanks.
To find the friendship requests sent to #user:
#friendship_requests = FriendshipRequests.find_by_to_user(#user.id)
and then the informations relative to the sending users can be retrieved as:
#friendship_requests.each do |request|
user = User.find_by_id request.from_user
# Do something
end
or, if you want to collect all of them
#users = #friendship_requests.map {|r| User.find_by_id r.from_user}

Django aggregate query

I have a model Page, which can have Posts on it. What I want to do is get every Page, plus the most recent Post on that page. If the Page has no Posts, I still want the page. (Sound familiar? This is a LEFT JOIN in SQL).
Here is what I currently have:
Page.objects.annotate(most_recent_post=Max('post__post_time'))
This only gets Pages, but it doesn't get Posts. How can I get the Posts as well?
Models:
class Page(models.Model):
name = models.CharField(max_length=50)
created = models.DateTimeField(auto_now_add = True)
enabled = models.BooleanField(default = True)
class Post(models.Model):
user = models.ForeignKey(User)
page = models.ForeignKey(Page)
post_time = models.DateTimeField(auto_now_add = True)
Depending on the relationship between the two, you should be able to follow the relationships quite easily, and increase performance by using select_related
Taking this:
class Page(models.Model):
...
class Post(models.Model):
page = ForeignKey(Page, ...)
You can follow the forward relationship (i.e. get all the posts and their associated pages) efficiently using select_related:
Post.objects.select_related('page').all()
This will result in only one (larger) query where all the page objects are prefetched.
In the reverse situation (like you have) where you want to get all pages and their associated posts, select_related won't work. See this,this and this question for more information about what you can do.
Probably your best bet is to use the techniques described in the django docs here: Following Links Backward.
After you do:
pages = Page.objects.annotate(most_recent_post=Max('post__post_time'))
posts = [page.post_set.filter(post_time=page.most_recent_post) for page in pages]
And then posts[0] should have the most recent post for pages[0] etc. I don't know if this is the most efficient solution, but this was the solution mentioned in another post about the lack of left joins in django.
You can create a database view that will contain all Page columns alongside with with necessary latest Post columns:
CREATE VIEW `testapp_pagewithrecentpost` AS
SELECT testapp_page.*, testapp_post.* -- I suggest as few post columns as possible here
FROM `testapp_page` LEFT JOIN `testapp_page`
ON test_page.id = test_post.page_id
AND test_post.post_time =
( SELECT MAX(test_post.post_time)
FROM test_post WHERE test_page.id = test_post.page_id );
Then you need to create a model with flag managed = False (so that manage.py sync won't break). You can also use inheritance from abstract Model to avoid column duplication:
class PageWithRecentPost(models.Model): # Or extend abstract BasePost ?
# Page columns goes here
# Post columns goes here
# We use LEFT JOIN, so all columns from the
# 'post' model will need blank=True, null=True
class Meta:
managed = False # Django will not handle creation/reset automatically
By doing that you can do what you initially wanted, so fetch from both tables in just one query:
pages_with_recent_post = PageWithRecentPost.objects.filter(...)
for page in pages_with_recent_post:
print page.name # Page column
print page.post_time # Post column
However this approach is not drawback free:
It's very DB engine-specific
You'll need to add VIEW creation SQL to your project
If your models are complex it's very likely that you'll need to resolve table column name clashes.
Model based on a database view will very likely be read-only (INSERT/UPDATE will fail).
It adds complexity to your project. Allowing for multiple queries is a definitely simpler solution.
Changes in Page/Post will require re-creating the view.