Using aggregation on subquery in Django ORM - sql

I have a table like this
Category
Subcategory
Sub_subcategory
Cat_1
Subcat_1
Sub_subcat_1
Cat_1
Subcat_1
Sub_subcat_2
Cat_1
Subcat_2
Sub_subcat_3
Cat_2
Subcat_2
Sub_subcat_4
Cat_3
Subcat_3
Sub_subcat_5
And I need to find in how many categories each subcategory appears.
So my expected output based on above table would be:
Subcategory
Total
Subcat_2
2
Subcat_1
1
Subcat_3
1
So I can get that by running following SQL query:
SELECT subcategory, count(*) total FROM (
SELECT DISTINCT subcategory, category FROM table_1
) as temp_table GROUP BY subcategory ORDER BY total DESC
I spent a lot of time trying to get the same result with Django ORM but wasn't able to get it done.
I expected this code to work:
subquery = Table1.objects.values('subcategory', 'category').distinct()
results = subquery.annotate(total=Count('*')).values('subcategory', 'total').order_by('-total')
But it works exactly the same as without 'distinct()' in subquery, so it counts all categories for every subcategory.
I also tried to find similar case in other questions, but those with subqueries usually relate to JOINing tables and using OuterRef, here it is more like getting results based on temporary table that is created by subquery.
Does anyone know how can I achieve that (or if it's even possible)?

this should do the job
dictionary = {}
queryset = YOUR_queryset_with_table_HERE.objects.all()
for i in queryset:
subcategory = i.subcategory
#check if subcategory already in dictionary
#if not create it with 1, else +=1
if dictionary["subcategory"]:
dictionary["subcategory"] += 1
else:
dictionary["subcategory"] = 1
print(dictionary)

Say you have two models in a many-to-many relationship similar to this:
class Category(models.Model):
category_name = models.CharField(max_length=30)
class SubCategory(models.Model):
subcategory_name = models.CharField(max_length=30)
category = models.ManyToManyField(Category)
Then this should give you the total number of categories in which each subcategory appears:
results = SubCategory.objects.all().annotate(total=Count('category')).values('subcategory_name', 'total')

Related

Django ORM equivalent of SQL Sum

I have a django model as follows:
class Order(models.Model):
cash=models.DecimalField(max_digits=11,decimal_places=2,default=0)
balance=models.DecimalField(max_digits=11,decimal_places=2,default=0)
current_ac=models.DecimalField(max_digits=11,decimal_places=2,default=0)
added_by = models.ForeignKey(User)
There can be multiple Orders and multiple users can create orders.
How can I get the sum of all orders for each columns for a particular user using Django queries, an SQL equivalent would be something like
Select sum(cash), sum (balance), sum(current_ac) from Orders where added_by = 1
You can get your expected result from the code below
from django.db.models import Sum
result = Order.objects.filter(added_by_id=1).aggregate(total_cash=Sum('cash'), total_balance =Sum('balance'), total_current_ac=Sum('current_ac'))
and it will give you a dictionary like this:
{'total_cash': Decimal('110.00'), 'total_balance': Decimal('110.00'), 'total_current_ac': Decimal('110.00')}

How to include static field without data from a dataset to carry it?

I'm improving a report that currently uses a static table using the lookup function to fill its data from a few different datasets. We're pretty sure this is causing the report to take a lot longer to run, so I'm trying to use a table that uses column groups to achieve the same effect from a single dataset.
Here's what my query currently looks like. This functions exactly as I want it to as long as there's data.
Select CatName, CatCount, Category = 'Category 1', Sorting = 1
FROM
(Select CatName, Count(CatName) as CatCount FROM DataSet WHERE Parameters)
UNION
Select CatName, CatCount, Category = 'Category 2', Sorting = 2
FROM
(Select CatName, Count(CatName) as CatCount FROM DataSet WHERE Parameters)
When there are CatNames and CatCounts to pull from the select statement, the Category works and is pulled by the table as a column group. I need all of the groups to exist at all times.
However, sometimes we don't have data that fits the parameters for a category. The result when that happens is that there isn't a row for the Category field to use and that group doesn't exist in the table. Is there any way I can force the Category field to exist regardless of the data?
If I understand the question correctly, then you may be able to use ISNULL. ISNULL returns either the value you were for which you were looking (check_expression) or the alternative (replacement_value) if check_expression is NULL.
ISNULL ( check_expression , replacement_value )
Select CatName, CatCount, Category = 'Category 2', Sorting = 2
FROM
(Select isnull(CatName,""), Count(CatName) as CatCount FROM DataSet WHERE Parameters)
EDIT
How about a left outer join?
Select b.CatName, b.CatCount, Category = 'Category 2', Sorting = 2
FROM
(select '' as CatName, 0 as Catcount) a left outer join (Select CatName, Count(CatName) as CatCount FROM DataSet WHERE Parameters) b on a.CatName = b.CatName
Found a solution. Took a few tries, not the prettiest, and we'll have to see if it actually improves performance, but it works the way we wanted. Generalized code:
Select C.CatName, C.CatCount, Category = 'Category 1', Sorting = 1
FROM
(Select Top 5 B.CatName, Count(B.CatName) as CatCount
FROM
(Select CatName = case when CatOnlyParam in (Category1Filter) then A.CatName else NULL end
FROM
(Select CatName FROM DataSet WHERE GeneralParameters) as A
) as B
order by CatCount
) as C
UNION
etc
Separating the parameters into different steps guarantees that there will be values for each category, even if those values are NULL. I'm sure there's a cleaner way to get the same effect, but this functions.
Working from the inside out:
Stage 1 (Select statement A): Selects the value from the dataset with very general parameters (between a start and end date, resolved or not, etc).
Stage 2 (Select statement B): Uses the case statement to only pull the data that is relevant for this department while leaving behind NULLs for the data that isn't.
Stage 3 (Select statement C): Takes the data from the list of names and NULLs and gets a count from it. Sorts by that count and takes the top 5. If a category has no data, then the nulls will get "counted" to 0 and passed on to the final step.
Stage 4 (Final select statement): Adds the static fields to the information from the previous step. A category without data will get passed to this as:
CatName: NULL
CatCount: 0
Category: "Category 1"
Sorting: 1
Then this is repeated for the other categories and UNION'd together. Any suggestions to improve this are more than welcome.

How to use SUM in this situation?

I have the following tables below and their schema:
INV
id, product code, name, ucost, tcost, desc, type, qoh
1,123,CPASS 700,1.00,5.00,CPASS 700 Lorem, COM,5
2,456,Shelf 5,2.00,6.00,Shelf 5 KJ, BR,3
GRP
id,type,desc
1,COM,COMPASS
2,BR,SHELF
Currently I have a query like this:
SELECT INV.*,GRP.DESCR AS CATEGORY
FROM INV LEFT JOIN GRP ON INV.TYPE = GRP.TYPE
WHERE INV.QOH = 0
There is no problems with that query.
Right now,I want to know the SUM of the TCOST of every INV record where their QOH is 0.
In this situation, does that I mean all I have to do is to write a separate query like the one below:
SELECT SUM(TCOST)
FROM INV
WHERE QOH = 0
Does it make any sense for me to try and combine those two queries as one ?
First understand that SUM is the aggregate function hence either you can run the Query like
(SELECT SUM(TCOST) FROM INV WHERE QOH=0) as total
This will return Sum of TCOST in INV Table for mentioned condition.
Another approach is finding the Sum based on the some column (e.g. Type)
you could write query like
SELECT Type , SUM(TCOST) FROM INV WHERE QOH=0 GROUP BY type ;
Its not clear on what criteria you want to sum . But I think above two approaches would provide you fare idea .
Mmm, you could maybe use a correlated query, though i'm not sure it's the best approach since I'm not sure I understand what your attempting to do:
SELECT INV.*,
GRP.DESCR AS CATEGORY ,
(SELECT SUM(TCOST) FROM INV WHERE QOH=0) as your_sum
FROM INV LEFT JOIN GRP ON INV.TYPE = GRP.TYPE
WHERE INV.QOH = 0
If you want only one value for the sum(), then your query is fine. If you want a new column with the sum, then use window functions:
SELECT INV.*, GRP.DESCR AS CATEGORY,
SUM(INV.TCOST) OVER () as sum_at_zero
FROM INV LEFT JOIN
GRP
ON INV.TYPE = GRP.TYPE
WHERE INV.QOH = 0;
It does not make sense to combine the queries by adding a row to the first one, because the columns are very different. A SQL result set requires that all rows have the same columns.

SQL Query: How to list joined values on the same row

I've got a table of job listings and a related table which contains the job listing ids and each of the field values in a 'field title' -> 'field value' format.
So to get my listing of jobs, I've JOINED the tables in my SQL Query, but am getting the results on multiple lines because of that. Let me illustrate.
Query is something like:
SELECT list.id, list.activation_date, list_field.value
FROM listings AS list
INNER JOIN listings_fields AS list_field ON list.id = list_field.id
WHERE list.activation_date > SOME VALUE
AND list_field.field_id IN ('Title', 'Category')
ORDER BY list.activation_date DESC, list_field.field_id DESC
The result looks like this:
51325 2012-07-31 Job Title 1
51325 2012-07-31 Category 1, Category 2
51324 2012-07-31 Job Title 2
51324 2012-07-31 Category 3
51323 2012-07-31 Job Title 3
51323 2012-07-31 Category 1, Category 3
I've got all the data I need, it's ordered consistently with title first and category second, but I can't think of how to get the result all in one row. This must be a common problem with a well-known trick, and I'm sorry I don't know it yet.
Still learning. If anyone can help, I'd really appreciate it. :-)
Something like
SELECT list.id, list.activation_date, list_fieldJ.value,list_fieldC.value
FROM listings AS list
INNER JOIN listings_fields AS list_fieldJ ON list.id = list_fieldJ.id
and List_fieldj = 'Title'
INNER JOIN listings_fields AS list_fieldC ON list.id = list_fieldC.id
and List_fieldC = 'Category'
WHERE list.activation_date > SOME VALUE
ORDER BY list.activation_date DESC
Basically join once for title and then again for Category. Needless to say it's a pain if you have more than few value types to get out. And the above assumes that a job will always have title and category. Optional ones you'd need an outer join on.

Mysql many to many query

Having a mental block with going around this query.
I have the following tables:
review_list: has most of the data, but in this case the only important thing is review_id, the id of the record that I am currently interested in (int)
variant_list: model (varchar), enabled (bool)
variant_review: model (varchar), id (int)
variant_review is a many to many table linking the review_id in review_list to the model(s) in variant_list review and contains (eg):
..
test1,22
test2,22
test4,22
test1,23
test2,23... etc
variant_list is a list of all possible models and whether they are enabled and contains (eg):
test1,TRUE
test2,TRUE
test3,TRUE
test4,TRUE
what I am after in mysql is a query that when given a review_id (ie, 22) will return a resultset that will list each value in variant_review.model, and whether it is present for the given review_id such as:
test1,1
test2,1
test3,0
test4,1
or similar, which I can farm off to some webpage with a list of checkboxes for the types. This would show all the models available and whether each one was present in the table
Given a bit more information about the column names:
Select variant_list.model
, Case When variant_review.model Is Not Null Then 1 Else 0 End As HasReview
From variant_list
Left join variant_review
On variant_review.model = variant_list.model
And variant_review.review_id = 22
Just for completeness, if it is the case that you can have multiple rows in the variant_review table with the same model and review_id, then you need to do it differently:
Select variant_list.model
, Case
When Exists (
Select 1
From variant_review As VR
Where VR.model = variant_list.model
And VR.review_id = 22
) Then 1
Else 0
End
From variant_list