Using Django I have the follow models:
class Player(models.Model):
name = models.CharField(max_length=64)
description = models.CharField(max_length=128)
groupPlayer = models.ForeignKey(GroupPlayer)
class GroupPlayer(models.Model):
description = models.CharField(max_length=128)
name = models.CharField(max_length=32)
Every player is linked to a group. Some group has no player. My aim is to translate the follow query in Django (which include LEFT OUTER JOIN and Group by):
select GroupPlayer.description, GroupPlayer.name, COUNT(Player.name) as gplayer
from (GroupPlayer LEFT OUTER JOIN Player ON GroupPlayer.id = Player.groupPlayer_id)
GROUP BY GroupPlayer.id
So far I have created 3 queryset, but I cannot put them together in order to get the result I wish:
queryset_player = Player.objects.values('groupPlayer_id').annotate(total=Count('groupPlayer__id'))
queryset_group = GroupPlayer.objects.select_retated(queryset_player)
queryset_group2 = GroupPlayer.objects.all().prefetch_related('player_set')
Anyone can help with that LEFT OUTER JOIN, Group by and count?
Thanks
from django.db.models import Count
groups = GroupPlayer.objects.annotate(num_players=Count('player'))
groups[0].num_players
Aggragation
Related
I have a view that gets data from different tables and views and returns it grouped.
Here is the code of view
CREATE VIEW analysis.vwOnboardingTableData
AS
SELECT USC.UseCaseCapabilityId,
USC.Name,
USC.JobId,
USC.Id AS UseCaseId,
COUNT(JSOR.SupplierKey) AS SupplierCount,
JSOR.OnboardingRouteId,
SUM(JS.SupplierPaymentDocCount) AS PaymentsCount,
SUM(JS.SupplierInvoiceDocCount) AS InvoicesCount,
SUM(JS.SupplierPurchaseOrderDocCount) AS PoCount,
SUM(JS.SupplierSpendValueCurrencyJob) AS Spend,
JM.JobSpendSource
FROM analysis.UseCases AS USC
INNER JOIN analysis.UseCaseSupplierStates AS JSOR
ON USC.Id = JSOR.UseCaseId
AND USC.JobId = JSOR.JobId
INNER JOIN analysis.vwJobSupplierWithScores AS JS
ON JSOR.SupplierKey = JS.SupplierKey
INNER JOIN analysis.vwJobMetrics AS JM
ON USC.JobId = JM.JobId
WHERE (USC.IsTemplate = 0)
AND (JSOR.IsQualified = 1)
GROUP BY JSOR.OnboardingRouteId,
USC.Name,
USC.JobId,
JM.JobSpendSource,
USC.UseCaseCapabilityId, USC.Id
go
For now, it is grouped by multiple fields to get those fields in SELECT. I need to group it only by USC.UseCaseCapabilityId and get the same fields without any aggregating functions like SUM, etc.
How I can achieve this?
I have two classes in Django linked through a ManyToManyField (the User class is the built-in User model):
from django.contrib.auth.models import User
class Activity():
participants = models.ManyToManyField(User, related_name='activity_participants')
I want to find all the activities in which two users are simultaneously participating.
I managed to solve my problem using a raw query (my app name is "core", therefore the "core" prefix in the table names):
SELECT ca.id FROM core_activity_participants AS cap, core_activity AS ca
INNER JOIN core_activity_participants AS cap2 ON cap.activity_id
WHERE cap.user_id == 1 AND cap2.user_id == 2
AND cap.activity_id == cap2.activity_id
AND ca.id == cap.activity_id
However, if possible, I'd like to avoid using raw queries, since it breaks uniformity from the rest of my app. How could I make this query, or one equivalent to it, using Django's ORM?
If you're using Django 1.11 or later the intersection queryset method will give you the records you want.
# u1 and u2 are User instances
u1_activities = Activity.objects.filter(participants=u1)
u2_activities = Activity.objects.filter(participants=u2)
common_activities = u1_activities.intersection(u2_activities)
Will produce a query something like this:
SELECT "core_activity"."id"
FROM "core_activity"
INNER JOIN "core_activity_participants"
ON ("core_activity"."id" = "core_activity_participants"."activity_id")
WHERE "core_activity_participants"."user_id" = 1
INTERSECT
SELECT "core_activity"."id"
FROM "core_activity"
INNER JOIN "core_activity_participants"
ON ("core_activity"."id" = "core_activity_participants"."activity_id")
WHERE "core_activity_participants"."user_id" = 2
You can also add extra querysets to the intersection if you want to check for activity overlap between more than 2 users.
Update:
Another approach, which works with older Django versions, would be
u1_activities = u1.activity_participants.values_list('pk', flat=True)
common_activities = u2.activity_participants.filter(pk__in=u1_activities)
Which produces a query like
SELECT "core_activity"."id"
FROM "core_activity"
INNER JOIN "core_activity_participants"
ON ("core_activity"."id" = "core_activity_participants"."activity_id")
WHERE (
"core_activity_participants"."user_id" = 2
AND "core_activity"."id" IN (
SELECT U0."id"
FROM "core_activity" U0
INNER JOIN "core_activity_participants" U1
ON (U0."id" = U1."activity_id")
WHERE U1."user_id" = 1
)
)
I am new to JOINS and testing my query, but it's just not working for me...
The situation:
The database has got the following columns:
links (contains unique data)
cl_link (contains the relation between links & cats)
cats (cat. descriptions
images (contains multiple images of one link)
cfvalues (contains the values of the multiple custom fiels
customfields (contains the multiple customfields)
I am using the following query, but the Joins are not working for me. Because I only get one image while sometimes there are multiple. And I only get one customfield instead of multiple and I get none cfvalues.
I guess something is wrong with the JOINS, but I am not sure. Can somebody help me out here?
The SQL
SELECT DISTINCT
rqypj_mt_links.link_name,
rqypj_mt_links.link_desc,
rqypj_mt_links.address,
rqypj_mt_links.city,
rqypj_mt_links.state,
rqypj_mt_links.country,
rqypj_mt_links.postcode,
rqypj_mt_links.telephone,
rqypj_mt_links.fax,
rqypj_mt_links.email,
rqypj_mt_links.website,
rqypj_mt_links.price,
rqypj_mt_links.lat,
rqypj_mt_links.lng,
rqypj_mt_links.zoom,
rqypj_mt_cats.cat_name,
rqypj_mt_images.filename,
rqypj_mt_cfvalues.value,
rqypj_mt_customfields.caption
FROM rqypj_mt_links
LEFT JOIN rqypj_mt_cl
ON rqypj_mt_links.link_id = rqypj_mt_cl.link_id
LEFT JOIN rqypj_mt_cats
ON rqypj_mt_cl.cat_id = rqypj_mt_cats.cat_id
LEFT JOIN rqypj_mt_images
ON rqypj_mt_links.link_id = rqypj_mt_images.link_id
LEFT JOIN rqypj_mt_cfvalues
ON rqypj_mt_links.link_id = rqypj_mt_cfvalues.link_id
LEFT JOIN rqypj_mt_customfields
ON rqypj_mt_customfields.cf_id = rqypj_mt_customfields.cf_id LIMIT 100
Thanks in advance!
Jelte
your last condition doesn't look right:
on rqypj_mt_customfields.cf_id = rqypj_mt_customfields.cf_id
translates to 1=1
Shouldn't it be:
on rqypj_mt_customfields.cf_id = rqypj_mt_cfvalues.cf_id
Probably because you don't have an order by and are using limit.
Change it to
order by rqypj_mt_links.link_id, rqypj_mt_cl.cat_id
limit 100
and then your multiple pictures for the same link should be together.
Also please consider use of alias to make your code easier to read:
SELECT DISTINCT
links.link_name,
links.link_desc,
links.address,
links.city,
links.state,
links.country,
links.postcode,
links.telephone,
links.fax,
links.email,
links.website,
links.price,
links.lat,
links.lng,
links.zoom,
cats.cat_name,
images.filename,
cfvalues.value,
--custom.caption
FROM rqypj_mt_links links
LEFT JOIN rqypj_mt_cl cl ON links.link_id = cl.link_id
LEFT JOIN rqypj_mt_cats cats ON cl.cat_id = cats.cat_id
LEFT JOIN rqypj_mt_images images ON links.link_id = images.link_id
LEFT JOIN rqypj_mt_cfvalues cfvalues ON links.link_id = cfvalues.link_id
--LEFT JOIN rqypj_mt_customfields custom ON custom.cf_id = custom.cf_id
ORDER BY links.link_id, cats.cat_id
LIMIT 100
I have a DB2 query as follows:
SELECT DISTINCT RETAILMASTERFILE.DOIDCD AS "RETAILMASTERFILE_DOIDCD",
RETAILMASTERFILE.COCOMO AS "RETAILMASTERFILE_COCOMO",
#XENOS.CUSTREF AS "XENOS_CUSTREF",
#XENOS.ADDUDT AS "XENOS_ADDUDT",
#XENOS.ADUPDD AS "XENOS_ADUPDD",
#XENOS.ADUPDT AS "XENOS_ADUPDT",
#XENOS.ADSTAT AS "XENOS_ADSTAT"
FROM RETAILMASTERFILE INNER JOIN
#XENOS ON RETAILMASTERFILE.DOCOMP = #XENOS.ADCOMP
AND RETAILMASTERFILE.COCOMO = #XENOS.ADDELN
WHERE (RETAILMASTERFILE.DOIDCD = 'CUST008')
AND (RETAILMASTERFILE.COCOMO = '345126032')
AND (RETAILMASTERFILE.DOCOMP = 'LONDON')
The problem is #XENOS.ADUPDT may not be unique which gives me an unwanted duplicate record.
Is there any way I can exclude this from consideration ? Everything I've tried so far within my limited knowledge and crude understanding of group by has so far broken my query.
Use GROUP BY instead:
SELECT RETAILMASTERFILE.DOIDCD AS "RETAILMASTERFILE_DOIDCD",
RETAILMASTERFILE.COCOMO AS "RETAILMASTERFILE_COCOMO",
#XENOS.CUSTREF AS "XENOS_CUSTREF",
#XENOS.ADDUDT AS "XENOS_ADDUDT",
#XENOS.ADUPDD AS "XENOS_ADUPDD",
MAX(#XENOS.ADUPDT) AS "XENOS_ADUPDT",
#XENOS.ADSTAT AS "XENOS_ADSTAT"
FROM RETAILMASTERFILE INNER JOIN
#XENOS
ON RETAILMASTERFILE.DOCOMP = #XENOS.ADCOMP AND
RETAILMASTERFILE.COCOMO = #XENOS.ADDELN
WHERE (RETAILMASTERFILE.DOIDCD = 'CUST008') AND (RETAILMASTERFILE.COCOMO = '345126032') AND
(RETAILMASTERFILE.DOCOMP = 'LONDON')
GROUP BY RETAILMASTERFILE.DOIDCD,
RETAILMASTERFILE.COCOMO,
#XENOS.CUSTREF,
#XENOS.ADDUDT,
#XENOS.ADUPDD,
#XENOS.ADSTAT;
I'm trying to run this query:
SELECT
Destaque.destaque, Noticia.id, Noticia.antetitulo,
Noticia.titulo, Noticia.lead, Noticia.legenda,
Noticia.publicacao, Seccao.descricao, Album.pasta,
Foto.ficheiro, Foto.descricao, Cronista.nome,
Cronista.profissao, Cronista.ficheiro,
AudioFile.*, AudioCollection.*, VideoFile.*, VideoCollection.*
FROM
nt_highlights AS Destaque
LEFT JOIN nt_noticias AS Noticia ON Destaque.noticia_id = Noticia.id
LEFT JOIN mm_fotos AS Foto ON Noticia.foto_id = Foto.id
LEFT JOIN nt_temas AS Seccao ON Noticia.tema_id = Seccao.id
LEFT JOIN mm_albuns AS Album ON Foto.album_id = Album.id
LEFT JOIN nt_cronistas AS Cronista ON Cronista.id = Noticia.cronista_id
LEFT JOIN ntNoticias_mmFiles AS Rel ON Rel.noticia_id = Noticia.id
LEFT JOIN mm_files AS AudioFile ON AudioFile.id = Rel.file_id
LEFT JOIN mm_coleccoes AS AudioCollection ON AudioFile.coleccao_id = AudioCollection.id
LEFT JOIN mm_files AS VideoFile ON VideoFile.id = Rel.file_id
LEFT JOIN mm_coleccoes AS VideoCollection ON VideoFile.coleccao_id = VideoCollection.id
WHERE
Destaque.area_id = 1
AND Noticia.paraPublicacao = 1
AND Noticia.publicacao <= NOW()
AND (AudioFile.mimeType != '' OR AudioFile.id IS NULL)
AND (VideoFile.mimeType = '' OR VideoFile.id IS NULL)
ORDER BY
Destaque.destaque
This will get me a number of articles (from nt_noticias) and the idea is to get at the same time a Video and an Audio file from the mm_files table.
What happens is that when I have an article with a sound and a video, MySQL will return 4 rows:
with the sound (video is null)
with the video (sound is null)
with all nulls
with the sound AND the video
How can I "force" it to return just one row per article with any existing video AND audio associated? What am I doing wrong here?
I think you want something like this:
SELECT
Destaque.destaque, Noticia.id, Noticia.antetitulo,
Noticia.titulo, Noticia.lead, Noticia.legenda,
Noticia.publicacao, Seccao.descricao, Album.pasta,
Foto.ficheiro, Foto.descricao, Cronista.nome,
Cronista.profissao, Cronista.ficheiro,
AudioFile.*, AudioCollection.*, VideoFile.*, VideoCollection.*
FROM
nt_highlights AS Destaque
LEFT JOIN nt_noticias AS Noticia ON Destaque.noticia_id = Noticia.id
LEFT JOIN mm_fotos AS Foto ON Noticia.foto_id = Foto.id
LEFT JOIN nt_temas AS Seccao ON Noticia.tema_id = Seccao.id
LEFT JOIN mm_albuns AS Album ON Foto.album_id = Album.id
LEFT JOIN nt_cronistas AS Cronista ON Cronista.id = Noticia.cronista_id
LEFT JOIN ntNoticias_mmFiles AS AudioRel ON Rel.noticia_id = Noticia.id
AND AudioRel.file_id IN (
SELECT file_id
FROM ntNoticias_mmFiles
WHERE noticia_id = Noticia.id AND IsAudioFile = 1 /* whatever the check is */
LIMIT 1
)
LEFT JOIN mm_files AS AudioFile ON AudioFile.id = Rel.file_id
LEFT JOIN mm_coleccoes AS AudioCollection ON AudioFile.coleccao_id = AudioCollection.id
LEFT JOIN ntNoticias_mmFiles AS VideoRel ON VideoRel.noticia_id = Noticia.id
AND VideoRel.file_id IN (
SELECT file_id
FROM ntNoticias_mmFiles
WHERE noticia_id = Noticia.id AND IsVideoFile = 1 /* whatever the check is */
LIMIT 1
)
LEFT JOIN mm_files AS VideoFile ON VideoFile.id = Rel.file_id
AND VideoFile.IsVideoFile = 1
LEFT JOIN mm_coleccoes AS VideoCollection ON VideoFile.coleccao_id = VideoCollection.id
WHERE
Destaque.area_id = 1
AND Noticia.paraPublicacao = 1
AND Noticia.publicacao <= NOW()
ORDER BY
Destaque.destaque
My thought was this:
You want one audio file and one video file, at most. There are several files available per Noticia, so you need to make sure that a maximum of one file per type gets into the join. This also means you have to join in the ntNoticias_mmFiles table twice — once per type.
This is what the sub-queries in the join conditions are supposed to do: Select one row per file type. Going on from there you LEFT JOIN the rest of the data in, just like you already do.
The JOIN will return all the combinations, that's the problem.
If you only have one audio and/or videofile per article then you might want to look at subselects.
In SQL Server this would look something like (untested code):
SELECT title,
(select TOP 1 audio from audio where audio.aid = articles.id) as Audio,
(select TOP 1 video from video where video.aid = articles.id) as Video
FROM articles
Be careful that on large datasets this can perform poorly as the subselects in this example are executed individually for each row returned to the outer query. For example, if you return 10,000 articles then a total of 20,001 queries would actually be executed on the server.
There are other possible answers to overcome this but they get more involved (I suspect you could do something with a derived table but it eludes me at the moment).
You probably want to optimize that join query into a view. It's a large query, and with that many joins, it's going to be pretty inefficient. Plus, a view helps you debug the joins and will basically simplifies by allowing you to write your joins (in the view) and the WHERE clause (in your select from the view) separately, which can help with debugging the queries.