Multiple count(*) with left join in one shot - sql

I have 2 tables like this:
Table users:
+--------------------------+-----------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------------+-----------------+------+-----+---------+----------------+
| user_id | int(8) unsigned | NO | PRI | NULL | auto_increment |
| user_email | varchar(40) | NO | UNI | | |
| user_login | varchar(30) | YES | | NULL | |
| user_password | varchar(40) | YES | | NULL | |
| user_firstname | varchar(30) | YES | | NULL | |
| user_lastname | varchar(50) | YES | | NULL | |
+--------------------------+-----------------+------+-----+---------+----------------+
Table users_oauth to link users with oauth, if there is no oauth entry for a user, the user has created an account with an email/password:
+----------------------+-----------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------------+-----------------+------+-----+---------+----------------+
| oauth_id | int(8) unsigned | NO | PRI | NULL | auto_increment |
| oauth_user_id | int(8) unsigned | NO | MUL | NULL | |
| oauth_google_id | varchar(30) | YES | UNI | NULL | |
| oauth_facebook_id | varchar(30) | YES | UNI | NULL | |
| oauth_windowslive_id | varchar(30) | YES | UNI | NULL | |
+----------------------+-----------------+------+-----+---------+----------------+
For a count between two dates, to know how many new users I do the following for facebook oauth:
SELECT date_format(`user_date_accountcreated`, "%Y-%m-%d") AS date, COUNT(*) AS total FROM users
LEFT JOIN users_oauth ON users_oauth.oauth_user_id = users.user_id
WHERE (user_date_accountcreated BETWEEN '2016-10-01 00:00:00' AND '2016-10-15 23:59:59') AND oauth_facebook_id IS NOT NULL
GROUP BY year(user_date_accountcreated), month(user_date_accountcreated), day(user_date_accountcreated)
And another request for new users with google oauth, the only difference is oauth_google_id IS NOT NULL instead of oauth_facebook_id IS NOT NULL:
SELECT date_format(`user_date_accountcreated`, "%Y-%m-%d") AS date, COUNT(*) AS total FROM users
LEFT JOIN users_oauth ON users_oauth.oauth_user_id = users.user_id
WHERE (user_date_accountcreated BETWEEN '2016-10-01 00:00:00' AND '2016-10-15 23:59:59') AND oauth_google_id IS NOT NULL
GROUP BY year(user_date_accountcreated), month(user_date_accountcreated), day(user_date_accountcreated)
And the last one for windows live oauth:
SELECT date_format(`user_date_accountcreated`, "%Y-%m-%d") AS date, COUNT(*) AS total FROM users
LEFT JOIN users_oauth ON users_oauth.oauth_user_id = users.user_id
WHERE (user_date_accountcreated BETWEEN '2016-10-01 00:00:00' AND '2016-10-15 23:59:59') AND oauth_windowslive_id IS NOT NULL
GROUP BY year(user_date_accountcreated), month(user_date_accountcreated), day(user_date_accountcreated)
Is there any way to merge with 3 requests in only one with COUNT(*) AS total_facebook, COUNT(*) AS total_google, COUNT(*) AS total_windowslive ?
Thanks

You can do it by moving the where conditions to the count. This is called conditional aggregation.
Also when youleft join and use a where condition, it is converted to an inner join. To avoid it move the date condition on to the left join.
SELECT
date_format(`user_date_accountcreated`, "%Y-%m-%d") AS `date`,
COUNT(case when oauth_facebook_id IS NOT NULL then 1 end ) AS total_facebook,
COUNT(case when oauth_google_id IS NOT NULL then 1 end) AS total_google,
COUNT(case when oauth_windowslive_id IS NOT NULL then 1 end) AS total_windowslive
FROM users
LEFT JOIN users_oauth ON users_oauth.oauth_user_id = users.user_id
AND user_date_accountcreated BETWEEN '2016-10-01 00:00:00' AND '2016-10-15 23:59:59'
GROUP BY date_format(`user_date_accountcreated`, "%Y-%m-%d")
--year(user_date_accountcreated), month(user_date_accountcreated), day(user_date_accountcreated)

Related

Multiple queries resulting in minimum number of occurrences

I need to find the smallest number of documents retrieved by any subject based on exp_condition. Exp_condition from the subjects table contains a '1' and a '2' column.
Here are the tables:
subjects table:
+-----------------+--------------+
| Field | Type |
+-----------------+--------------+
| username | varchar(255) |
| user_type | varchar(10) |
| years | int |
| low_grade | int |
| high_grade | int |
| on_line | varchar(10) |
| on_line_sources | varchar(255) |
| location | varchar(5) |
| exp_condition | int |
+-----------------+--------------+
tasks table:
+------------+--------------+
| Field | Type |
+------------+--------------+
| username | varchar(255) |
| task | varchar(5) |
| confidence | int |
| sim_helpd | int |
+------------+--------------+
docs table:
+--------------+--------------+
| Field | Type |
+--------------+--------------+
| username | varchar(255) |
| task | varchar(5) |
| doc_type | varchar(10) |
| used_tool | int |
| relevant | int |
| motivational | int |
| concepts | int |
| background | int |
| grade_level | int |
| hands_on | int |
| attachments | int |
+--------------+--------------+
I'm able to generate the number of subjects and number of documents for both exp_condition values. I'm allowed to use multiple queries, but I'm not sure how.
Code for generating number of subjects for exp_condition 1 and 2:
select count(distinct(t2.username))
from tasks as t1
inner join subjects as t2
on t1.username = t2.username group by exp_condition;
Code for generating number of documents for exp_condition 1 and 2:
select count(*), exp_condition
from docs as t1
left join subjects as t2
on t1.username = t2.username
group by exp_condition;
Expected output: two separate numbers for smallest number of documents retrieved by any subject based on exp_condition.
Thanks in advance.
You can use a subquery or a CTE
SubQuery
SELECT exp_condition, MIN(A) as Tasks, MIN(B) as Docs FROM (
SELECT exp_condition, COUNT(DISTINCT t2.username) A, COUNT(DISTINCT (t3.username) B
FROM subjects s
LEFT JOIN tasks T2 ON s.username = t2.username
LEFT JOIN docs T3 ON s.username = t3.username
GROUP BY exp_condition
) A
GROUP BY ex_condition
CTE
;WITH CTE AS (
SELECT exp_condition, COUNT(DISTINCT t2.username) A, COUNT(DISTINCT (t3.username) B
FROM subjects s
LEFT JOIN tasks T2 ON s.username = t2.username
LEFT JOIN docs T3 ON s.username = t3.username
GROUP BY exp_condition
)
SELECT exp_condition, MIN(A) as Tasks, MIN(B) as Docs
FROM CTE
GROUP BY ex_condition

SQL select columns ordered by a query

I have tried and googled everything I can think of but I couldn't find an answer I have this simple database
technicien
+------------------+------------+------+-----+
| Field | Type | Null | Key |
+------------------+------------+------+-----+
| employe id | number(4) | NO | PRI |
| name | char(11) | NO | |
| salary | int(11) | NO | |
+------------------+------------+------+-----+
maintenance
+------------------+------------+------+---------+
| Field | Type | Null | Key |
+------------------+------------+------+---------+
| employe id | number(4) | NO | foreign |
| IP | char(11) | NO | foreign |
| maintenance_date | int(11) | NO | |
+------------------+------------+------+---------+
pc
+------------------+------------+------+---------+
| Field | Type | Null | Key |
+------------------+------------+------+---------+
| value | number(4) | NO | foreign |
| IP | char(11) | NO | PRI |
| price | int(11) | NO | |
+------------------+------------+------+---------+
what I need is to show the name, id, and salary of every technicien who has done a maintenance ordered by the total number of maintenances effected.
SELECT technicien.name, technicien.employeId, technicien.salary
FROM technicien
INNER JOIN maintenance
ON maintenance.employeId = technicien.employeId
INNER JOIN pc
ON maintenance.IP = pc.IP
ORDER BY COUNT(maintenance.maintenance_date)
What you need is to GROUPing with respect to the columns of table technicien, and ORDERing by count of maintenance tasks :
select t.*, count(0) cnt_maintenance
from technicien t
inner join maintenance m on ( t.employee_id = m.employee_id )
inner join pc p on ( p.value = m.IP )
group by t.employee_id, t.name, t.salary
order by count(0);
SQL Fiddle Demo

Sub-query producing an empty set

I am trying to provide a list of active quarterbacks who play for teams who had 20 or more sacks during the season.
I have information in two tables, one of them is a view(v_active_quarterbacks) which shows which quarterbacks are active, the other is the table team_game_stats.
I created the command that lists the teams that have 20+ sacks.
SELECT SUM(sacks)
FROM team_game_stats
GROUP BY team_code
HAVING SUM(sacks) > 20;
I now need to connect this to v_active_quarterbacks so that I can get a list. I have tried the following but it just provides an empty set.
SELECT player_code
FROM v_active_quaterbacks
INNER JOIN team_game_stats ON v_active_quaterbacks.team_code = team_game_stats.team_code
WHERE sacks IN (SELECT SUM(sacks)
FROM team_game_stats
GROUP BY team_code
HAVING SUM(sacks) > 20);
Here is the view description:
+-----------------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------------------+-------------+------+-----+---------+-------+
| player_code | int(11) | NO | | NULL | |
| first_name | varchar(30) | YES | | NULL | |
| last_name | varchar(30) | YES | | NULL | |
| team_code | int(11) | YES | | NULL | |
| uniform_number | varchar(3) | YES | | NULL | |
| passes_player_code | int(11) | YES | | NULL | |
| COUNT(passes.attempt) | bigint(21) | NO | | 0 | |
+-----------------------+-------------+------+-----+---------+-------+
At this point I am confused and stuck. Any help would be appreciated.
You could use a an inner join on subselect for team_code and sum
SELECT player_code , T.sum_sacks
FROM v_active_quaterbacks
INNER JOIN team_game_stats ON v_active_quaterbacks.team_code = team_game_stats.team_code
INNER JOIN (
SELECT team_code, SUM(sacks) sum_sacks
FROM team_game_stats
GROUP BY team_code
HAVING SUM(sacks) > 20
) T on T.team_code = v_active_quaterbacks.team_code

4 table query / join. getting duplicate rows

So I have written a query that will grab an order (this is for an ecommerce type site), and from that order id it will get all order items (ecom_order_items), print options (c_print_options) and images (images). The eoi_p_id is currently a foreign key from the images table.
This works fine and the query is:
SELECT
eoi_parentid, eoi_p_id, eoi_po_id, eoi_quantity,
i_id, i_parentid,
po_name, po_price
FROM ecom_order_items, images, c_print_options WHERE eoi_parentid = '1' AND i_id = eoi_p_id AND po_id = eoi_po_id;
The above would grab all the stuff I need for order #1
Now to complicate things I added an extra table (ecom_products), which needs to act in a similar way to the images table. The eoi_p_id can also point at a foreign key in this table too. I have added an extra field 'eoi_type' which will either have the value 'image', or 'product'.
Now items in the order could be made up of a mix of items from images or ecom_products. Whatever I try it either ends up with too many records, wont actually output any with eoi_type = 'product', and just generally wont work. Any ideas on how to achieve what I am after? Can provide SQL samples if needed?
SELECT
eoi_id, eoi_parentid, eoi_p_id, eoi_po_id, eoi_po_id_2, eoi_quantity, eoi_type,
i_id, i_parentid,
po_name, po_price, po_id,
ep_id
FROM ecom_order_items, images, c_print_options, ecom_products WHERE eoi_parentid = '9' AND i_id = eoi_p_id AND po_id = eoi_po_id
The above outputs duplicate rows and doesnt work as expected. Am I going about this the wrong way? Should I have seperate foreign key fields for the eoi_p_id depending it its an image or a product?
Should I be using JOINs?
Here is a mysql explain of the tables in question
ecom_products
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| ep_id | int(8) | NO | PRI | NULL | auto_increment |
| ep_title | varchar(255) | NO | | NULL | |
| ep_link | text | NO | | NULL | |
| ep_desc | text | NO | | NULL | |
| ep_imgdrop | text | NO | | NULL | |
| ep_price | decimal(6,2) | NO | | NULL | |
| ep_category | varchar(255) | NO | | NULL | |
| ep_hide | tinyint(1) | NO | | 0 | |
| ep_featured | tinyint(1) | NO | | 0 | |
+-------------+--------------+------+-----+---------+----------------+
ecom_order_items
+--------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+----------------+
| eoi_id | int(8) | NO | PRI | NULL | auto_increment |
| eoi_parentid | int(8) | NO | | NULL | |
| eoi_type | varchar(32) | NO | | NULL | |
| eoi_p_id | int(8) | NO | | NULL | |
| eoi_po_id | int(8) | NO | | NULL | |
| eoi_quantity | int(4) | NO | | NULL | |
+--------------+-------------+------+-----+---------+----------------+
c_print_options
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| po_id | int(8) | NO | PRI | NULL | auto_increment |
| po_name | varchar(255) | NO | | NULL | |
| po_price | decimal(6,2) | NO | | NULL | |
+------------+--------------+------+-----+---------+----------------+
images
+--------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+----------------+
| i_id | int(8) | NO | PRI | NULL | auto_increment |
| i_filename | varchar(255) | NO | | NULL | |
| i_data | longtext | NO | | NULL | |
| i_parentid | int(8) | NO | | NULL | |
+--------------+--------------+------+-----+---------+----------------+
You're missing a join condition for ecom_products either in the WHERE or FROM Clause. This is how it would be done using ANSI-92 joins
SELECT
eoi_id,
eoi_parentid,
eoi_p_id,
eoi_po_id,
eoi_po_id_2,
eoi_quantity,
eoi_type,
i_id,
i_parentid,
po_name,
po_price,
po_id,
ep_id
FROM
ecom_order_items,
LEFT JOIN images
ON i_id = eoi_p_id
LEFT JOIN c_print_options
ON po_id = eoi_po_id
INNER JOIN ecom_products
ON eoi_p_id = ep_id
WHERE
eoi_parentid = '9'
ANSI 92 joins are preferred and its a little clearer what's a join and what's filtering. That said you could just add AND eoi_p_id = ep_id to you where clause.
This is how I'd write the first query. I prefer to use joins.
SELECT eoi_parentid, eoi_p_id, eoi_po_id, eoi_quantity, i_id, i_parentid, po_name, po_price
FROM ecom_order_items
INNER JOIN images
ON i_id = eoi_p_id
INNER JOIN c_print_options
ON po_id = eoi_po_id
WHERE eoi_parentid = '1'
For your second query I would use a UNION on two queries, one for images and one for products.
SELECT eoi_id, eoi_parentid, eoi_p_id, eoi_po_id, eoi_po_id_2, eoi_quantity, eoi_type, i_id, i_parentid, po_name, po_price, po_id, ep_id
FROM ecom_order_items
INNER JOIN images
ON i_id = eoi_p_id
INNER JOIN c_print_options
ON po_id = eoi_po_id
WHERE eoi_type = 'image' AND i_id = eoi_p_id --Image conditions
AND eoi_parentid = '9'
AND po_id = eoi_po_id
UNION
SELECT eoi_id, eoi_parentid, eoi_p_id, eoi_po_id, eoi_po_id_2, eoi_quantity, eoi_type, i_id, i_parentid, po_name, po_price, po_id, ep_id
FROM ecom_order_items
INNER JOIN images
ON i_id = eoi_p_id
INNER JOIN c_print_options
ON po_id = eoi_po_id
WHERE eoi_type = 'product' AND ep_id = eoi_p_id -- Product conditions
AND eoi_parentid = '9'
AND po_id = eoi_po_id

help optimizing query (shows strength of two-way relationships between contacts)

i have a contact_relationship table that stores the reported strength of a relationship between one contact and another at a given point in time.
mysql> desc contact_relationship;
+------------------+-----------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+-----------+------+-----+-------------------+-----------------------------+
| relationship_id | int(11) | YES | | NULL | |
| contact_id | int(11) | YES | MUL | NULL | |
| other_contact_id | int(11) | YES | | NULL | |
| strength | int(11) | YES | | NULL | |
| recorded | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+------------------+-----------+------+-----+-------------------+-----------------------------+
now i want to get a list of two-way relationships between contacts (meaning there are two rows, one with contact a specifying a relationship strength with contact b and another with contact b specifying a strength for contact a -- the strength of the two-way relationship is the smaller of those two strength values).
this is the query i've come up with but it is pretty slow:
select
mrcr1.contact_id,
mrcr1.other_contact_id,
case when (mrcr1.strength < mrcr2.strength) then
mrcr1.strength
else
mrcr2.strength
end strength
from (
select
cr1.*
from (
select
contact_id,
other_contact_id,
max(recorded) as max_recorded
from
contact_relationship
group by
contact_id,
other_contact_id
) as cr2
inner join contact_relationship cr1 on
cr1.contact_id = cr2.contact_id
and cr1.other_contact_id = cr2.other_contact_id
and cr1.recorded = cr2.max_recorded
) as mrcr1,
(
select
cr3.*
from (
select
contact_id,
other_contact_id,
max(recorded) as max_recorded
from
contact_relationship
group by
contact_id,
other_contact_id
) as cr4
inner join contact_relationship cr3 on
cr3.contact_id = cr4.contact_id
and cr3.other_contact_id = cr4.other_contact_id
and cr3.recorded = cr4.max_recorded
) as mrcr2
where
mrcr1.contact_id = mrcr2.other_contact_id
and mrcr1.other_contact_id = mrcr2.contact_id
and mrcr1.contact_id != mrcr1.other_contact_id
and mrcr2.contact_id != mrcr2.other_contact_id
and mrcr1.contact_id <= mrcr1.other_contact_id;
anyone have any recommendations of how to speed it up?
note that because a user may specify the strength of his relationship with a particular user more than once, you must only grab the most recent record for each pair of contacts.
update: here is the result of explaining the query...
+----+-------------+----------------------+-------+----------------------------------------------------------------------------------------+------------------------------+---------+-------------------------------------+-------+--------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------------+-------+----------------------------------------------------------------------------------------+------------------------------+---------+-------------------------------------+-------+--------------------------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 36029 | Using where |
| 1 | PRIMARY | <derived4> | ALL | NULL | NULL | NULL | NULL | 36029 | Using where; Using join buffer |
| 4 | DERIVED | <derived5> | ALL | NULL | NULL | NULL | NULL | 36021 | |
| 4 | DERIVED | cr3 | ref | contact_relationship_index_1,contact_relationship_index_2,contact_relationship_index_3 | contact_relationship_index_2 | 10 | cr4.contact_id,cr4.other_contact_id | 1 | Using where |
| 5 | DERIVED | contact_relationship | index | NULL | contact_relationship_index_3 | 14 | NULL | 37973 | Using index |
| 2 | DERIVED | <derived3> | ALL | NULL | NULL | NULL | NULL | 36021 | |
| 2 | DERIVED | cr1 | ref | contact_relationship_index_1,contact_relationship_index_2,contact_relationship_index_3 | contact_relationship_index_2 | 10 | cr2.contact_id,cr2.other_contact_id | 1 | Using where |
| 3 | DERIVED | contact_relationship | index | NULL | contact_relationship_index_3 | 14 | NULL | 37973 | Using index |
+----+-------------+----------------------+-------+----------------------------------------------------------------------------------------+------------------------------+---------+-------------------------------------+-------+--------------------------------+
You are losing a lot lot lot of time selecting the most recent record. 2 options :
1- Change the way you are stocking data, and have a table with only recent record, and an other table more like historical record.
2- Use analytic request to select the most recent record, if your DBMS allows you to do this. Something like
Select first_value(strength) over(partition by contact_id, other_contact_id order by recorded desc)
from contact_relationship
Once you have the good record line, I think your query will go a lot faster.
Scorpi0's answer got me to thinking maybe I could use a temp table...
create temporary table mrcr1 (
contact_id int,
other_contact_id int,
strength int,
index mrcr1_index_1 (
contact_id,
other_contact_id
)
) replace as
select
cr1.contact_id,
cr1.other_contact_id,
cr1.strength from (
select
contact_id,
other_contact_id,
max(recorded) as max_recorded
from
contact_relationship
group by
contact_id, other_contact_id
) as cr2
inner join
contact_relationship cr1 on
cr1.contact_id = cr2.contact_id
and cr1.other_contact_id = cr2.other_contact_id
and cr1.recorded = cr2.max_recorded;
which i had to do twice (second time into a temp table named mrcr2) because mysql has a limitation where you can't alias the same temp table twice in one query.
with my two temp tables created my query then becomes:
select
mrcr1.contact_id,
mrcr1.other_contact_id,
case when (mrcr1.strength < mrcr2.strength) then
mrcr1.strength
else
mrcr2.strength
end strength
from
mrcr1,
mrcr2
where
mrcr1.contact_id = mrcr2.other_contact_id
and mrcr1.other_contact_id = mrcr2.contact_id
and mrcr1.contact_id != mrcr1.other_contact_id
and mrcr2.contact_id != mrcr2.other_contact_id
and mrcr1.contact_id <= mrcr1.other_contact_id;