I am using the following CTE. The first part collects all unique people and the second left joins the unique people with events during a particular time frame. I am expecting that all the rows be returned from my unique people table even if they don't have an event within the time frame. But this doesn't appear to be the case.
WITH DISTINCT_ATTENDING(ATTENDING) AS
(
SELECT DISTINCT ATTENDING
FROM PEOPLE
WHERE ATTENDING IS NOT NULL
), -- returns 62 records
EVENT_HISTORY(ATTENDING, TOTAL) AS
(
SELECT C.ATTENDING,
COUNT(C.ID)
FROM DISTINCT_ATTENDING D
LEFT JOIN PEOPLE C
ON C.ATTENDING = D.ATTENDING
AND TO_DATE(C.DATE, 'YYYYMMDD') < TO_DATE('20140101', 'YYYYMMDD')
GROUP BY C.ATTENDING
ORDER BY C.ATTENDING
)
SELECT * FROM EVENT_HISTORY; -- returns 49 rows
What am I doing wrong here?
Jonny
The problem is inthe column "C.ATTENDING", just change for "D.ATTENDING"
SELECT D.ATTENDING,
COUNT(C.ID)
FROM DISTINCT_ATTENDING D
LEFT JOIN PEOPLE C
ON C.ATTENDING = D.ATTENDING
AND TO_DATE(C.DATE, 'YYYYMMDD') < TO_DATE('20140101', 'YYYYMMDD')
GROUP BY D.ATTENDING
ORDER BY D.ATTENDING
Your query seems too complicated. I think the following does the same thing:
SELECT P.ATTENDING,
SUM(CASE WHEN TO_DATE(P.DATE, 'YYYYMMDD') < TO_DATE('20140101', 'YYYYMMDD')
THEN 1 ELSE 0 END)
FROM PEOPLE P
WHERE P.ATTENDING IS NOT NLL
GROUP BY P.ATTENDING
ORDER BY P.ATTENDING ;
Your problem is that you are aggregating by a column in the second table of a left join. This is NULL when there is no match.
Related
I need to perform multiple JOINs, I am grabbing the data from multiple tables and JOINing on id. The tricky part is that one table I need to join twice. Here is the code:
(
SELECT
content.brand_identifier AS brand_name,
CAST(timestamp(furniture.date) AS DATE) AS order_date,
total_hearst_commission
FROM
`furniture_table` AS furniture
LEFT JOIN `content_table` AS content ON furniture.site_content_id = content.site_content_id
WHERE
(
timestamp(furniture.date) >= TIMESTAMP('2020-06-01 00:00:00')
)
)
UNION
(
SELECT
flowers.a_merchant_name AS merchant_name
FROM
`flowers_table` AS flowers
LEFT JOIN `content` AS content ON flowers.site_content_id = content.site_content_id
)
GROUP BY
1,
2,
3,
4
ORDER BY
4 DESC
LIMIT
500
I thought I could use UNION but it gives me an error Syntax error: Expected keyword ALL or keyword DISTINCT but got "("
I'm not able to comment, but like GHB states, the queries do not have the same number of columns; therefore, UNION will not work here.
I think it would be helpful to know why sub-queries are needed in the first place. I'm guessing this query does not product the results you want, so please elaborate on why that is.
select
f.a_merchant_name as merchant_name,
c.brand_identifier as brand_name,
CAST(timestamp(f.date) AS DATE) AS order_date,
total_hearst_commission
from furniture_table f
left join content_table c on c.site_content_id = f.site_content_id
where timestamp(f.date) >= TIMESTAMP('2020-06-01 00:00:00')
group by 1,2,3,4
Given some rows with duplicate names and different timestamps, I would like to select the row with the newest timestamp, if the duplicate name occurs within say, 45 minutes, of the first timestamp.
Here's what worked in PostgreSQL:
SELECT i.ts AS base_timestamp, j.ts AS newer_timestamp, i.name
FROM tbl i
LEFT JOIN LATERAL
(SELECT j.ts
FROM tbl j
WHERE i.name = j.name
AND j.ts > i.ts
AND j.ts < (i.ts + INTERVAL '45 minutes')
) j ON TRUE
WHERE j.ts is NULL
Great explanation of LATERAL here:
https://heap.io/blog/engineering/postgresqls-powerful-new-join-type-lateral
LATERAL join is like a SQL foreach loop, in which PostgreSQL will iterate over each row in a result set and evaluate a subquery using that row as a parameter.
So it's like a correlated subquery, but in the join.
Then I simply take only the rows where there is no newer timestamp (WHERE j.ts is NULL).
How can I do this in BigQuery?
EDIT: I've created an example of the PostgreSQL grouping on SQLFiddle as requested in the comments.
Input:
('Duplication Example','2019-06-22 19:10:25'),
('Duplication Example','2019-06-22 23:58:31'),
('Duplication Example','2019-06-23 00:08:00')
Output (middle row having timestamp 23:58:31 removed):
base_timestamp newer_timestamp name
2019-06-22T19:10:25Z (null) Duplication Example
2019-06-23T00:08:00Z (null) Duplication Example
Your case looks kind of like a task for window functions. But since you seem to be interested in lateral joins more than in solving the problem you presented:
In BigQuery there is afaik only an implicit version of lateral joins: when joining with unnested arrays.
This showcases the idea:
WITH t AS (
SELECT 'a' as id, [2,3] as arr
UNION ALL SELECT 'b', [56, 7]
)
SELECT * EXCEPT(arr)
FROM t LEFT JOIN UNNEST(arr)
This can be archived with a WINDOW function.
SELECT
name,
MAX(timestamp) AS timestamp_new
FROM
(
SELECT
i.name,
COUNT(*) OVER (PARTITION BY i.name ORDER BY i.ts RANGE BETWEEN 45 * 60 * 1000 PRECEDING AND CURRENT ROW) as 45_window_count,
i.ts AS timestamp
FROM
tbl i
)
WHERE 45_window_count > 1
GROUP BY user
If I have following table in Postgres:
order_dtls
Order_id Order_date Customer_name
-------------------------------------
1 11/09/17 Xyz
2 15/09/17 Lmn
3 12/09/17 Xyz
4 18/09/17 Abc
5 15/09/17 Xyz
6 25/09/17 Lmn
7 19/09/17 Abc
I want to retrieve such customer who has placed orders on 2 consecutive days.
In above case Xyz and Abc customers should be returned by query as result.
There are many ways to do this. Use an EXISTS semi-join followed by DISTINCT or GROUP BY, should be among the fastest.
Postgres syntax:
SELECT DISTINCT customer_name
FROM order_dtls o
WHERE EXISTS (
SELEST 1 FROM order_dtls
WHERE customer_name = o.customer_name
AND order_date = o.order_date + 1 -- simple syntax for data type "date" in Postgres!
);
If the table is big, be sure to have an index on (customer_name, order_date) to make it fast - index items in this order.
To clarify, since Oto happened to post almost the same solution a bit faster:
DISTINCT is an SQL construct, a syntax element, not a function. Do not use parentheses like DISTINCT (customer_name). Would be short for DISTINCT ROW(customer_name) - a row constructor unrelated to DISTINCT - and just noise for the simple case with a single expression, because Postgres removes the pointless row wrapper for a single element automatically. But if you wrap more than one expression like that, you get an actual row type - an anonymous record actually, since no row type is given. Most certainly not what you want.
What is a row constructor used for?
Also, don't confuse DISTINCT with DISTINCT ON (expr, ...). See:
Select first row in each GROUP BY group?
Try something like...
SELECT `order_dtls`.*
FROM `order_dtls`
INNER JOIN `order_dtls` AS mirror
ON `order_dtls`.`Order_id` <> `mirror`.`Order_id`
AND `order_dtls`.`Customer_name` = `mirror`.`Customer_name`
AND DATEDIFF(`order_dtls`.`Order_date`, `mirror`.`Order_date`) = 1
The way I would think of it doing it would be to join the table the date part with itselft on the next date and joining it with the Customer_name too.
This way you can ensure that the same customer_name done an order on 2 consecutive days.
For MySQL:
SELECT distinct *
FROM order_dtls t1
INNER JOIN order_dtls t2 on
t1.Order_date = DATE_ADD(t2.Order_date, INTERVAL 1 DAY) and
t1.Customer_name = t2.Customer_name
The result you should also select it with the Distinct keyword to ensure the same customer is not displayed more than 1 time.
For postgresql:
select distinct(Customer_name) from your_table
where exists
(select 1 from your_table t1
where
Customer_name = your_table.Customer_name and Order_date = your_table.Order_date+1 )
Same for MySQL, just instead of your_table.Order_date+1 use: DATE_ADD(your_table.Order_date , INTERVAL 1 DAY)
This should work:
SELECT A.customer_name
FROM order_dtls A
INNER JOIN (SELECT customer_name, order_date FROM order_dtls) as B
ON(A.customer_name = B.customer_name and Datediff(B.Order_date, A.Order_date) =1)
group by A.customer_name
Hi guys, I am stuck with a stubborn problem which I am unable to solve. Am trying to compile a report wherein all the dates coming from different tables would need to come into a single date field in the report. Ofcourse, the max or the most recent date from all these date columns needs to be added to the single date column for the report. I have multiple users of multiple branches/courses for whom the report would be generated.
There are multiple blogs and the latest date w.r.t to the blogtitle needs to be grouped, i.e. max(date_value) from the six date columns should give the greatest or latest date for that blogtitle.
Expected Result:
select u.batch_uid as ext_person_key, u.user_id, cm.batch_uid as ext_crs_key, cm.crs_id, ir.role_id as
insti_role, (CASE when b.JOURNAL_IND = 'N' then
'BLOG' else 'JOURNAL' end) as item_type, gm.title as item_name, gm.disp_title as ITEM_DISP_NAME, be.blog_pk1 as be_blogPk1, bc.blog_entry_pk1 as bc_blog_entry_pk1,bc.pk1,
b.ENTRY_mod_DATE as b_ENTRY_mod_DATE ,b.CMT_mod_DATE as BlogCmtModDate, be.CMT_mod_DATE as be_cmnt_mod_Date,
b.UPDATE_DATE as BlogUpDate, be.UPDATE_DATE as be_UPDATE_DATE,
bc.creation_date as bc_creation_date,
be.CREATOR_USER_ID as be_CREATOR_USER_ID , bc.creator_user_id as bc_creator_user_id,
b.TITLE as BlogTitle, be.TITLE as be_TITLE,
be.DESCRIPTION as be_DESCRIPTION, bc.DESCRIPTION as bc_DESCRIPTION
FROM users u
INNER JOIN insti_roles ir on u.insti_roles_pk1 = ir.pk1
INNER JOIN crs_users cu ON u.pk1 = cu.users_pk1
INNER JOIN crs_mast cm on cu.crsmast_pk1 = cm.pk1
INNER JOIN blogs b on b.crsmast_pk1 = cm.pk1
INNER JOIN blog_entry be on b.pk1=be.blog_pk1 AND be.creator_user_id = cu.pk1
LEFT JOIN blog_CMT bc on be.pk1=bc.blog_entry_pk1 and bc.CREATOR_USER_ID=cu.pk1
JOIN gradeledger_mast gm ON gm.crsmast_pk1 = cm.pk1 and b.grade_handler = gm.linkId
WHERE cu.ROLE='S' AND BE.STATUS='2' AND B.ALLOW_GRADING='Y' AND u.row_status='0'
AND u.available_ind ='Y' and cm.row_status='0' and and u.batch_uid='userA_157'
I am getting a resultset for the above query with multiple date columns which I want > > to input into a single columnn. The dates have to be the most recent, i.e. max of the dates in the date columns.
I have successfully done the Unpivot by using a view to store the above
resultset and put all the dates in one column. However, I do not
want to use a view or a table to store the resultset and then do
Unipivot simply because I cannot keep creating views for every user
one would query for.
The max(date_value) from the date columns need to be put in one single column. They are as follows:
* 1) b.entry_mod_date, 2) b.cmt_mod_date ,3) be.cmt_mod_date , 4) b.update_Date ,5) be.update_date, 6) bc.creation_date *
Apologies that I could not provide the desc of all the tables and the
fields being used.
Any help to get the above mentioned max of the dates from these
multiple date columns into a single column without using a view or a
table would be greatly appreciated.*
It is not clear what results you want, but the easiest solution is to use greatest().
with t as (
YOURQUERYHERE
)
select t.*,
greatest(entry_mod_date, cmt_mod_date, cmt_mod_date, update_Date,
update_date, bc.creation_date
) as greatestdate
from t;
select <columns>,
case
when greatest (b_ENTRY_mod_DATE) >= greatest (BlogCmtModDate) and greatest(b_ENTRY_mod_DATE) >= greatest(BlogUpDate)
then greatest( b_ENTRY_mod_DATE )
--<same implementation to compare each time BlogCmtModDate and BlogUpDate separately to get the greatest then 'date'>
,<columns>
FROM table
<rest of the query>
UNION ALL
Select <columns>,
case
when greatest (be_cmnt_mod_Date) >= greatest (be_UPDATE_DATE)
then greatest( be_cmnt_mod_Date )
when greatest (be_UPDATE_DATE) >= greatest (be_cmnt_mod_Date)
then greatest( be_UPDATE_DATE )
,<columns>
FROM table
<rest of the query>
UNION ALL
Select <columns>,
GREATEST(bc_creation_date)
,<columns>
FROM table
<rest of the query>
I have a problem as I am not so strong on queries.
I have a query with consists of a union of two select queries :
SELECT em.emp_code,
em.emp_name,
COALESCE(SUM(pe.hours_allotted),0) AS hours,
pe.dated
FROM employee_master em
LEFT JOIN project_employee pe ON (pe.Emp_code = em.emp_code)
WHERE (dated >= '2011-03-14'
AND dated < '2011-03-20' )
OR dated IS NULL
GROUP BY em.emp_code
UNION
(SELECT em.emp_code,
em.emp_name,
'0' AS hours,
pe.dated
FROM employee_master em
LEFT JOIN project_employee pe ON (pe.Emp_code = em.emp_code)
WHERE (dated >= '2011-03-14'
AND dated < '2011-03-20' )
OR dated IS NOT NULL
GROUP BY em.Emp_code)
ORDER BY emp_name;
Now the result sets are returning for example as:
ecode ename hours
----------------------
201 Alak basu 10
201 alak basu 0
The first result is from first select statement of the union where hours = 10
and hours = 0 is from second select statement of union.
What I want is:
ecode ename hours
----------------------------
201 alak basu 10
Like in the case there should be only one result per ecode. How to group it like summing up the hours on as group by ecode so that it gives me only one result as above?
You can always do something like:
select emp_code, min(emp_name) as emp_name, sum(hours)
from (
<your original query here>
) as e
group by emp_code
order by emp_name;
If the desired result is to sum all hours for a single employee code into a single row, and the second query after the UNION will only ever return zero hours, it seems like the best solution here is to get rid of the UNION.
EDIT: After further clarification, here's what I think the SQL should probably look like:
SELECT em.emp_code,
em.emp_name,
COALESCE(pe.hours, 0) AS hours
FROM employee_master em
LEFT JOIN (
SELECT emp_code,
SUM(hours_allotted) AS hours
FROM project_employee
WHERE dated >= '2011-03-14' AND
dated < '2011-03-20'
GROUP BY emp_code
) pe ON (pe.emp_code = em.emp_code)
ORDER BY em.emp_name;
What it's doing:
Perform a subquery to filter all project_employee entries to the ones within the specified date range. (Note that there is no need for NULL or NOT NULL checks at all here. Either the date is in range, or it is not.)
Sum the hours for each employee code generated in the subquery.
Take all employees in the employee_master table, and search for matching entries in the filtered, summed project_employee subquery result set. (Since it is a LEFT JOIN, every employee in the master table will have an entry, even if none of the filtered project_employee entries matched.)
In the case that there is no match, the pe.hours column will be NULL, causing the COALESCE to revert to its second value of zero.
Order results by emp_name.