I have the following SQL query where I have two datasets CSDB and MMSR. I merge these two datasets to th Final dataset. So far everything works fine. Now I want to aggregate the data on trade_date but I get the error message: AnalysisException: Could not resolve column/field reference: 'transaction_nominal_value'. I am working in Impala. How can I resolve the problem?
--CSDB
WITH CSDB AS (
SELECT isin, nominal_currency, amount_outstanding, issue_price, amount_issued, yield, original_maturity, residual_maturity
FROM csdb_pq
where nominal_currency = "EUR"
),
--MMSR
MMSR AS (
SELECT transaction_nominal_amount, maturity_days, deal_rate, collateral_haircut, collateral_isin, collateral_nominal_amount, trade_date
FROM datashop_store_business_mmsr.secured_vl_pq
WHERE collateral_isin IS NOT NULL
),
---Join
Final AS (
SELECT *
FROM MMSR
LEFT JOIN CSDB
ON MMSR.collateral_isin = CSDB.isin
)
--Aggregate Data
SELECT trade_date, AVG(transaction_nominal_amount)
FROM Final
GROUP BY trade_date;
Should be transaction_nominal_amount, not transaction_nominal_value per your table elements within your CTE. Just a simple column name error and it should work.
Related
SQL Code:
SELECT community_table.community_name,
community_table.id,
DATE(timestamp) as date,
ifnull(COUNT(distinct app_opened.user_id), 0) as num_opened_DAU,
lag(COUNT(distinct app_opened.user_id)) OVER
(ORDER BY community_table.community_name, community_table.id, DATE(timestamp)) as pre_Value
FROM *** app_opened
LEFT JOIN (
SELECT DISTINCT id, community_id_2, context_traits_first_name, context_traits_last_name
FROM (
SELECT *
FROM ***,
UNNEST (JSON_EXTRACT_ARRAY(context_traits_community_ids, "$")) as community_id_2
)
GROUP by community_id_2, id, context_traits_first_name, context_traits_last_name) as community_id_table
ON community_id_table.id = app_opened.user_id
LEFT JOIN (
SELECT DISTINCT id, name as community_name
FROM ***) as community_table
ON TO_JSON_STRING(community_table.id) = community_id_table.community_id_2
WHERE app_opened.user_id is not null AND
EXTRACT(DAYOFWEEK FROM DATE(timestamp)) = 2 AND
community_table.community_name is not null
GROUP BY community_table.community_name, community_table.id, DATE(timestamp)
Error Message:
I am quite confused on what could be going wrong here, as the error says that timestamp is not grouped, even though I have grouped it at the bottom. I tried including just timestamp rather than Date(timestamp), but that ruins the table data that I am trying to create, where I find the number of users on a single day. Does anyone have any other ideas? My goal is for a single row, get the previous row's data, but because I am grouping by specific metrics, I need to make sure they are ordered by them as well. Thank you so much!
I think you simply need to modify OVER part as:
OVER (PARTITION BY community_table.community_name, community_table.id, DATE(timestamp)) as pre_Value
UPDATE. Seems that the problem was caused by using DATE() function within OVER so it can be solved by using DATE(timestamp) inside of subquery and passing alias to OVER
I am working on a data set which has the following columns :
unique_ID Date
a 2018_09_08
a 2018_09_18
a 2018_09_28
d 2018_09_08
I am looking to select those Unique_IDs which are occurring on all three dates i.e 2018_09_08, 2018_09_18 and 2018_09_28.
My output should be just 'a'.
There is a long solution to this problem - Extract unique_IDs per date and create external table on top of all three of them and then use join on three tables to get unique IDs for all three dates. I believe there should be a better solution as we have just 3 dates in this case which might rise later so I am looking for a more generalized solution.
Here is the query that I have written - select distinct(unique_ID) from table_name where Date = '2018_09_08' and Date = '2018_09_18' and Date = '2018_09_28' which is returning null.
I am also trying to write a sub-query but I doubt HIVE supports such sub queries in this case. Here is what I have written :
select count(distinct(unique_ID)) from (
(select distinct(unique_ID) from table_name where Date = '2018_09_08') a
union all
(select distinct(unique_ID) from table_name where Date = '2018_09_18') b
union all
(select distinct(unique_ID) from table_name where Date = '2018_09_28') c
);
and I am getting following parsing error : FAILED: ParseException line 3:0 missing ) at 'union' near ')' line 4:87 missing EOF at 'b' near ')'
How could we get the Unique_IDs in this case ?
This can be accomplished with group by and having.
select unique_id,count(distinct date)
from tbl
where date in ('2018_09_08','2018_09_18','2018_09_28')
group by id
having count(distinct date) = 3
, v2p AS (
SELECT DISTINCT video_asin, product_asin
from video_table
where video_asin IN (
‘ABC’,
‘CDF’,
‘DEF’,
‘FRW’)
)
, video_data AS (
SELECT distinct
start_date::date,
video_view,
video_asin,
page_asin
FROM video_metrics
WHERE
page_asin IN (select DISTINCT product_asin from v2p)
AND start_date between TO_DATE('01/26/2018','MM-DD-YYYY') and TO_DATE('02/04/2018','MM-DD-YYYY') + 0.9999
)
select sum(video_view) from video_data;
I’m running simple query, but I’m getting strange results with IN and NOT IN .
(1). When I do page_asin IN (select DISTINCT product_asin from v2p), I am getting 1,109,567 as select sum(video_view) from video_data;
(2). When I do the same query but with NOT IN page_asin NOT IN (select DISTINCT product_asin from v2p), I am getting 7,032,405 as sum.
(3). If I remove this whole line (page_asin IN (select DISTINCT product_asin from v2p),) to get all results, I am getting 8,148,803. as sum.
But I supposed that if I add (1) and (2) I should get (3).
But in reality, I am getting: 1,109,567 + 7,032,405 = 8,141,972 AND NOT 8,148,803. from (3).
Why that happens? Why I am missing ~7000 video views?
A NULL value would be neither IN nor NOT IN a list.
So, page_asin must be NULL on some rows.
As a note: The select distinct is redundant in the subquery. There is no reason to include it.
I'm receiving an error in the query below. The Sub-query works as it should; 3 columns, ID, Type, Amount. I'm having an issue with the Pivot Syntax, according to SSMS "near the keyword 'FOR'".
The query is mirroring a question in SO, and I see a ton of other similar questions but I can't seem to find the issue. I've tried using brackets, eliminating the Joins in the sub-query and joining after the data is pivoted but nothing has worked so far.
SELECT
dboBillingAdjustments.CMSId
,DataPivot.Certification
,[DataPivot.Contractor Supplement]
FROM
(
SELECT
Businesses.CMSId
,BillingAdjustments.Type
,BillingAdjustments.ContractorAmount
FROM BillingAdjustments
JOIN BillingAdjustmentTypes on BillingAdjustmentTypes.Id = BillingAdjustments.TypeId
JOIN Businesses on Businesses.UId = BillingAdjustments.BusinessId
WHERE
DateKey = 20171104 AND
IsSettled = 1
) as dboBillingAdjustments
PIVOT
(
BillingAdjustments.ContractorAmount FOR Type in ([Certification], [Contractor Supplement])
) as DataPivot
With PIVOT you need an aggregation function MAX(),MIN(),AVG()...
for example:
PIVOT
(
MAX(BillingAdjustments.ContractorAmount)
FOR Type in ([Certification], [Contractor Supplement])
) as DataPivot
I can't use Dynamic Value bcoz of Error stating
"Lookup Error - SQL Server Database Error: Cannot perform an aggregate function on an expression containing an aggregate or a subquery."
Here is the Scenario :
Query 1
select pr.PRDCT,sum(CASE when pr.DEFINITIONCD='NOP' and pr.PERIOD='D' then pr.PRAMOUNT else 0 END)
as 'NOP D' from PRODUCTWISE_REPORT pr group by pr.PRDCT
Query 2
select DEFINITIONTYPECD from REPORTKPIMAPTXN where DEFINITIONTYPECD='NOP' and REPORTSEQ = (select REPORTSEQ from report_m where REPORTCD='MIS_Product_Wise_Report')
Query 2 returns 'NOP'
so when I put Query 2 in Query 1 for 'NOP', it throws Error
How to resolve this when I've to User Dynamic Query 2 ?
Your second query looks it could be rewritten with a join instead of that subselect. something like this. Of course you are still going to have some issues because your first query has two columns and this has only 1 column. You will have to add another column (can be NULL) to this query before the UNION will actually work.
select r.DEFINITIONTYPECD
from REPORTKPIMAPTXN r
INNER JOIN report_m m on m.REPORTSEQ = r.REPORTSEQ
where DEFINITIONTYPECD = 'NOP'
and r.REPORTCD = 'MIS_Product_Wise_Report'