I have been trying to write some SQL queries that joins 5 tables and selects a subset of the columns to create a new table. I am trying to convert the SQL query into an AWS Glue job, but I am running into a No Viable Alternative at Input error.
I am attaching my query below:
CREATE VIEW cust_ref as
select
------^^^ This is where the error appears.
user_acc_id_int as user_acc_id,
user_acc_skey_int as user_acc_skey,
dg_cust_ref_no
from dim_customer dc where crnt_indc = 1,
address_ref as (select id as address_id, zip from dim_address da ),
zip_ref as (select zip_id, zip from dim_zip)
SELECT
transaction_id,
order_id,
dc.user_acc_id as customer_id,
quantity,
redeem_order_id,
pre_tax_amount,
total_amount,
date(created_at) as sales_date,
user_order_skey,
batch_id,
created_by,
created_dttm,
user_order_skey_int,
nvl(dpart.partner_id,-1) as partner_id,
nvl(dprod.prod_id, -1) as prod_id,
nvl(tstat.trn_status_id, -1) as trn_status_id,
nvl(rtstat.trn_status_id, -1) as redeem_order_status_id,
nvl(ttype.trn_type_id, -1) as trn_type_id
from fact_user_order_stg fuos
--customer lookup--
left join cust_ref dsc on fuos.dg_customer_ref_no = dsc.dg_cust_ref_no
left join cust_ref dsc on fuos.src_dg_customer_ref_no = dsc.dg_cust_ref_no
left join cust_ref ddc on fuos.dst_dg_customer_ref_no = ddc.dg_cust_ref_no
--product lookup
left join (select prod_id , prod_sku from cem_product_staging cps where crnt_indc = 1) dprod
on fuos.product_sku = dprod.prod_sku
--partner lookup--
left join dim_partner dpart on upper(substring(order_id,1,3)) = upper(dpart.partner_prefix)
--zip lookup 2 step--
left join address_ref ba on fuos.billing_address_id = ba.address_id
--left join zip_ref bz on ba.zip = bz.zip
left join address_ref sa on fuos.shipping_address_id = sa.address_id
--left join zip_ref sz on sa.zip = sz.zip
--transaction type, status lookups to follow
left join dim_trn_status tstat on upper(fuos.status) = upper(tstat.trn_status)
left join dim_trn_status rtstat on upper(fuos.redeem_order_status) = upper(tstat.trn_status)
left join dim_trn_type ttype on upper(fuos.\"type\") = upper(ttype.trn_type);
Any guidance would be helpful.
I HAVE TRIED ANSWERS FROM RELATED POSTS which included using "" to fix case sensitive syntax, using a view instead of a WITH clause etc but none of them fixed the problem.
Related
I have the following SQL select statement that I use to get a subset of products, or wines:
SELECT pv.SkProdVariantId AS id,
pa.Colour AS colour,
FROM Dim.ProductVariant AS pv
JOIN ProductAttributes_new AS pa
ON pv.SkProdVariantId = pa.SkProdVariantId
WHERE pv.ProdTypeName = 'Wines'
The length of this table generated is 3,905. I want to get all the transactional data for these products.
At the moment I'm using this select statement
SELECT c.CalDate AS timestamp,
f.SkProductVariantId AS sku_id,
f.Quantity AS quantity
FROM fact.FTransactions AS f
LEFT JOIN Dim.Calendar AS c
ON f.SkDateId = c.SkDateId
LEFT JOIN (
SELECT pv.SkProdVariantId AS id,
pa.Colour AS colour,
FROM Dim.ProductVariant AS pv
JOIN ProductAttributes_new AS pa
ON pv.SkProdVariantId = pa.SkProdVariantId
WHERE pv.ProdTypeName = 'Wines'
) AS s
ON s.id = f.SkProductVariantId
WHERE c.CalDate LIKE '%2019%'
The calendar dates are correct, but the number of unique products returned is 5,648, rather than the expected 3,905 from the select query.
Why does my LEFT JOIN on the first select query not work as I expect it to, please?
Thanks for any help!
If you want all the rows form your query, it needs to be the first reference in the LEFT JOIN. Then, I am guessing that you want transaction in 2019:
select . . .
from (SELECT pv.SkProdVariantId AS id, pa.Colour AS colour,
FROM Dim.ProductVariant pv JOIN
ProductAttributes_new pa
ON pv.SkProdVariantId = pa.SkProdVariantId
WHERE pv.ProdTypeName = 'Wines'
) s LEFT JOIN
(fact.FTransactions f JOIN
Dim.Calendar c
ON f.SkDateId = c.SkDateId AND
c.CalDate >= '2019-01-01' AND
c.CalDate < '2020-01-01'
)
ON s.id = f.SkProductVariantId;
Note that this assumes that CalDate is really a date and not a string. LIKE should only be used on strings.
You misunderstand somehow how outer joins work. See Gordon's answer and my request comment on that.
As to the task: It seems you want to select transactions of 2019, but you want to restrict your results to wine products. We typically restrict query results in the WHERE clause. You can use IN or EXISTS for that.
SELECT
c.CalDate AS timestamp,
f.SkProductVariantId AS sku_id,
f.Quantity AS quantity
FROM fact.FTransactions AS f
INNER JOIN Dim.Calendar AS c ON f.SkDateId = c.SkDateId
WHERE DATEPART(YEAR, c.CalDate) = 2019
AND f.SkProductVariantId IN
(
SELECT pv.SkProdVariantId
FROM Dim.ProductVariant AS pv
WHERE pv.ProdTypeName = 'Wines'
);
(I've removed the join to ProductAttributes_new, because it doesn't seem to play any part in this query.)
EDIT:
The result supposed to be like this:
desired result
I have this query:
SELECT DISTINCT mitarbeiter.mitarbnr, mitarbeiter.login, mitarbeiter.name1, mitarbeiter.name2
FROM vertragspos
left join vertrag_ek_vk_zuord ON vertragspos.id = vertrag_ek_vk_zuord.ek_vertragspos_id
left join mitarbeiter ON vertrag_ek_vk_zuord.anlage_mitarbnr = mitarbeiter.mitarbnr
left join vertragskopf ON vertragskopf.id = vertragspos.vertrag_id
left join
(
SELECT wkurse.*, fremdwaehrung.wsymbol
FROM wkurse
INNER join
(
SELECT lfdnr, Max(tag) AS maxTag
FROM wkurse
WHERE tag < SYSDATE
GROUP BY lfdnr
) t1
ON wkurse.lfdnr = t1.lfdnr AND wkurse.Tag = t1.maxTag
INNER JOIN fremdwaehrung ON wkurse.lfdnr = fremdwaehrung.lfdnr
) wkurse ON vertragskopf.blfdwaehrung = wkurse.lfdnr
left join
(
SELECT vertrag_ID, Sum (preis) preis, Sum (menge) menge, Sum (preis * menge / Decode (vertragskopf.zahlintervall, 1,1,2,2,3,3,4,6,5,12,1) / wkurse.kurs) vertragswert
FROM vertragspos
GROUP BY vertrag_ID
) s ON vertragskopf.id = s.vertrag_id
But I always get an error on line 21 Pos 145:
ORA-00904 WKURSE.KURS invalid identifier
The WKURSE table is supposed be joined already above, but why do I still get error?
How can I do join with all these tables?
I need to join all these tables:
Mitarbeiter, Vertragspos, vertrag_ek_vk_zuord, wkurse, fremdwaehrung, vertragskopf.
What is the right syntax? I'm using SQL Tool 1,8 b38
Thank you.
Because LEFT JOIN is executed on entire dataset, and not in row-by-row manner. So there's no wkurse.kurs available in the execution context of subquery. Since you join that tables, you can place the calculation in the top-most select statement.
EDIT:
After you edited the statement, it became clear where does vertragskopf.zahlintervall came from. But I don't know where are you going to use calculated vertragswert (now it is absent in the query), so I've put it in the result. As I'm not a SQL parser and have no idea of your tables, so I cannot check the code, but calculation now can be resolved (all the values are available in calculation context).
SELECT DISTINCT mitarbeiter.mitarbnr, mitarbeiter.login, mitarbeiter.name1, mitarbeiter.name2, s.amount / Decode (vertragskopf.zahlintervall, 1,1,2,2,3,3,4,6,5,12,1) / wkurse.kurs) vertragswert
FROM vertragspos
left join vertrag_ek_vk_zuord ON vertragspos.id = vertrag_ek_vk_zuord.ek_vertragspos_id
left join mitarbeiter ON vertrag_ek_vk_zuord.anlage_mitarbnr = mitarbeiter.mitarbnr
left join vertragskopf ON vertragskopf.id = vertragspos.vertrag_id
left join (
SELECT wkurse.*, fremdwaehrung.wsymbol
FROM wkurse
INNER join (
SELECT lfdnr, Max(tag) AS maxTag
FROM wkurse
WHERE tag < SYSDATE
GROUP BY lfdnr
) t1
ON wkurse.lfdnr = t1.lfdnr AND wkurse.Tag = t1.maxTag
INNER JOIN fremdwaehrung ON wkurse.lfdnr = fremdwaehrung.lfdnr
) wkurse ON vertragskopf.blfdwaehrung = wkurse.lfdnr
left join (
SELECT vertrag_ID, Sum (preis) preis, Sum (menge) menge, Sum (preis * menge) as amount
FROM vertragspos
GROUP BY vertrag_ID
) s ON vertragskopf.id = s.vertrag_id
Rewriting the code using WITH clause makes it much clearer than select from select.
Also get the rate on last day before today in oracle is as simple as
select wkurse.lfdnr
, max(wkurse.kurs) keep (dense_rank first order by wkurse.tag desc) as rate
from wkurse
where tag < sysdate
group by wkurse.lfdnr
One option is a lateral join:
left join lateral
(SELECT vertrag_ID, Sum(preis) as preis, Sum(menge) as menge,
Sum (preis * menge / Decode (vertragskopf.zahlintervall, 1,1,2,2,3,3,4,6,5,12,1) / wkurse.kurs) vertragswert
FROM vertragspos
GROUP BY vertrag_ID
) s
ON vertragskopf.id = s.vertrag_id
I am working on SQL query which should return the list of Managers and the staff who reports to them.
Unfortunately there is no separate table for Employee or Staff but a single 'resource' table called ahsresources.
The managers are identified with a relation called 'C0'.
Even after trying various Joins, I am unable to extract the list. The idea is that a manager will run the report to see his reportees, as well as those staff who report to his own reportees
Example -
Now, if lets say HDY is running the query, then its should return him the below result
Below is the query I have created, but for the matter of understanding the issue, you can use the above example.
select a.description as manager1,a.rel_value as MGID,a.resource_id as Reportee1_MGR2,r.name,a.date_to as date, r.date_to,a1.resource_id as MG3ID,r1.name as Rep3Name,
a2.resource_id as MG4ID,r2.name as Rep4Name
from ahsrelvalue a
LEFT OUTER JOIN ahsresources r
ON r.resource_id = a.resource_id and r.client = a.client and a.date_to='12/31/2099'
LEFT OUTER JOIN ahsrelvalue a1
ON a1.rel_Value = a.resource_id and a1.client = a.client and a1.date_to = '12/31/2099'
LEFT OUTER JOIN ahsrelvalue a2
ON a2.rel_Value = a1.resource_id and a2.client = a1.client and a2.date_to = '12/31/2099'
LEFT OUTER JOIN ahsresources r1
ON r1.resource_id = a1.resource_id and r1.client = a1.client and a1.date_to='12/31/2099'
LEFT OUTER JOIN ahsresources r2
ON r2.resource_id = a2.resource_id and r2.client = a2.client and a2.date_to='12/31/2099'
where a.rel_Value = '$?resid' and a.rel_attr_id='C0' and r.date_to = '12/31/2099' and r1.date_to ='12/31/2099'
and r.status !='C' and r1.status!='C' and r2.status!='C'
In SQL Server, you can use a recursive query to traverse this hierarchical dataset:
with cte as (
select t.* from mytable where managerID = 6
union all
select t.*
from cte c
inner join mytable t on t.managerID = c.staffID
)
select * from cte
I have a database with 9 tables and I need to create a suitable view that looks at all tables.
The query I have for the left hand side of the ERD Diagram is working ok for the County, Country, CustomerType and Associations. I also have the code for Accounts working (I think), however when I start to add in the AccountType table, it doesn't recognize the AccountTypeID in the Accounts table.
So now I have two Queries that work but I need to join the two queries in order to see all the data to create the view. Query 1 had all the data (using multiple joins) for all the tables that interacts with the Customer Table and Query 2 has all the data (using multiple joins) for all the tables that interact with the Accounts Table.
Query for left hand side:
SELECT
Cus.CustomerId,
Cus.FirstName,
Cus.LastName,
Cty.CountyName,
Cry.CountryName,
Cut.CustomerType,
Ass.AssociationType,
Ass.CustomerId1,
Ass.CustomerId2,
Acc.AccountId
FROM
Customers Cus
LEFT JOIN
(
SELECT
Cty.CountyId,
Cty.CountyName
FROM
County Cty
WHERE
Cty.IsDeleted = 0)
AS
Cty
ON
Cus.CountyId = Cty.CountyId
LEFT JOIN
(
SELECT
Cry.CountryId,
Cry.CountryName
FROM
Country Cry
WHERE
Cry.IsDeleted = 0)
AS
Cry
ON
Cus.CountryId = Cry.CountryId
LEFT JOIN
(
SELECT
Cut.CustomerTypeId,
Cut.CustomerType
FROM
CustomerTypes Cut
WHERE
Cut.IsDeleted = 0)
AS
Cut
ON
Cus.CustomerTypeId = Cut.CustomerTypeId
LEFT JOIN
(
SELECT
Ass.AssociationId,
Ass.AssociationType,
Ass.CustomerId1,
Ass.CustomerId2
FROM
Associations Ass
WHERE
Ass.IsDeleted = 0)
AS
Ass
ON
Cus.AssociationId = Ass.AssociationId
RIGHT JOIN
(
SELECT
Acc.AccountId,
Acc.CustomerId
FROM
Accounts Acc
WHERE
Acc.IsDeleted = 0)
AS
Acc
ON
Cus.CustomerId = Acc.CustomerId
WHERE
Cus.IsDeleted = 0
QUERY for right hand side.
SELECT
Acc.AccountId,
Acc.CustomerId,
Act.AccountType,
Tra.TransactionType,
Sec.SecurityType
FROM
Accounts Acc
LEFT JOIN
(
SELECT
Act.AccountTypeId,
Act.AccountType
FROM
AccountType Act
WHERE
Act.IsDeleted = 0)
AS
Act
ON
Acc.AccountTypeId = Act.AccountTypeId
LEFT JOIN
(
SELECT
TransactionId,
TransactionType
FROM
Transactions Tra
WHERE
Tra.IsDeleted = 0)
AS
Tra
ON
Acc.TransactionId = Tra.TransactionId
RIGHT JOIN
(
SELECT
SecurityId,
SecurityType
FROM
Securities Sec
WHERE
Sec.IsDeleted = 0)
AS
Sec
ON
Acc.SecurityId = Sec.SecurityId
WHERE
Acc.IsDeleted = 0
Any ideas? Thanks Anthony.
-- WITH POD was causing the issue, removing this code reduced 2 year pull to 3 mins.
Will post new question to figure out best way to include POD data.
--Edit for clarity, I am a read only user to these tables.
I wrote the below query, but it takes a very long time to execute (20min).
It is currently limited to 1 month, but user wants at least 1 year preferably 2. I assume this would scale time to hours.
Can anyone take a look at let me know if there is a BKM I am not using to improve performance?
Or if there is a better method for a report of this size? At 2 years, it would return ~100K rows from 17 tables.
WITH
POD AS
(
SELECT SHIPMENTS.Delivery
,SHIPMENTS.Shipment_Number
,PROOF_OF_DELIVERY.Shipping_Carrier
,PROOF_OF_DELIVERY.Tracking_Number
,PROOF_OF_DELIVERY.Ship_Method
,PROOF_OF_DELIVERY.POD_Signature
,PROOF_OF_DELIVERY.POD_Date
,PROOF_OF_DELIVERY.POD_Time
FROM
SHIPMENTS
LEFT JOIN PROOF_OF_DELIVERY
ON SHIPMENTS.Shipment_Number = PROOF_OF_DELIVERY.Delivery_Or_Shipment
WHERE Load_Date IN
(
SELECT MAX(Load_Date)
FROM PROOF_OF_DELIVERY
GROUP BY Delivery_Or_Shipment
)
)
SELECT DISTINCT GI.GOODS_ISSUE_DOCUMENT_ID
,GI.SALES_ORDER_ID
,GI.SALES_ORDER_LINE_ID
,GI.SALES_ORDER_TYPE_CODE
,GI.DELIVERY_HEADER_ID
,GI.DELIVERY_ITEM_ID
,FD.FISCAL_MONTH_CODE
,GI.MATERIAL_NUMBER
,GI.SHIPPED_QTY
,SO.ORDERER_NAME
,SO.CREATED_BY
,SO.CONTACT_PERSON
,GI.SOLD_TO_CUSTOMER_ID
,GI.SHIP_TO_CUSTOMER_ID
,GI.ORIGINAL_COMMIT_DATE
,GI.SHIP_FROM_PLANT_ID
,GI.ACTUAL_PGI_DATE
,GI.CUSTOMER_PO_NUMBER
,GI.SHIPPED_PRICE
,(GI.SHIPPED_PRICE * GI.SHIPPED_QTY) AS EXT_SHIPPED_PRICE
,GI.SALES_ORGANIZATION_CODE
,GI.DELIVERY_NOTE_PRIORITY_CODE
,FD.FISCAL_WEEK_CODE
,DV.DIVISION_CODE
,DN.Delivery_Item_Creation_Date
,SOLD.CUSTOMER_SHORT_NAME AS SOLD_TO_CUSTOMER_SHORT_NAME
,SHIP.CUSTOMER_SHORT_NAME AS SHIP_TO_CUSTOMER_SHORT_NAME
,SHIP.Customer_Site_Name
,SHIP.REGION_NAME
,MATD.MATERIAL_DESCRIPTION
,MATD.STANDARD_COST
,(MATD.STANDARD_COST * GI.SHIPPED_QTY) AS EXT_STANDARD_COST
,MATD.GLOBAL_EVENT
,PLT.LEAD_TIME_FOR_ORIGINAL_COMMIT
,OPRM.BASE_PART_CODE
,MATD.PRODUCT_INSP_MEMO
,MATD.MATERIAL_PRICING_GROUP_CODE
,MATD.MATERIAL_STATUS AS MMPP
,PIM.PIM_PBG_GROUPING
,SOL.SHIPPING_CONDITION
,SVO.SERVICE_ORDER_NUM
,SO.CREATION_TIME AS SO_CREATION_TIME
,SOL.CREATED_TIME AS SO_LINE_CREATED_TIME
,SOL.SHIPPING_POINT
,SDT.SALES_DOCUMENT_TYPE_CODE AS SVO_DOCUMENT_TYPE_CODE
,EQU.EQUIPMENT_NUM
,EQU.SERIAL_NUMBER
,EQU.CUSTOMERTOOLID
,POD.Shipment_Number
,POD.Shipping_Carrier
,POD.Tracking_Number
,POD.Ship_Method
,POD.POD_Signature
,POD.POD_Date
,POD.POD_Time
,DATEDIFF(dd,SO.CREATION_TIME,GI.ACTUAL_PGI_DATE) AS Cycle_Time_to_PGI_Days
,DATEDIFF(hh,SO.CREATION_TIME,GI.ACTUAL_PGI_DATE) AS Cycle_Time_to_PGI_Hours
FROM GOODS_ISSUE AS GI
INNER JOIN dbo.Delivery_Notes AS DN
ON GI.DELIVERY_HEADER_ID = DN.DELIVERY_HEADER_CODE AND GI.DELIVERY_ITEM_ID = DN.DELIVERY_ITEM_CODE
INNER JOIN dbo.Customer_View AS SOLD
ON GI.SOLD_TO_CUSTOMER_ID = SOLD.CUSTOMER_CODE
INNER JOIN dbo.Customer_View AS SHIP
ON GI.SOLD_TO_CUSTOMER_ID = SHIP.CUSTOMER_CODE
INNER JOIN dbo.MATERIAL_DETAILS AS MATD
ON GI.MATERIAL_NUMBER = MATD.MATERIAL_NUMBER
INNER JOIN dbo.OPR_MATERIAL_DIM AS OPRM
ON OPRM.MATERIAL_NUMBER = GI.MATERIAL_NUMBER
LEFT JOIN dbo.SM_DATE_DIM AS FD
ON CAST(FD.CALENDAR_DAY AS DATE) = CAST(GI.ACTUAL_PGI_DATE AS DATE)
LEFT JOIN dbo.DIM_PUBLISHED_LEAD_TIME_COMMIT AS PLT
ON PLT.MATERIAL_NUMBER = OPRM.BASE_PART_CODE
LEFT JOIN dbo.PRODUCT_INSP_MEMO_DIM AS PIM
ON PIM.PRODUCT_INSP_MEMO = MATD.PRODUCT_INSP_MEMO
INNER JOIN dbo.SM_SALES_ORDER_LINE_FACT AS SOL
ON SOL.SALES_ORDER_CODE = GI.SALES_ORDER_ID AND SOL.SALES_ORDER_LINE_CODE = GI.SALES_ORDER_LINE_ID
INNER JOIN dbo.SM_SALES_ORDER_FACT AS SO
ON SO.SALES_ORDER_CODE = GI.SALES_ORDER_ID
INNER JOIN dbo.SM_DIVISION_DIM AS DV
ON SO.DIVISION_SID = DV.DIVISION_SID
LEFT JOIN dbo.SERVICE_ORDER_FACT AS SVO
ON SVO.SERVICE_ORDER_NUM = SO.SERVICE_ORDER_NUMBER
LEFT JOIN dbo.SM_SALES_DOCUMENT_TYPE_DIM AS SDT
ON SDT.SALES_DOCUMENT_TYPE_SID = SVO.SALES_DOCUMENT_TYPE_SID
LEFT JOIN dbo.SM_EQUIPMENT_DIM AS EQU
ON EQU.EQUIPMENT_SID = SVO.EQUIPMENT_SID
LEFT JOIN POD
ON POD.Delivery = GI.DELIVERY_HEADER_ID
WHERE GI.ACTUAL_PGI_DATE > GETDATE()-32
AND SOLD_TO_CUSTOMER_ID IN (0010000252,0010000898,0010001121,0010001409,0010001842,0010001852,0010001879,0010001977,0010001978,0010002021,0010002202,0010002227,0010002982,0010003118,0010003176,0010003294,0010005492,0010006904,0010007048,0010007080,0010010381,0010010572,0010010905,0010011999,0010012014,0010012048,0010012571,0010013124,0010013711,0010013713,0010013824,0010014180,0010014188,0010014333,0010015059,0010015313,0010015414,0010015541,0010015544,0010015550)
A CTE is just syntax
I suspect that CTE is evaluated many times
Materialze the CTE to #temp with indexe(s) so it is run once
This cast will hurt it
Make those columns true dates and index them
ON CAST(FD.CALENDAR_DAY AS DATE) = CAST(GI.ACTUAL_PGI_DATE AS DATE)
That where negates the left so you can just do a join
Also that MAX(Load_Date) could match on another shipment
SELECT SHIPMENTS.Delivery
,SHIPMENTS.Shipment_Number
,PROOF_OF_DELIVERY.Shipping_Carrier
,PROOF_OF_DELIVERY.Tracking_Number
,PROOF_OF_DELIVERY.Ship_Method
,PROOF_OF_DELIVERY.POD_Signature
,PROOF_OF_DELIVERY.POD_Date
,PROOF_OF_DELIVERY.POD_Time
FROM SHIPMENTS
JOIN PROOF_OF_DELIVERY
ON SHIPMENTS.Shipment_Number = PROOF_OF_DELIVERY.Delivery_Or_Shipment
WHERE PROOF_OF_DELIVERY.Load_Date IN
(
SELECT MAX(Load_Date)
FROM PROOF_OF_DELIVERY
GROUP BY Delivery_Or_Shipment
)
Pull this up into the join
INNER JOIN dbo.Customer_View AS SOLD
ON GI.SOLD_TO_CUSTOMER_ID = SOLD.CUSTOMER_CODE
AND GI.SOLD_TO_CUSTOMER_ID IN (0010000252,0010000898,0010001121,0010001409,0010001842,0010001852,0010001879,0010001977,0010001978,0010002021,0010002202,0010002227,0010002982,0010003118,0010003176,0010003294,0010005492,0010006904,0010007048,0010007080,0010010381,0010010572,0010010905,0010011999,0010012014,0010012048,0010012571,0010013124,0010013711,0010013713,0010013824,0010014180,0010014188,0010014333,0010015059,0010015313,0010015414,0010015541,0010015544,0010015550)