I have a model, which "merges" data in a table. If the unique key (song_id) already exists, I will update the last_loaded column. If however last_loaded from the sink table is older than 1 day, I will also update the first loaded column, else it should just update last_loaded.
The problem is, with my solution I have to reference the model in itself, which does not work in dbt. Has somebody else maybe got a solution for this?
This is the config for my current model:
{{config(materialized='incremental',unique_key='song_id',merge_update_columns = ['last_loaded', 'first_loaded'],schema = "mart")}}
The original query in redshift looks like this:
WITH fact_intermediate AS(
SELECT
st.song_id,
st.album_id,
st.artist_id,
d1.date_id as first_loaded,
d1.date_id as last_loaded,
st.song_duration_ms
FROM stage.stg_chart_songs st
INNER JOIN mart.dim_dates d1 ON current_date = d1.year || '-' || d1.month || '-' || d1.day)
SELECT
fi.song_id,
fi.artist_id,
fi.album_id,
CASE
WHEN current_date - TO_DATE(fc.last_loaded, 'yyyy-mm-dd') > 1 THEN current_date::varchar
ELSE fc.first_loaded
END AS first_loaded,
CURRENT_DATE AS last_loaded
FROM fact_intermediate fi
left outer join mart.fact_charts fc ON fi.song_id = fc.song_id;
I am absolutely no expert with sql and dbt, has someone maybe got a idea how to solve my issue?
Use the {{ this }} variable which lets you avoid circular references (source).
Related
I am trying to create a table in Dbeaver and I have encountered an issue. I have posted my code below. My code works fine if a well has completion types of both 'Frac' of 'MSFRAC'.However, this well with a uwi= '100042204410W500' has only 'MSFRAC' as completion type. When I run my code, it shows nothing in output (image 2).I wrote a small part of code at the top to check this well's information and you can see the output shows that all well completion types are MSFRAC.
What cause this empty output issue? As you can see here, I am using common expression tables for key words "FRAC","MSFRAC",and"%PERF". So my assumption is if FRAC and %PERF% has no record for this well, even tho MSFRAC has data, it will cause the output to be empty???
Please help!!!! Much Appreciated1
WITH frac as (
SELECT
c.uwi,
case
when AVG(c.base_depth - c.top_depth) =0
then null
else AVG(c.base_depth - c.top_depth)
end AS Avg_Frac_Stage_Spacing,
COUNT(c.completion_type) AS Frac_Stage_Count,
min(c.completion_date) as Min_Completion_Date,
max(c.completion_date) as Max_Completion_Date
FROM
well_completion AS c
WHERE
c.completion_type = 'FRAC'
--or c.completion_type='MSFRAC'
GROUP BY
c.uwi
),
msfrac as (
SELECT
c.uwi,
case
when max(c.base_depth - c.top_depth) >500
then (sum(c.base_depth)-sum(c.top_depth)-max(c.base_depth - c.top_depth))/nullif((count(c.completion_type)-1),0)
else AVG(c.base_depth - c.top_depth)
end AS Avg_Msfrac_Stage_Spacing,
case
when max(c.base_depth - c.top_depth) >500
then count(c.completion_type)-1
else count(c.completion_type)
end AS Msfrac_Stage_Count
FROM
well_completion AS c
WHERE
c.completion_type = 'MSFRAC'
--or c.completion_type='MSFRAC'
GROUP BY
c.uwi
),
jetperf as (
SELECT
c.uwi,
COUNT(c.completion_type) AS Perf_Count,
avg(c.base_depth-c.top_depth ) as Avg_cluster_interval
--Avg(c.x_perf_shots) as SPM
--Avg((c.base_depth - c.top_depth)*(c.x_perf_shots)) as Avg_counts_per_cluster
FROM
well_completion AS c
WHERE
c.completion_type like '%PERF%'
--and (c.base_depth - c.top_depth) <= 2
GROUP BY
c.uwi
)
select
f.uwi,
f.Frac_Stage_Count+m.Msfrac_Stage_Count as Stage_Count,
f.Min_Completion_Date,
f.Max_Completion_Date,
(m.Avg_Msfrac_Stage_Spacing*m.Msfrac_Stage_Count+f.Avg_Frac_Stage_Spacing*f.Frac_Stage_Count)/(m.Msfrac_Stage_Count+f.Frac_Stage_Count) as Stage_Spacing,
--f.completion_strat_unit_id,
round(j.Perf_count/(f.Frac_Stage_Count+m.Msfrac_Stage_Count),2) as Avg_number_of_clusters,
j.Avg_cluster_interval,
j.perf_Count,
--j.SPM,
--j.Avg_counts_per_cluster,
j.Avg_cluster_interval/f.Avg_frac_stage_spacing as completion_multipledividebysingle_ratio,
(f.Avg_Frac_Stage_Spacing+m.Avg_Msfrac_Stage_Spacing)/j.Perf_Count *(f.Frac_Stage_Count+m.Msfrac_Stage_Count) AS Avg_cluster_spacing,
m.Avg_Msfrac_Stage_Spacing,
ihs_ppdm3_own1.well.profile_type as Profile_type,
ihs_ppdm3_own1.well."operator" as Operator,
ihs_ppdm3_own1.well.province_state as Province_state,
ihs_ppdm3_own1.well.x_td_tvd as x_td_tvd,
ihs_ppdm3_own1.well.max_tvd as max_tvd,
ihs_ppdm3_own1.well.td_strat_unit_id as td_strat_unit
FROM
frac as f
inner join
jetperf as j
ON
f.uwi = j.uwi
inner join
ihs_ppdm3_own1.well
ON
ihs_ppdm3_own1.well.uwi=f.uwi
inner join
msfrac as m
ON
m.uwi=f.uwi
where ihs_ppdm3_own1.well.profile_type ='H'and f.uwi ='100042204410W500'
I'm trying to write an Oracle query to join data from 4 different tables. The code is below:
SELECT
PROJ.PRJ_NO, PROJ.PRJ_NAME, PROJ.PRJ_BEG_DATE, PROJ.PRJ_END_DATE, PORT.TIER1_NAME, PORT.TIER2_NAME, PORT.TIER3_NAME, MAX(A.FIS_WK_END_DATE) AS "FISCAL_WEEK", SUM(A.ABDOL) AS "AAB_DOL", SUM(A.VHDOL) AS "AVH_DOL", SUM(A.ADOL) AS "AA_DOL", SUM(A.DCDOL) AS "ADC_DOL", SUM(A.DCGADOL) AS "ADC_GA_DOL", SUM(A.COM) AS "AM_DOL", SUM(A.FE) AS "AFE_DOL", SUM(A.IE) AS "AIE_DOL", SUM(A.OTHER) AS "AR_DOL", SUM(A.MTSFT) AS "AS_FT", SUM(A.MTSST) AS "AS_ST", SUM(A.ACTST) AS "AL_ST", SUM(A.ACTFT) AS "ALL_FT", MAX(P.SNAPSHOT_DATE) as "SNAP_DATE", P.FINSCN_TYPE, SUM(P.ABDOL) AS "PAB_DOL", SUM(P.VHDOL) AS "PVH_DOL", SUM(P.DCDOL) AS "PDC_DOL", SUM(P.TCI_DOL) AS "PCI_GA_DOL", SUM(P.GADOL) AS "PN_GA_DOL", SUM(P.COM) AS "PN_COM", SUM(P.FEE) AS "PN_FEE", SUM(P.D_IE) AS "PN_MOIE", SUM(P.OTHER) AS "PN_OTHER"
FROM PROJ_TASK_VW PROJ
LEFT JOIN PORTFOLIO_VW PORT
ON PROJ.TASKNO = PORT.TASKNO
LEFT JOIN ACTUAL_VW A
ON PROJ.TASKNO = A.CURR_TASKNO
LEFT JOIN BUDG_DOLL_VW P
ON PROJ.TASKNO = P.CURR_TASKNO
WHERE TO_CHAR(PROJ.PRJ_END_DATE, 'YYYY-MM-DD') > '2018-10-01'
AND PROJ.P_FLAG = 'N'
AND (PROJ.P_TYPE LIKE 'D-%' OR PROJ.P_TYPE LIKE '%MR%' OR PROJ.P_TYPE LIKE '%ID%')
AND (SUBSTR(PROJ.PRJ_NO,3,3) != 'BP' AND SUBSTR(PROJ.PRJ_NO,3,3) != 'PJ')
AND (P.FINSCN_TYPE = 'SR' OR P.FINSCN_TYPE = 'BUG')
AND (A.ABDOL + A.VHDOL + A.ADOL + A.DCDOL + A.DCGADOL + A.COM +
A.FE + A.IE + A.OTHER) <> 0
GROUP BY
PROJ.PRJ_NO,
PROJ.PRJ_NAME,
PROJ.PRJ_BEG_DATE,
PROJ.PRJ_END_DATE,
PORT.TIER1_NAME,
PORT.TIER2_NAME,
PORT.TIER3_NAME,
P.FINSCN_TYPE
My overall intent is to bring all of the select fields into a single table using left joins (using table "PROJ" as the parent table and the remaining tables providing child data based on the data returned from the "PROJ" table. When the query is ran it times out after about 30mins. Is there a better way to write this query to where I can build the table I need without timing out???
First, there's no way to answer this question without an execution plan. What columns do you have indexed? But here are some things I noticed.
WHERE TO_CHAR(PROJ.PRJ_END_DATE, 'YYYY-MM-DD') > '2018-10-01'
Your column is a date, so you should be comparing to a date, rather than converting to a VARCHAR2 and doing an inequality on strings.
AND (PROJ.P_TYPE LIKE 'D-%' OR PROJ.P_TYPE LIKE '%MR%' OR PROJ.P_TYPE LIKE '%ID%')
I'm not sure, but these will likely not be very performant because of the wildcards. Indexes might make these better, but I never remember how wildcard searches work with indexes.
AND (SUBSTR(PROJ.PRJ_NO,3,3) != 'BP' AND SUBSTR(PROJ.PRJ_NO,3,3) != 'PJ')
These do nothing since your two SUBSTRs return strings of 3 characters long and you are comparing them to 2 character long strings.
AND (A.ABDOL + A.VHDOL + A.ADOL + A.DCDOL + A.DCGADOL + A.COM + A.FE + A.IE + A.OTHER) <> 0
Do you actually care about the sum here, or are you just checking that one or more of these values is non-zero. If these values are always > 0, then you're better off replacing this with:
AND ( a.ABDOL > 0 OR A.VHDOL > 0 ...
Select m.CEAREGA, m.Crotal, rem.IdRexistro, m.IdMostraExt,
to_char(m.DataEntrega, 'DD/MM/YYYY') As Entrada, e.Descricion AS Ensaio,
to_char(rem.DataValidacion, 'DD/MM/YYYY') as DataValidacion, r.Descricion as Resultado,
to_char(rem.Valor) as Valor, es.Descricion as Especie, tm.Descricion as Mostra,
(select LISTAGG(mo.Descricion, ':::') WITHIN GROUP (order by mo.Descricion)
from motivo_ensaio_mostra mm
join motivo_ensaio mo on mo.CodMotivo=mm.CodMotivo and mm.codLab=mo.codLab
where mm.IdRexistro=rem.IdRexistro and mm.IdMostra=rem.IdMostra and mm.CodLab=rem.CodLab
group by mm.IdRexistro, mm.IdMostra) as Motivo,
(select LISTAGG(trim(remi.NomeDesc || ' ' || remi.PrimeiroApelido || ' ' || remi.SegundoApelido), ':::')
WITHIN GROUP (order by remi.PrimeiroApelido, remi.SegundoApelido, remi.NomeDesc)
from rexistro_remitente rm
join remitente remi on remi.NIFCIF=rm.NIFCIF and rm.codLab=remi.codLab
where rm.IdRexistro=rem.IdRexistro and rm.CodLab=rem.CodLab
group by rm.IdRexistro) as Remitente
from resultado_ensaio_mostra rem
join resultado r on r.CodResult=rem.CodResult and r.codLab = rem.codLab and r.CODTIPORESULT='P'
join mostra m on m.IdRexistro=rem.IdRexistro and m.IdMostra=rem.IdMostra and m.codLab = rem.codLab
and m.VlxBaixa=0 and m.EstadoMostra<>10330004 and LENGTH(m.Crotal<5) and m.IdMostra > 201800000
join especie es on es.CodEspec=m.CodEspec and es.codLab = m.codLab
join tipo_mostra tm on tm.CodTipoMost=m.CodTipoMost and tm.codLab = m.codLab
It shows the Oracle Error: ORA-00907
I can´t find the missing parenthesis or where is the error. Could anyone can help me out?
Thanks in advance.
Changing your formatting style may help.
For sub-queries, I try to keep the open and close brackets visibly associated with each other. I tend to keep them in the same column, and the content of the sub-query indented.
Similarly, make each predicate or calculation clearly separate from each other. I prefer to have them on separate lines, it makes for longer code, but narrower code; which is much more friendly for tools such as diff (and so also git).
This gives me the following which I can visually debug much faster than your example. (In fact, I'd say that I can't visually debug yours, each individual line or expression is too hard to isolate and parse.)
SELECT
m.CEAREGA,
m.Crotal,
rem.IdRexistro,
m.IdMostraExt,
to_char(m.DataEntrega, 'DD/MM/YYYY') As Entrada,
e.Descricion AS Ensaio,
to_char(rem.DataValidacion, 'DD/MM/YYYY') as DataValidacion,
r.Descricion as Resultado,
to_char(rem.Valor) as Valor,
es.Descricion as Especie,
tm.Descricion as Mostra,
(
select
LISTAGG(mo.Descricion, ':::')
WITHIN GROUP (order by mo.Descricion)
from
motivo_ensaio_mostra mm
join
motivo_ensaio mo
on mo.CodMotivo=mm.CodMotivo
and mm.codLab=mo.codLab
where
mm.IdRexistro=rem.IdRexistro
and mm.IdMostra=rem.IdMostra
and mm.CodLab=rem.CodLab
group by
mm.IdRexistro, mm.IdMostra
) as Motivo,
(
select
LISTAGG(trim(remi.NomeDesc || ' ' || remi.PrimeiroApelido || ' ' || remi.SegundoApelido), ':::')
WITHIN GROUP (order by remi.PrimeiroApelido, remi.SegundoApelido, remi.NomeDesc)
from
rexistro_remitente rm
join
remitente remi
on remi.NIFCIF=rm.NIFCIF
and rm.codLab=remi.codLab
where
rm.IdRexistro=rem.IdRexistro
and rm.CodLab=rem.CodLab
group by
rm.IdRexistro
) as Remitente
from
resultado_ensaio_mostra rem
join
resultado r
on r.CodResult = rem.CodResult
and r.codLab = rem.codLab
and r.CODTIPORESULT='P'
join
mostra m
on m.IdRexistro = rem.IdRexistro
and m.IdMostra = rem.IdMostra
and m.codLab = rem.codLab
and m.VlxBaixa=0
and m.EstadoMostra<>10330004
and LENGTH(m.Crotal<5)
and m.IdMostra > 201800000
join
especie es
on es.CodEspec=m.CodEspec
and es.codLab = m.codLab
join
tipo_mostra tm
on tm.CodTipoMost=m.CodTipoMost
and tm.codLab = m.codLab
This leads me to the conclusion that the brackets are not the problem. So, it's likely to be some other syntax error near a bracket.
As per an answer that came up while I was reformatting your code, it appears to be LENGTH(m.Crotal<5) which should be LENGTH(m.Crotal) < 5?
(In essence, there is a ) missing before the <, and also an extra one present after the 5...)
What an ugly code; try to get used to format it, it is easier to maintain it that way.
Anyway: error comes from AND LENGTH (m.Crotal < 5) line.
Problem:
A Database collumn has a Tristate (0,1,2).
Each of the values are used serversidely.
The Clientcode (which cant be changed anymore) is only able to understand '0,1'.
In the Clients view '1' is identic with '2'. So I want to change the SQL Query in the Database to return '1', if the specific value is > 0.
My current Solution is combining 2 Selects (using UNION SELECT) with different WHERE-Clauses and returning '1' or '0' as static values. Now I'm looking for a solution to 'translate' the value within only ONE SELECT statement.
This is my current Solution:
SELECT
dbo.Nachricht.NachrichtID, dbo.Nachricht.Bezeichnung, '1' AS BetrifftKontoeinrichtung,
FROM dbo.Nachricht INNER JOIN dbo.AdditionalData
ON dbo.Nachricht.NachrichtID = dbo.AdditionalData.NachrichtID
WHERE (dbo.Nachricht.NachrichtID in ( 450,439 ))
AND dbo.AdditionalData.BetrifftKontoeinrichtung > 0
UNION SELECT
dbo.Nachricht.NachrichtID, dbo.Nachricht.Bezeichnung, '0' AS BetrifftKontoeinrichtung,
FROM dbo.Nachricht INNER JOIN dbo.AdditionalData
ON dbo.Nachricht.NachrichtID = dbo.AdditionalData.NachrichtID
WHERE (dbo.Nachricht.NachrichtID in ( 450,439 ))
AND dbo.AdditionalData.BetrifftKontoeinrichtung = 0
You can use a case statement, like this:
SELECT
dbo.Nachricht.NachrichtID, dbo.Nachricht.Bezeichnung,
CASE WHEN dbo.AdditionalData.BetrifftKontoeinrichtung = 0
THEN '0' ELSE '1'
END AS BetrifftKontoeinrichtung,
FROM dbo.Nachricht
INNER JOIN dbo.AdditionalData
ON dbo.Nachricht.NachrichtID = dbo.AdditionalData.NachrichtID
WHERE (dbo.Nachricht.NachrichtID in ( 450,439 ))
Looks like you need to use CASE. A decent tutorial here
http://www.databasejournal.com/features/mssql/article.php/3288921/T-SQL-Programming-Part-5---Using-the-CASE-Function.htm
See the worked example
If you just CAST(CAST(val AS BIT) AS INT) you will get integer 0 for 0 and integer 1 for everything else.
I’m trying to run this SQL using get external.
It works, but when I try to rename the sub-queries or anything for that matter it remove it.
I tried as, as and the name in '', as then the name in "",
and the same with space. What is the right way to do that?
Relevant SQL:
SELECT list_name, app_name,
(SELECT fname + ' ' + lname
FROM dbo.d_agent_define map
WHERE map.agent_id = tac.agent_id) as agent_login,
input, CONVERT(varchar,DATEADD(ss,TAC_BEG_tstamp,'01/01/1970'))
FROM dbo.maps_report_list list
JOIN dbo.report_tac_agent tac ON (tac.list_id = list.list_id)
WHERE input = 'SYS_ERR'
AND app_name = 'CHARLOTT'
AND convert(VARCHAR,DATEADD(ss,day_tstamp,'01/01/1970'),101) = '09/10/2008'
AND list_name LIKE 'NRBAD%'
ORDER BY agent_login,CONVERT(VARCHAR,DATEADD(ss,TAC_BEG_tstamp,'01/01/1970'))
You could get rid of your dbo.d_agent_define subquery and just add in a join to the agent define table.
Would this code work?
select list_name, app_name,
map.fname + ' ' + map.lname as agent_login,
input,
convert(varchar,dateadd(ss,TAC_BEG_tstamp,'01/01/1970')) as tac_seconds
from dbo.maps_report_list list
join dbo.report_tac_agent tac
on (tac.list_id = list.list_id)
join dbo.d_agent_define map
on (map.agent_id = tac.agent_id)
where input = 'SYS_ERR'
and app_name = 'CHARLOTT'
and convert(varchar,dateadd(ss,day_tstamp,'01/01/1970'),101) = '09/10/2008'
and list_name LIKE 'NRBAD%'
order by agent_login,convert(varchar,dateadd(ss,TAC_BEG_tstamp,'01/01/1970'))
Note that I named your dateadd column because it did not have a name. I also tried to keep your convention of how you do a join. There are a few things that I would do different with this query to make it more readable, but I only focused on getting rid of the subquery problem.
I did not do this, but I would recommend that you qualify all of your columns with the table from which you are getting them.
To remove the sub query in the SELECT statement I suggest the following:
SELECT list_name, app_name, map.fname + ' ' + map.lname as agent_login, input, convert(varchar,dateadd(ss, TAC_BEG_tstamp, '01/01/1970))
FROM dbo.maps_report_list inner join
(dbo.report_tac_agent as tac inner join dbo.d_agent_define as map ON (tac.agent_id=map.agent_id)) ON list.list_id = tac.list_id
WHERE input = 'SYS_ERR' and app_name = 'CHARLOTT' and convert(varchar,dateadd(ss,day_tstamp,'01/01/1970'),101) = '09/10/2008'
and list_name LIKE 'NRBAD%' order by agent_login,convert(varchar,dateadd(ss,TAC_BEG_tstamp,'01/01/1970'))
I used parentheses to create the inner join between dbo.report_tac_agent and dbo.d_agent_define first. This is now a set of join data.
The combination of those tables are then joined to your list table, which I am assuming is the driving table here. If I am understand what you are trying to do with your sub select, this should work for you.
As stated by the other poster you should use table names on your columns (e.g. map.fname), it just makes things easy to understand. I didn't in my example because I am note 100% sure which columns go with which tables. Please let me know if this doesn't do it for you and how the data it returns is wrong. That will make it easier to solve in needed.