Trying to get better at understand sql queries - sql
I'm a new developer in a corporate IT group. I feel completely lost looking at alot of the sql that's written in our code. There are many queries that are literally hundreds of lines long. Here's an example of a "smaller" one (the table/field names have been modified, but the structure is the same). Do I just suck at this or is it really difficult to understand? How would you approach rewriting it to be more manageable/understandable?
SELECT pd.commission_header_id COMMISSION_HEADER_ID
,pd.sales_rep_id DIRECT_SALESREP_ID
,pd.org_id ORG_ID
,LEAST(100, innview.spilt_percent) SPLIT_PCT
,pd.source SOURCE
,pd.application APPLICATION
,pd.plan PLANE
,pd.pay_freq pay_freq
,pd.emp_app_count EMP_APP_COUNT
,pd.client_name CLIENT_NAME
,pd.fed_id FED_ID
,pd.off_nbr OFF_NBR
,pd.payroll_num payroll_num
,TO_DATE(pd.product_start_date,'DD-MON-YY') PRODUCT_START_DATE
,pd.loss_date LOSS_DATE
,pd.acquisitions_ind ACQUISITIONS_IND
,pd.source_of_business SOURCE_OF_BUSINESS
,pd.prev_br_clt_nbr PREV_BR_CLT_NBR
,pd.previous_method PREVIOUS_METHOD
,pd.natl_acct_number NATL_ACCT_NUMBER
,pd.product_name PRODUCT_NAME
,pd.lost_reason LOST_REASON
,pd.client_id CLIENT_ID
,pd.lead_nbr LEAD_NBR
,pd.processed_date PROCESSED_DATE
,pd.rep_type REP_TYPE
,pd.order_number ORDER_NUMBER
,pd.conversion_type CONVERSION_TYPE
,SUBSTR(original_billing_info,1,INSTR(original_billing_info,'|',1,1)-1) BILLING_APPLICATION
,SUBSTR(original_billing_info,INSTR(original_billing_info,'|',1,1)+1,INSTR(original_billing_info,'|',1,2)-INSTR(original_billing_info,'|',1,1)-1) BILLING_CODE
,SUBSTR(original_billing_info,INSTR(original_billing_info,'|',1,2)+1,INSTR(original_billing_info,'|',1,3)-INSTR(original_billing_info,'|',1,2)-1) BILLING_PROD_GRP_NM
,SUBSTR(original_billing_info,INSTR(original_billing_info,'|',1,3)+1) BILLING_FIELD_FOR_CHRGS
,SUBSTR(original_discount_info,1,INSTR(original_discount_info,'|',1,1)-1) DISCOUNT_APPLICATION
,SUBSTR(original_discount_info,INSTR(original_discount_info,'|',1,1)+1,INSTR(original_discount_info,'|',1,2)-INSTR(original_discount_info,'|',1,1)-1) DISCOUNT_CODE
,CASE WHEN (NVL(innview.trans,0)) = 0
THEN 0
ELSE ( NVL(innview.comm,0)) / ( NVL(innview.trans,0))
END ORIGINAL_COMM_PCT
,pd.record_type RECORD_TYPE
,innview.trans PAR_SUM
,NVL(ocp.oic_clients_pk, pxmis.oic_client_products_pk_seq.nextval) CLIENT_PRODUCT_ID
,pd.associated_with ASSOCIATED_WITH_ID
,pd.refferal_sale refferal_sale
,pd.parent_client_nbr PARENT_CLIENT_NBR
FROM pxmis.pd_commissions pd
JOIN pxmis.oic_lookback_months olm ON (pd.application = olm.application
AND pd.plan = olm.plan)
LEFT OUTER JOIN pxmis.oic_client_products ocp ON (pd.off_nbr = ocp.off_nbr
AND pd.payroll_num = ocp.payroll_num
AND pd.application = ocp.application
AND pd.plan = ocp.plan
AND pd.sales_rep_id = ocp.direct_salesrep_id
AND pd.source = ocp.source
AND ocp.record_type in (:1,:2)
AND NVL(pd.refferal_sale,'NON REF') = NVL(ocp.refferal_sale,'NON REF'))
JOIN (SELECT SUM(NVL(transaction_amount,0)) as trans
,SUM(commission_amount) as comm
,SUM(pd2.oic_split_percent) as spilt_percent
,pd2.application as application
,pd2.plan as code
,pd2.sales_rep_id as sales_rep_id
,pd2.payroll_num as client_nbr
,LPAD(pd2.off_nbr,4,0) as off_nbr
FROM pxmis.pd_commissions pd2
JOIN pxmis.oic_lookback_months olm2 ON (pd2.application = olm2.application
AND pd2.plan = olm2.plan)
WHERE CASE WHEN (pd2.record_type IN ('SAPBA','ASAPBA'))
THEN pd2.SUB_RECORD_TYPE
ELSE 'Eligible'
END = 'Eligible'
AND pd2.record_type IN ('SDBDA', 'ASDBDA',
decode(:3,'CHCST','SAPBA',NULL),
decode(:4,'CHCST','ASAPBA',NULL),
'SAPBP','ASAPBP')
AND pd2.processed_date BETWEEN ADD_MONTHS(:5,(olm2.lookback_months_high - 1) * -1) AND ADD_MONTHS(:6, (olm2.lookback_months_low - 2) * -1) - 1
and :7 between olm2.effective_start_date and olm2.effective_end_date
and olm2.function_type = :8
GROUP BY pd2.application
,pd2.plan
,pd2.sales_rep_id
,pd2.payroll_num
,LPAD(pd2.off_nbr,4,0)) innview on (innview.application = pd.application
AND innview.code = pd.plan
AND innview.sales_rep_id = pd.sales_rep_id
AND innview.client_nbr = pd.payroll_num
AND innview.off_nbr = LPAD(pd.off_nbr,4,0))
WHERE :9 BETWEEN olm.effective_start_date AND olm.effective_end_date
AND pd.sales_rep_id <> -3
AND pd.record_type IN ('SAPBP','ASAPBP')
AND NOT EXISTS(SELECT 'X'
FROM pxmis.pd_commissions pd3
WHERE pd3.application = pd.application
AND pd3.plan = pd.plan
AND pd3.sales_rep_id = pd.sales_rep_id
AND pd3.payroll_num = pd.payroll_num
AND (pd3.record_type in ('CHCST','ACHCST')
or ( pd3.record_type = 'SAPBA'
and pd3.sub_record_type = 'Chargeback')))
AND pd.commission_header_id = (SELECT MAX(pd4.commission_header_id)
FROM pxmis.pd_commissions pd4
WHERE pd4.application = pd.application
AND pd4.plan = pd.plan
AND pd4.sales_rep_id = pd.sales_rep_id
AND pd4.payroll_num = pd.payroll_num
AND pd4.record_type = pd.record_type)
AND olm.function_type = :10
AND pd.processed_date BETWEEN ADD_MONTHS(:11,(olm.lookback_months_high - 1) * -1) AND ADD_MONTHS(:12, (olm.lookback_months_low - 2) * -1) - 1
AND pd.source = :13
AND pay_freq IS NOT NULL
AND pd.core_conversion_revenue IS NULL;
Sidenote: (Due to my reputation I can not make comments:)
This is my first time answering. Hopefully it will help.
I do not know if it might help, but most of the SQL script (maybe all of it), can be rewritten in other languages. For example you can use JOINS in C# .NET LINQ.
But a way I usually separate the SQL script, is to look for the keywords and see what they do. Do not confuse the big letters from the small ones.
The SELECT function selects all of the datas.
The TO_DATE function says that it will convert the a string to date (source: http://www.techonthenet.com/oracle/functions/to_date.php)
The LEAST function returns the smallest number between the variables you have as parameters (Source: http://www.techonthenet.com/oracle/functions/least.php)
You can do this for every function, and try and grasp what the script does.
I think you could end the script here: FROM pxmis.pd_commissions pd
Also I have seen that you do a JOIN.
Here you could properly make a new script and run that code.
Right now I might not be able to make it more readable. (Mostly because I work with .NET C# linqs).
Again hopefully it helped in some way.
Related
Inserting Replace Function into Query
I found this solution which I'm pretty sure will do exactly what I'm trying to accomplish, unfortunately, all my SQL is self-taught and I can't figure out how to integrate the solution into the query I'm working with.
I tried putting everything above the SELECT statement, but I couldn't set the #testString variable to the COND_NOTE.TEXT field, which makes sense, since all of that happens inside the main query select.
Apologies for essentially asking someone to give me a crash course in presumably a pretty basic application of SQL syntax.
SELECT
CASE WHEN DM_IAM_D_I_ROOT.ZZCAP_FACILITY = 'N200' THEN 'MT'
WHEN DM_IAM_D_I_ROOT.ZZCAP_FACILITY = 'N202' THEN 'PI'
WHEN DM_IAM_D_I_ROOT.ZZCAP_FACILITY = 'N204' THEN 'FL'
ELSE 'Err' END AS FAC
,right(DM_IAM_D_I_ROOT.ISSUE_ID,12) AS ISS_ID
--The following line is the one that I want to use the solution from the referenced question in.
,REPLACE(REPLACE(COND_NOTE.TEXT, '"', '"'), 'br/>', '') NOTES
,COND_NOTE.LINE_COUNTER
FROM
DM_IAM_D_I_ROOT
LEFT JOIN
(
SELECT
DM_BOBF_D_TXCROOT.HOST_KEY
,DM_BOBF_D_TXCROOT.MANDT
,DM_BOBF_D_TXCTXT.TEXT_TYPE
,DM_BOBF_D_TXCCON.LINE_COUNTER
,DM_BOBF_D_TXCCON.TEXT
FROM
DM_BOBF_D_TXCROOT
INNER JOIN DM_BOBF_D_TXCTXT
ON DM_BOBF_D_TXCROOT.DB_KEY = DM_BOBF_D_TXCTXT.PARENT_KEY AND DM_BOBF_D_TXCROOT.MANDT = DM_BOBF_D_TXCTXT.MANDT
INNER JOIN DM_BOBF_D_TXCCON
ON DM_BOBF_D_TXCTXT.DB_KEY = DM_BOBF_D_TXCCON.PARENT_KEY AND DM_BOBF_D_TXCTXT.MANDT = DM_BOBF_D_TXCCON.MANDT
WHERE
DM_BOBF_D_TXCROOT.MANDT = '100'
AND DM_BOBF_D_TXCTXT.TEXT_TYPE = 'ZCOND'
) COND_NOTE ON DM_IAM_D_I_ROOT.DB_KEY = COND_NOTE.HOST_KEY AND DM_IAM_D_I_ROOT.MANDT = COND_NOTE.MANDT
WHERE
DM_IAM_D_I_ROOT.ISSUE_TYPE = 'CAP'
AND DM_IAM_D_I_ROOT.MANDT = '100'
AND DM_IAM_D_I_ROOT.LCYCLE_CD NOT IN ('10', '64')
ORDER BY ZZCAP_FACILITY, ISSUE_ID, LINE_COUNTER
How to dynamic, handle nested WHERE AND/OR queries using Rails and SQL
I'm currently building a feature that requires me to loop over an hash, and for each key in the hash, dynamically modify an SQL query.
The actual SQL query should look something like this:
select * from space_dates d
inner join space_prices p on p.space_date_id = d.id
where d.space_id = ?
and d.date between ? and ?
and (
(p.price_type = 'monthly' and p.price_cents <> 9360) or
(p.price_type = 'daily' and p.price_cents <> 66198) or
(p.price_type = 'hourly' and p.price_cents <> 66198) # This part should be added in dynamically
)
The last and query is to be added dynamically, as you can see, I basically need only one of the conditions to be true but not all.
query = space.dates
.joins(:price)
.where('date between ? and ?', start_date, end_date)
# We are looping over the rails enum (hash) and getting the key for each key value pair, alongside the index
SpacePrice.price_types.each_with_index do |(price_type, _), index|
amount_cents = space.send("#{price_type}_price").price_cents
query = if index.positive? # It's not the first item so we want to chain it as an 'OR'
query.or(
space.dates
.joins(:price)
.where('space_prices.price_type = ?', price_type)
.where('space_prices.price_cents <> ?', amount_cents)
)
else
query # It's the first item, chain it as an and
.where('space_prices.price_type = ?', price_type)
.where('space_prices.price_cents <> ?', amount_cents)
end
end
The output of this in rails is:
SELECT "space_dates".* FROM "space_dates"
INNER JOIN "space_prices" ON "space_prices"."space_date_id" = "space_dates"."id"
WHERE "space_dates"."space_id" = $1 AND (
(
(date between '2020-06-11' and '2020-06-11') AND
(space_prices.price_type = 'hourly') AND (space_prices.price_cents <> 9360) OR
(space_prices.price_type = 'daily') AND (space_prices.price_cents <> 66198)) OR
(space_prices.price_type = 'monthly') AND (space_prices.price_cents <> 5500)
) LIMIT $2
Which isn't as expected. I need to wrap the last few lines in another set of round brackets in order to produce the same output. I'm not sure how to go about this using ActiveRecord.
It's not possible for me to use find_by_sql since this would be dynamically generated SQL too.
So, I managed to solve this in about an hour using Arel with rails
dt = SpaceDate.arel_table
pt = SpacePrice.arel_table
combined_clauses = SpacePrice.price_types.map do |price_type, _|
amount_cents = space.send("#{price_type}_price").price_cents
pt[:price_type]
.eq(price_type)
.and(pt[:price_cents].not_eq(amount_cents))
end.reduce(&:or)
space.dates
.joins(:price)
.where(dt[:date].between(start_date..end_date).and(combined_clauses))
end
And the SQL output is:
SELECT "space_dates".* FROM "space_dates"
INNER JOIN "space_prices" ON "space_prices"."space_date_id" = "space_dates"."id"
WHERE "space_dates"."space_id" = $1
AND "space_dates"."date" BETWEEN '2020-06-11' AND '2020-06-15'
AND (
("space_prices"."price_type" = 'hourly'
AND "space_prices"."price_cents" != 9360
OR "space_prices"."price_type" = 'daily'
AND "space_prices"."price_cents" != 66198)
OR "space_prices"."price_type" = 'monthly'
AND "space_prices"."price_cents" != 5500
) LIMIT $2
What I ended up doing was:
Creating an array of clauses based on the enum key and the price_cents
Reduced the clauses and joined them using or
Added this to the main query with an and operator and the combined_clauses
Case Statement that runs sql
I am applying a mask to data and believe the best way is to use a Case Statement. However, I need the case statement to run a sub query. When I pull data, it will either be a number or appear as 99999999999v999b:99999999999v999- Using TO_NUMBER(REGEXP_REPLACE(RD.subm_quantity, '^(\d+)(-)?$', '\2\1'))/1000 as "Submitted_Quantity" This will convert it to a number. So if 00000000100000 is present, it will convert to 100 However, I need a case to not divide when not needed. To determine if I need to divide, I need to add a rule in the below sql: if the result is 99999999999v999b:99999999999v999-, apply the conversion; if not, just output RD.subm_quantity. How can I get a case statement to run a query? Running in TOAD for Oracle: select m.mask FROM Valiuser.ivd_mapping m, Valiuser.ivd_mappingset s, Valiuser.ivd_mapping_record r, Valiuser.ivd_transaction_file tf, VALIUSER.ivd_transaction_record_details RD WHERE s.mappingset_id = r.mappingset_id AND r.mapping_record_id = m.mapping_record_ID AND m.repository_column_id = '34' AND s.mappingset_id = tf.MAPPINGSET_ID AND rd.file_id = tf.file_id AND rd.TRANSACTION_RECORD_ID =
If the mask and original subm_quantity are both available from the query you showed, which seems to be the same as that includes the rd table you're referencing in the conversion, then I think you want something like this: case when m.mask = '99999999999v999b:99999999999v999-' then TO_NUMBER(REGEXP_REPLACE(rd.subm_quantity, '^(\d+)(-)?$', '\2\1')) / 1000 else rd.subm_quantity end as "Submitted_Quantity" rather than a subquery. So plugged into your current query that would make it: SELECT case when m.mask = '99999999999v999b:99999999999v999-' then TO_NUMBER(REGEXP_REPLACE(rd.subm_quantity, '^(\d+)(-)?$', '\2\1')) / 1000 else rd.subm_quantity end as "Submitted_Quantity" FROM Valiuser.ivd_mapping m, Valiuser.ivd_mappingset s, Valiuser.ivd_mapping_record r, Valiuser.ivd_transaction_file tf, Valiuser.ivd_transaction_record_details rd WHERE s.mappingset_id = r.mappingset_id AND r.mapping_record_id = m.mapping_record_ID AND m.repository_column_id = '34' AND s.mappingset_id = tf.mappingset_id AND rd.file_id = tf.file_id AND rd.Transaction_Record_Id = <?> or with modern join syntax instead of the old version, something like: SELECT case when m.mask = '99999999999v999b:99999999999v999-' then TO_NUMBER(REGEXP_REPLACE(rd.subm_quantity, '^(\d+)(-)?$', '\2\1')) / 1000 else rd.subm_quantity end as "Submitted_Quantity" FROM Valiuser.ivd_mapping m JOIN Valiuser.ivd_mapping_record r ON r.mapping_record_id = m.mapping_record_ID JOIN Valiuser.ivd_mappingset s ON s.mappingset_id = r.mappingset_id JOIN Valiuser.ivd_transaction_file tf ON tf.mappingset_id = s.mappingset_id JOIN Valiuser.ivd_transaction_record_details rd ON rd.file_id = tf.file_id WHERE m.repository_column_id = '34' AND rd.transaction_record_id = <?>
Oracle sql issue with returning rows that have null values on table joins
I am working on a view (not originally created by myself) and what I am trying to do is return requests (table alias req) that dont have hours allocated to them (table alias te) and i am kinda struggling to get any kind of outer join to return this on the following sql: SELECT te.tc_date_yyyy_ww, te.tc_date_yyyy_mm, te.tc_date_yyyy, te.dc_date, te.nc_request_id_fk, te.nc_subrequest_id_fk, te.tc_clock, te.nc_hours, te.nc_amount, te.nc_discipline_id_fk, te.tc_discipline_desc, req.tc_request_name, req.mc_project_cost, req.nt_fetr_project, req.nc_bus_unit_id_fk, req.nc_location_id_fk, req.nc_cemt_status_id_fk, coord.tc_user_fname || ' ' || coord.tc_user_lname AS projectcoodname, assoc.tc_fname || ' ' || assoc.tc_lname AS empname, assoc_disc.tc_long_desc AS user_discipline, sr.tc_name AS child_project, sr.tc_cost_center_or_wbs, sr.tc_cost_center_nbr, sr.tc_cost_center_desc, sr.tc_profit_center_desc, sr.tc_profit_center_nbr, sr.tc_wbs_nbr, sr.tc_wbs_desc, sr.nc_child_type_id_fk, sre.nc_estimate, stat_li.tc_med_desc AS req_status, NVL(PP.NC_PRIORITY,0) AS PRIORITY FROM te07fear.tbye10_time_entry te, te07fear.tbxg100_requests req, (SELECT * FROM te07fear.tbye05_form_users WHERE tc_role_id_fk = 'project_coordinator') coord, te07fear.tbye07_subrequest sr, te07fear.tbye08_subrequest_est sre, te07fear.tbye02_list_items stat_li, te07fear.tbye04_associates assoc, te07fear.tbxg95_dropdowns assoc_disc, TE07FEAR.TBYF43_PROJECT_PRIORITY PP WHERE te.tc_clock = assoc.tc_clock AND te.nc_subrequest_id_fk = sr.nc_subrequest_id(+) AND te.nc_subrequest_id_fk = sre.nc_subrequest_id_fk(+) AND te.tc_clock = sre.tc_clock(+) AND te.nc_request_id_fk = req.nc_request_id AND te.nc_request_id_fk = coord.nc_form_id_fk AND assoc.nc_discipline_id_fk = assoc_disc.nc_key_id AND req.nc_cemt_status_id_fk = stat_li.nc_value_id AND TE.NC_REQUEST_ID_FK = PP.NC_REQUEST_ID_FK AND TE.NC_SUBREQUEST_ID_FK = PP.NC_SUB_REQUEST_ID_FK AND TE.TC_DATE_YYYY_WW = PP.TC_DATE_YYYY_WW AND TE.TC_CLOCK = PP.TC_CLOCK_NUMBER Any help or ideas would be greatly appreciated.I am assuming that a left outer join should enable me to to this. Thanks JC
As far as I understand, you actually need NOT EXISTS clause, not outer join. Here's a simplified snippet, to show you what I mean (skipped most of the query, but you'l get the idea): SELECT req.tc_request_name, req.mc_project_cost, req.nt_fetr_project, req.nc_bus_unit_id_fk, req.nc_location_id_fk, req.nc_cemt_status_id_fk FROM te07fear.tbxg100_requests req WHERE NOT EXISTS (select 1 from te07fear.tbye10_time_entry te where te.nc_request_id_fk = req.nc_request_id) ; Let me know if that's the case and if it serves the purpose. Cheers
How to perform multiple AND/OR meta_query using using $wpdb
SCENARIO: I have a custom post type with start and end dates saved as post meta. This is to allow the admin to schedule dates to show AND hide a post. There is also the option to leave these dates blank, in which case the post just shows by default. As meta_query doesn't appear to allow you to perform an AND/OR comparison, I've dabbled with creating a custom SQL query using $wpdb. SQL is new territory for me. Currently, this works for the start date:
$today = date('Y-m-d');
$results = $wpdb->get_results($wpdb->prepare("
SELECT
$wpdb->posts.post_title,
$wpdb->posts.ID,
$wpdb->posts.post_content
FROM
$wpdb->posts,
$wpdb->postmeta
WHERE
$wpdb->posts.ID = $wpdb->postmeta.post_id
AND $wpdb->posts.post_type = 'announcements'
AND ($wpdb->postmeta.meta_key = 'sap_start_date'
AND ($wpdb->postmeta.meta_value <= '$today' OR $wpdb->postmeta.meta_value = ''))
"));
However, when I try and amend the above to query the end date, I get nothing back. I tried changing the WHERE section to:
WHERE
$wpdb->posts.ID = $wpdb->postmeta.post_id
AND $wpdb->posts.post_type = 'announcements'
AND ($wpdb->postmeta.meta_key = 'sap_start_date'
AND ($wpdb->postmeta.meta_value <= '$today' OR $wpdb->postmeta.meta_value = ''))
AND ($wpdb->postmeta.meta_key = 'sap_end_date'
AND ($wpdb->postmeta.meta_value >= '$today' OR $wpdb->postmeta.meta_value = ''))
This returns an empty array.
QUESTION: How would I write this query to check both the sap_start_date and sap_end_date post meta values?
I figured out my problem was that I was using a logical AND against the same meta key object twice; how can postmeta.meta_key be equal to start_date and end_date? Using an answer from this SO question I managed to re-write the query to use aliases so that I could get the postmeta twice and query the two different keys: SELECT $wpdb->posts.post_title, $wpdb->posts.ID, $wpdb->posts.post_content FROM $wpdb->posts, $wpdb->postmeta AS postmeta1, $wpdb->postmeta AS postmeta2 WHERE $wpdb->posts.ID = postmeta1.post_id AND $wpdb->posts.ID = postmeta2.post_id AND $wpdb->posts.post_type = 'announcements' AND (postmeta1.meta_key = 'sap_start_date' AND (postmeta1.meta_value <= '$today' OR postmeta1.meta_value = '')) AND (postmeta2.meta_key = 'sap_end_date' AND (postmeta2.meta_value >= '$today' OR postmeta2.meta_value = '')) This works, although I'm still a little fuzzy on just how it works. If anyone could shed some further light on this purely for educational purposes it would be much appreciated.