How can this SQL query be improved? - sql

This code actually works, but I guess it can be done in an easier way, or even more efficient (because I guess comparing strings like that is not a problem but may be). I tried to convert everything into datetime and not string but failed.
This code takes the date and time row from one table, CONCATing them and compares it to the datetime row from another table. The result is: 2019-12-02 09:00:00
It is just a regular Date, Time and Datetime parameters in the table. Like this Date 2019-11-17, Time 09:00:00, Datetime 2019-01-15 16:00:00
SELECT
MAX(CONCAT(f_ini, CONCAT( " ", h_ini)))
FROM posible_work
WHERE
fk_id_asigned = 100573 AND
(SELECT MAX(CONCAT(f_ini, CONCAT( " ", h_ini)))
FROM posible_work) >
(SELECT MAX(RIGHT(f_fin,19)) AS FechaAVI FROM next_work WHERE fk_id_worker = 100573)

Apart from the fact that you force concat() to be called twice when you can use it with 3 arguments (or more), one additional problem would be this condition:
(SELECT MAX(CONCAT(f_ini, CONCAT( " ", h_ini)))
FROM posible_work) >
(SELECT MAX(RIGHT(f_fin,19)) AS FechaAVI FROM next_work WHERE fk_id_worker = 100573)
Why not use in the left side of the inequality just concat(...) and you repeat SELECT...MAX(..)...?
I don't think that this breaks your code's logic:
SELECT
MAX(CONCAT(f_ini, ' ', h_ini))
FROM posible_work
WHERE
fk_id_asigned = 100573
AND
CONCAT(f_ini, ' ', h_ini) >
(SELECT MAX(RIGHT(f_fin,19)) AS FechaAVI FROM next_work WHERE fk_id_worker = 100573)
Also it could be better if you did not hardcode 100573 twice.
Just use aliases properly:
SELECT
MAX(CONCAT(p.f_ini, ' ', p.h_ini))
FROM posible_work p
WHERE
p.fk_id_asigned = 100573
AND
CONCAT(p.f_ini, ' ', p.h_ini) >
(SELECT MAX(RIGHT(f_fin,19)) AS FechaAVI FROM next_work WHERE fk_id_worker = p.fk_id_asigned)

Related

SQL 2-table query results need to be concatenated

My current SQL:
SELECT B.MESSAGENO, B.LINENO, B.LINEDATA
FROM BILL.MESSAGE AS B, BILL.ACTIVITYAS A
WHERE A.MSGNO = D.MESSAGENO AND A.FUPTEAM = 'DBWB'
AND A.ACTIVITY = 'STOPPAY' AND A.STATUS = 'WAIT'
AND A.COMPANY = D.COMPANY
MESSAGENO LINENO LINEDATA
1234567 1 CHEQUE NO : 9999999 RUN NO : 55555
1234567 2 DATE ISSUED: 12/25/2020 AMOUNT : 710.51
1234567 3 PAYEE : LASTNAME, FIRSTNAME
1234567 4 ACCOUNT NO : 12345-67890
1234567 5 USER : USERNAME
there are 550 sets of 5 lines per MESSAGENO
What I am trying to figure out is how I can get something like where LINENO = 1, concatenate LINEDATA so I just get 9999999 as checkno, where LINENO = 2, concatenate LINEDATA so I get 710.51 as amount, where LINENO = 3, concatenate LINEDATA so I get LASTNAME, FIRSTNAME as payee, where LINENO = 4, concatenate LINEDATA so I get LASTNAME, FIRSTNAME as payee, and lastly, the same thing for USERNAME.
I just cannot seems to conceptualize this. Every time I try, my brain starts turning into macaroni. Any help is appreciated.
UPDATED ANSWER, extracts all fields from stored strings:
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=e22d26866053ea6088aa78dc23c4d809
Check this fiddle.
It uses a SUBSTRING_INDEX in the internal query to split the fields at the : or by a combination of : and " ". I used the two spaces because I wasn't sure what your actual whitespace was, and when I copied the data out of your post it was just spaces.
Then MAX is used in the outer query to get everything on one line, grouping by the messageNo. Since some lines have two pieces of data to extract, a second string parser was added. Here is the code from the fiddle, for reference. In this case, the table test was created from the output of OP's initial query, since I didn't have enough info to build both tables and completely recreate his output.
SELECT MESSAGENO,
MAX(if(LINENO = 1, extractFirst, null)) as checkNo,
MAX(if(LINENO = 1, extractLast, null)) as runNo,
MAX(if(LINENO = 2, extractFirst, null)) as issued,
MAX(if(LINENO = 2, extractLast, null)) as amount,
MAX(if(LINENO = 3, extractFirst, null)) as payee,
MAX(if(LINENO = 4, extractFirst, null)) as accountNo,
MAX(if(LINENO = 5, extractFirst, null)) as username
FROM (
SELECT MESSAGENO, LINENO,
trim(substring_index(substring_index(LINEDATA, ": ", -2), " ", 1)) as extractFirst,
trim(substring_index(LINEDATA, ":", -1)) as extractLast
FROM test
) z
GROUP BY MESSAGENO
Again, you will be much better off to alter your tables so that you can use simpler queries, as shown in the last fiddle.
===============================================
ORIGINAL ANSWER (demo of string parsing, suggestion for data model change)
You can achieve this with some string parsing BUT ABSOLUTELY DO NOT unless you have no choice. The reason you are having trouble is because your data shouldn't be stored this way.
I've included a fiddle incorporating this case statement and substring_index to extract the data. I have assumed mySQL 8 because you didn't specify SQL version.
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=068a49a2c819c08018691e54bcdf91e5
case when LINENO = 1 then trim(substring_index(substring_index(LINEDATA, "RUN NO", 1), ":", -1))
else trim(substring_index(LINEDATA, ":", -1)) end
as LDATA
See this fiddle for the full statement. I have just inserted the data from your join into a test table, instead of trying to recreate all your tables, since I don't have access to all the data I would need for that. In future, set up a fiddle like this one with some representative data and the SQL version, and it will be easier for people to help you.
=========================================
I think this is a better layout for you, with all data stored as the proper type and a field defined for each one and the extra text stripped out:
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=11d52189b005740cdc53175466374635

OPENJSON - modify statement to ignore first part of the string

We receive auto-generated emails from an application, and we export those to our database as they arrive at the Inbox. The table is called dbo.MailArchive.
Up until recently, the body of the email has always looked like this...
Status: Completed
Successful actions count: 250
Page load count: 250
...except with different numbers and statuses. Note that there is a carriage return on the blank line after Page load count.
The entirety of this data gets written to a field called Mail_Body - then we run the following statement using OPENJSON to parse those lines into their own columns in the record:
DECLARE #PI varchar(7) = '%[^' + CHAR(13) + CHAR(10) + ']%';
SELECT j.Status,
j.Successful_Actions_Count,
j.Page_Load_Count
FROM dbo.MailArchive m
CROSS APPLY(VALUES(REVERSE(m.Mail_Body),PATINDEX(#PI,REVERSE(m.Mail_Body)))) PI(SY,I)
CROSS APPLY(VALUES(REVERSE(STUFF(PI.SY,1,PI.I,''))))S(FixedString)
CROSS APPLY OPENJSON (CONCAT('{"', REPLACE(REPLACE(S.FixedString, ': ', '":"'), CHAR(13) + CHAR(10), '","'), '"}'))
WITH (Status varchar(100) '$.Status',
Successful_Actions_Count int '$."Successful actions count"',
Page_Load_Count int '$."Page load count"') j;
Beginning today, there are certain emails where the body of the email looks like this:
Agent did not meet defined success criteria on this run.
Status: Completed
Successful actions count: 250
Page load count: 250
To clarify, that's one new line at the top, a carriage return at the end of that line, and a carriage return on the blank line between the new line and the Status line. At this time, there is no consistent way to predict which emails will come in with this new line, and which ones won't.
How can I modify our OPENJSON statement to say, If this first line exists in the body, skip/ignore it and parse lines 3 through 5, else just do exactly what I have above? Or perhaps even better to future-proof it, always ignore everything before the word Status?
Since your data has new leading and trailing rows, I think a simple aggregation in concert with a string_split() and a CROSS APPLY would be more effective than my previous XML answer and the current JSON approach
Example or dbFiddle
Select A.ID
,Status = stuff(Pos1,1,charindex(':',Pos1),'')
,Action = try_convert(int,stuff(Pos2,1,charindex(':',Pos2),''))
,PageCnt = try_convert(int,stuff(Pos3,1,charindex(':',Pos3),''))
From YourTable A
Cross Apply (
Select [Pos1] = max(case when Value like 'Status:%' then value end)
,[Pos2] = max(case when Value like '%actions count:%' then value end)
,[Pos3] = max(case when Value like 'Page load count:%' then value end)
From string_split(SomeCol,char(10))
) B
Returns
ID Status Action PageCnt
1 Completed 250 250
Note: Use an OUTER APPLY if you want to see NULLs

Oracle Query is timing out

I'm trying to write an Oracle query to join data from 4 different tables. The code is below:
SELECT
PROJ.PRJ_NO, PROJ.PRJ_NAME, PROJ.PRJ_BEG_DATE, PROJ.PRJ_END_DATE, PORT.TIER1_NAME, PORT.TIER2_NAME, PORT.TIER3_NAME, MAX(A.FIS_WK_END_DATE) AS "FISCAL_WEEK", SUM(A.ABDOL) AS "AAB_DOL", SUM(A.VHDOL) AS "AVH_DOL", SUM(A.ADOL) AS "AA_DOL", SUM(A.DCDOL) AS "ADC_DOL", SUM(A.DCGADOL) AS "ADC_GA_DOL", SUM(A.COM) AS "AM_DOL", SUM(A.FE) AS "AFE_DOL", SUM(A.IE) AS "AIE_DOL", SUM(A.OTHER) AS "AR_DOL", SUM(A.MTSFT) AS "AS_FT", SUM(A.MTSST) AS "AS_ST", SUM(A.ACTST) AS "AL_ST", SUM(A.ACTFT) AS "ALL_FT", MAX(P.SNAPSHOT_DATE) as "SNAP_DATE", P.FINSCN_TYPE, SUM(P.ABDOL) AS "PAB_DOL", SUM(P.VHDOL) AS "PVH_DOL", SUM(P.DCDOL) AS "PDC_DOL", SUM(P.TCI_DOL) AS "PCI_GA_DOL", SUM(P.GADOL) AS "PN_GA_DOL", SUM(P.COM) AS "PN_COM", SUM(P.FEE) AS "PN_FEE", SUM(P.D_IE) AS "PN_MOIE", SUM(P.OTHER) AS "PN_OTHER"
FROM PROJ_TASK_VW PROJ
LEFT JOIN PORTFOLIO_VW PORT
ON PROJ.TASKNO = PORT.TASKNO
LEFT JOIN ACTUAL_VW A
ON PROJ.TASKNO = A.CURR_TASKNO
LEFT JOIN BUDG_DOLL_VW P
ON PROJ.TASKNO = P.CURR_TASKNO
WHERE TO_CHAR(PROJ.PRJ_END_DATE, 'YYYY-MM-DD') > '2018-10-01'
AND PROJ.P_FLAG = 'N'
AND (PROJ.P_TYPE LIKE 'D-%' OR PROJ.P_TYPE LIKE '%MR%' OR PROJ.P_TYPE LIKE '%ID%')
AND (SUBSTR(PROJ.PRJ_NO,3,3) != 'BP' AND SUBSTR(PROJ.PRJ_NO,3,3) != 'PJ')
AND (P.FINSCN_TYPE = 'SR' OR P.FINSCN_TYPE = 'BUG')
AND (A.ABDOL + A.VHDOL + A.ADOL + A.DCDOL + A.DCGADOL + A.COM +
A.FE + A.IE + A.OTHER) <> 0
GROUP BY
PROJ.PRJ_NO,
PROJ.PRJ_NAME,
PROJ.PRJ_BEG_DATE,
PROJ.PRJ_END_DATE,
PORT.TIER1_NAME,
PORT.TIER2_NAME,
PORT.TIER3_NAME,
P.FINSCN_TYPE
My overall intent is to bring all of the select fields into a single table using left joins (using table "PROJ" as the parent table and the remaining tables providing child data based on the data returned from the "PROJ" table. When the query is ran it times out after about 30mins. Is there a better way to write this query to where I can build the table I need without timing out???
First, there's no way to answer this question without an execution plan. What columns do you have indexed? But here are some things I noticed.
WHERE TO_CHAR(PROJ.PRJ_END_DATE, 'YYYY-MM-DD') > '2018-10-01'
Your column is a date, so you should be comparing to a date, rather than converting to a VARCHAR2 and doing an inequality on strings.
AND (PROJ.P_TYPE LIKE 'D-%' OR PROJ.P_TYPE LIKE '%MR%' OR PROJ.P_TYPE LIKE '%ID%')
I'm not sure, but these will likely not be very performant because of the wildcards. Indexes might make these better, but I never remember how wildcard searches work with indexes.
AND (SUBSTR(PROJ.PRJ_NO,3,3) != 'BP' AND SUBSTR(PROJ.PRJ_NO,3,3) != 'PJ')
These do nothing since your two SUBSTRs return strings of 3 characters long and you are comparing them to 2 character long strings.
AND (A.ABDOL + A.VHDOL + A.ADOL + A.DCDOL + A.DCGADOL + A.COM + A.FE + A.IE + A.OTHER) <> 0
Do you actually care about the sum here, or are you just checking that one or more of these values is non-zero. If these values are always > 0, then you're better off replacing this with:
AND ( a.ABDOL > 0 OR A.VHDOL > 0 ...

LINQ to SQL selecting records and converting dates

I'm trying to select records from a table based on a date using Linq to SQL. Unfortunately the date is split across two tables - the Hours table has the day and the related JobTime table has the month and year in two columns.
I have the following query:
Dim qry = From h As Hour In ctx.Hours Where Convert.ToDateTime(h.day & "/" & h.JobTime.month & "/" & h.JobTime.year & " 00:00:00") > Convert.ToDateTime("01/01/2012 00:00:00")
This gives me the error "Arithmetic overflow error converting expression to data type datetime."
Looking at the SQL query in SQL server profiler, I see:
exec sp_executesql N'SELECT [t0].[JobTimeID], [t0].[day], [t0].[hours]
FROM [dbo].[tbl_pm_hours] AS [t0]
INNER JOIN [dbo].[tbl_pm_jobtimes] AS [t1] ON [t1].[JobTimeID] = [t0].[JobTimeID]
WHERE (CONVERT(DateTime,(((((CONVERT(NVarChar,[t0].[day])) + #p0) + (CONVERT(NVarChar,COALESCE([t1].[month],NULL)))) + #p1) + (CONVERT(NVarChar,COALESCE([t1].[year],NULL)))) + #p2)) > #p3',N'#p0 nvarchar(4000),#p1 nvarchar(4000),#p2 nvarchar(4000),#p3 datetime',#p0=N'/',#p1=N'/',#p2=N' 00:00:00',#p3='2012-01-31 00:00:00'
I can see that it's not passing in the date to search for correctly but I'm not sure how to correct it.
Can anyone please help?
Thanks,
Emma
The direct cause of the error may have to do with this issue.
As said there, the conversions you use are a very inefficient way to build a query. On top of that, it is inefficient because the expressions are not sargable. I.e. you are using a computed value from database columns in a comparison which disables the query analyzer to use indexes to jump to individual column values. So, you could try to fix the error by doctoring the direct cause, but I think it's better to rewrite the query in a way that only the single column values are used in comparions.
I've worked this out in C#:
var cfg = new DateTime(12,6,12);
int year = 12, month = 6, day = 13; // Try some more values here.
// Date from components > datetime value?
bool gt = (
year > cfg.Year || (
(year == cfg.Year && month > cfg.Month) || (
year == cfg.Year && month == cfg.Month && day > cfg.Day)
)
);
You see that it's not as straightforward as it may look at first, but it works. There are much more comparisons to work out, but I'm sure that the ability to use indexes will easily outweigh this.
A more straightforward, but not sargable, way is to use sortable dates, like 20120101 and compare those (as integers).

Using iif clause in Ms Access for a column with "Yes/No" Data Type

I have a column called DayShift in a table which is of Yes/No data type (Boolean). The result Output I want is: if the value is true, display "Day" else display night.
I have tried the following:
SELECT iif(DayShift=Yes,"Day","Night") as Shift FROM table1;
SELECT iif(DayShift,"Day","Night") as Shift FROM table1;
SELECT iif(DayShift=True,"Day","Night") as Shift FROM table1;
SELECT iif(DayShift=1,"Day","Night") as Shift FROM table1;
But none of the above work. It just gives me a list of blank check boxes in the output datasheet window. I am using Ms Access 2003. Any help appreciated.
Update:
After a bit of research that the yes/no data type in Ms Access 2003 cannot handle null values appropriately. Hence, the error. Check this link for details.
Update 2
Real Query with the joins. Didnt mention it since i though the information provided above would work.
SELECT tblovertime.contfirstname AS [First Name],
tblovertime.contlastname AS [Last Name],
tblovertime.employeenumber AS [Employee Number],
tblsignup.thedate AS [Sign Up Date],
Iif([tblOvertime.DayShift] =- 1, "Day", "Night") AS shift,
(SELECT Mid(MIN(Iif(sector = 1, "," & sector, NULL)) & MIN(
Iif(sector = 2, "," & sector, NULL)) & MIN(
Iif(sector = 3, "," & sector, NULL)) & MIN(
Iif(sector = 4, "," & sector, NULL)), 2) AS concat
FROM tblempsectorlist
WHERE tblempsectorlist.empnum = tblsignup.employeenumber
GROUP BY empnum) AS sectors,
tblovertime.timedatecontact AS [Date Contacted],
tblovertimestatus.name AS status
FROM (tblsignup
INNER JOIN tblovertime
ON ( tblsignup.thedate = tblovertime.otdate )
AND ( tblsignup.employeenumber = tblovertime.employeenumber ))
INNER JOIN tblovertimestatus
ON Clng(tblovertime.statusid) = tblovertimestatus.statusid
WHERE (( ( tblsignup.thedate ) ># 1 / 1 / 2011 # ))
ORDER BY tblsignup.thedate;
Your second one has it right
SELECT iif(DayShift,"Day","Night") as Shift FROM table1;
I suggest trying the following to see what's actually being evaluated
SELECT iif(DayShift,"Day","Night") as Shift, DayShift FROM table1;
You could equally do
SELECT iif(DayShift = -1,"Day","Night") as Shift FROM table1;
Since MS Access is storing true as -1 and false as 0 (it's not as intuitive as true = 1, but it's probably faster to evaluate in twos-compliment)
-- edit --
Since you appear to be using a join, which can result in Nul's for Yes/No's, use the nz() function.
select iff(nz(DayShift, 0), "Day","Night") as Shift FROM table1;
When DayShift comes out null, this will return 0 (false/no) as a result instead.
This one might be stupid, but ....
In case you have some Null values in the DayShift field, Access will not be able to evaluate the formula. You could write your test this way:
iif(Nz(DayShift,0)=-1,"Day","Night")
I ran the following query successfully:
SELECT IIf([Table1]![isDayShift]=True,"Day","Night") AS Shift
FROM Table1;
I built this in Access 2007 (sorry, I didn't have a copy of 2003 lying around). The only difference I saw was that it gave the full path to the field instead of just the field name. However, that shouldn't be an issue. If it is giving you checkboxes, it would seem that it is not seeing this field as a text field but rather as a checkbox field still.
Bottom line is the above query should work.
You are binding to "DayShift" rather then the derived "Shift" in the form.
The 3rd query (=True) show when run in isolation should show Day/Night.
The Access database engine (ACE, Jet, whatever) has a clever/stupid feature that allows an expression in the SELECT clause to refer to an AS clause ("column alias") in the same SELECT clause if that AS clause is to the left of the expression. This makes it easy to test expressions with test data without using a table at all (assuming ANSI-92 Query Mode, I think) e.g. try executing any of the following statements individually: all should work and produce the expected result:
SELECT CBOOL(TRUE) AS DayShift,
IIF(DayShift = TRUE, 'Day', 'Night') AS Shift;
SELECT CBOOL(FALSE) AS DayShift,
IIF(DayShift = TRUE, 'Day', 'Night') AS Shift;
SELECT NULL AS DayShift,
IIF(DayShift = TRUE, 'Day', 'Night') AS Shift;
SELECT CBOOL(TRUE) AS DayShift,
IIF(DayShift, 'Day', 'Night') AS Shift;
SELECT CBOOL(FALSE) AS DayShift,
IIF(DayShift, 'Day', 'Night') AS Shift;
SELECT NULL AS DayShift,
IIF(DayShift, 'Day', 'Night') AS Shift;