So i have a requirement where I need to read through records of all records of a file and insert them into another file if they meet a set of rules which are described in another table as shown below..
A record after it has been read from the first file has to meet all the sequences of at least one Rule to make it eligible to be written into the Second table.
For example once a record is read from CAR file, the rules below have to be checked till all sequences of atleast one rule set is satisfied. For this I was planning to Create a dynamic SQL program something of this sort. But this does not work as Prepared SQL does not support host variables.
If any body can suggest or provide any guidance on how to create SQL statemtns dynamically and check if records satisfy the required rules for them to be entered into the second file, it would be great
So basically what I am looking for is once I select a field from a table, how do I store it somehere to do further validation and checking.
Update
:
Based on the intelligent advice from Danny117, I have come up with the below code:
H Option(*NoDebugIO:*SrcStmt)
D RULEDS E DS EXTNAME(RULESTABLE)
D MAXRUL S 1 0
D MAXSEQ S 1 0
D STMT S 512
D WHERESTMT S 512 INZ('')
D FullSqlStmt S 512 INZ('')
D RULINDEX S 1 0 INZ(1)
D SEQINDEX S 1 0 INZ(1)
D APOS C CONST('''')
/Free
Exec SQL SELECT MAX(RULENO)INTO :MAXRUL FROM RULESTABLE;
Exec SQL DECLARE RULCRS CURSOR FOR SELECT * FROM RULESTABLE;
Exec SQL OPEN RULCRS;
Exec SQL FETCH RULCRS INTO :RULEDS;
DoW (Sqlcod = 0 AND RULINDEX <= MAXRUL);
Exec SQL SELECT MAX(SEQNO) INTO :MAXSEQ FROM RULESTABLE
WHERE RULENO=:RULINDEX ;
DoW (SEQINDEX <= MAXSEQ);
If (Position <> '');
Field = 'SUBSTR('+%Trim(Field)+','+%Trim(Position)+','
+'1'+')';
EndIf;
WhereStmt = %Trim(WhereStmt) + ' ' + %Trim(field)+ ' ' +
%Trim(condition) + ' ' + APOS + %Trim(Value) + APOS;
If (SeqIndex < MaxSeq);
WhereStmt = %Trim(WhereStmt) + ' AND ';
EndIf;
Exec SQL FETCH NEXT FROM RULCRS INTO :RULEDS;
SeqIndex = SeqIndex + 1;
EndDo;
FullSqlStmt = %Trim('INSERT INTO ITMRVAT SELECT * +
FROM ITMRVA WHERE '+ %Trim(WhereStmt));
Exec SQL Prepare InsertStmt from :FullSqlStmt;
Exec SQL EXECUTE InsertStmt;
RulIndex = RulIndex + 1;
EndDo;
This produces SQL statement as shown below which is what I want. Now let me go ahead and look at the other parts of the code.
> EVAL FullSqlStmt
FULLSQLSTMT =
....5...10...15...20...25...30...35...40...45...50...55...60
1 'INSERT INTO ITMRVAT SELECT * FROM ITMRVA WHERE STID = 'PLD' '
61 'AND ENGNO LIKE '%415015%' AND SUBSTR(ENGNO,1,1) = 'R' AND SU'
121 'BSTR(ENGNO,5,1) = 'Y' '
181 ' '
241 ' '
301 ' '
361 ' '
421 ' '
481 ' '
But the issue is now as I mentioned in my comment to Danny, how to handle if a new rule involving second table is specified..
Embedded SQL does allow for 'dynamic statements' in ILE languages. You are able to have a query within a character field and then pass it into the Embedded SQL.
Dcl-S lQuery Varchar(100);
lQuery = 'SELECT * FROM CUST';
EXEC SQL
PREPARE SCust FROM :lQuery;
EXEC SQL
DECLARE SearchCust CURSOR FOR SCust;
//Continue working with cursor..
You may want to just prepare, execute and return a result set:
lQuery = 'SELECT * FROM CUST WHERE ID = ' + %Char(CustID);
EXEC SQL
PREPARE SCust FROM :lQuery;
DECLARE c1 CURSOR FOR SCust;
OPEN c1;
FETCH c1 INTO :CustDS;
CLOSE c1;
Optional extra: You may also want to use field markers (?) in your query.
//'SELECT * FROM CUST WHERE CUSTID = ?';
EXEC SQL OPEN SearchCust USING :CustID;
//'INSERT INTO CUST VALUES(?,?)';
EXEC SQL EXECUTE CUST USING :CustID;
You have to translate the rules into a join statement or a where clause. The join statement is more complex so go that route.
If you were smart (and you are) consider saving the rules as a SQL clause that you can join or use in a where clause. Its infinitely flexible this way a more modern design.
rule 1 / car.year = 1990 and car.engno like '%43243%' and substring(car.vin,12,1) = 'X'
eval statement =
insert into sometable
Select car.* from car
join sysibm.sysdummy1
on car.year = 1990
and car.engno lile '%43243%'
...etc on to rule 2 starting with "OR"
or car.year = PLD
and car.engno like '%1234%'
...etc other rules starting with "OR"
exec immediate statement
Related
I believe that the cursor used in this code is the reason for some major performance issues, however I am new to TSQL.
Following script runs on SQL SERVER 2008. I am trying to redo it so I use JOIN statements instead, however I have not been able to do so successfully.
DECLARE AIRAMSDET CURSOR FOR
SELECT BILL, RECIEPT, NAME
FROM Client_Table
WHERE IsProcessed = 1
AND TYPE IN ('Sub','First_Time','Old') AND LEN(BILL) > 1
OPEN AIRAMSDET
FETCH AIRAMSDET into #VARBILL, #VARRECIEPT, #VARNAME
WHILE ##Fetch_Status = 0
BEGIN
UPDATE archieve
SET entry = left(#VARBILL + '- '+ #VARNAME)
WHERE archiveID = #VARBILL
END
It should be something like following
UPDATE ARCHIEVE
SET ENTRY = CT.BILL + '-' + CT.NAME
FROM CLIENT_TABLE CT
WHERE
ARCHIEVE.ARCHIVEID = CT.BILL
AND CT.ISPROCESSED = 1
AND CT.TYPE IN ('Sub','First_Time','Old') AND LEN(BILL) > 1
I have not included LEFT() as its use in your query wasn't very clear. Left takes an integer_expression as its second parameter while you are passing ##VARNAME which most likely is a VARCHAR. Please add that as you deem fit.
After reading several articles about SQLRPGLE and retrieving data and storing them in data structure arrays, I came up with dynamic sql statements.
This works fine as long as I am using these dynamic to-replace fields for my where condition. But as soo as I am using these ? parameter in the select part, or in general as replacement for database fields, the result is blank.
Here is the DDS definition and the program:
TESTPF
A**************************************************************************
A*
A*-------------------------------------------------------------------------
A*
A R TESTPFR
A
A FLD01 2S 0
A FLD02 20A
A
A**************************************************************************
I have already filled this file with some dummy data. Here is what's inside:
runqry () qtemp/testpf
FLD01 FLD02
000001 1 Text 01
000002 2 Text 02
000003 3 Text 03
000004 4 Text 04
000005 5 Text 05
000006 6 Text 06
000007 7 Text 07
000008 8 Text 08
000009 9 Text 09
000010 10 Text 10
And this is the program:
TST001I
D**********************************************************************************************
D* Standalone Fields
D*---------------------------------------------------------------------------------------------
D stm s 500a inz(*blanks)
D fieldName01 s 10a inz(*blanks)
D fieldName02 s 10a inz(*blanks)
D fieldName03 s 2a inz(*blanks)
D text s 20a inz(*blanks)
D
C**********************************************************************************************
C* M A I N P R O G R A M M
C**********************************************************************************************
stm = 'SELECT fld02 FROM testpf WHERE fld01 = 1';
exec sql prepare s1 from :stm;
exec sql declare c1 cursor for s1;
exec sql open c1;
exec sql fetch c1 into :text;
exec sql close c1;
dsply text; // Prints 'Text 01'
text = *blanks;
stm = 'SELECT fld02 FROM testpf WHERE fld01 = ?';
exec sql prepare s2 from :stm;
exec sql declare c2 cursor for s2;
fieldName03 = '2';
exec sql open c2 using :fieldName03;
exec sql fetch c2 into :text;
exec sql close c2;
dsply text; // Prints 'Text 02'
text = *blanks;
stm = 'SELECT ? FROM testpf WHERE fld01 = 3';
exec sql prepare s3 from :stm;
exec sql declare c3 cursor for s3;
fieldName01 = 'FLD02';
exec sql open c3 using :fieldName01;
exec sql fetch c3 into :text;
exec sql close c3;
dsply text; // Prints ' '
text = *blanks;
stm = 'SELECT ? FROM testpf WHERE ? = ?';
exec sql prepare s4 from :stm;
exec sql declare c4 cursor for s4;
fieldName01 = 'FLD02';
fieldName02 = 'FLD01';
fieldName03 = '4';
exec sql open c4 using :fieldName01, :fieldName02, :fieldName03;
exec sql fetch c4 into :text;
exec sql close c4;
dsply text; // Prints ' '
text = *blanks;
*inlr = *on;
C**********************************************************************************************
This is the output:
DSPLY Text 01
DSPLY Text 02
DSPLY
DSPLY
DSPLY
May someone help me and explain why this is the case?
When using a prepared statement, you can use ? as a parameter marker wherever you can use a host variable in a static statement. Of your four sample prepared statements, the first 3 should work, though the third one will not return what you seem to expect as it is equivalent to:
SELECT 'FLD02' FROM testpf WHERE fld01 = 3
I would expect to receive the value 'FLD02' as the result, not the value in column FLD02. This is because the ? is not a string replacement marker, but a parameter field marker. You can't use it to select a column, but you can use it to provide a value for comparisons, or a constant to be output.
The fourth sample is valid SQL, but it is equivalent to:
SELECT 'FLD02' FROM testpf WHERE 'FLD01' = '4'
This will return nothing since 'FLD01' does not equal '4'.
Another consequence of this is that the ? can be used to provide a numeric value to the prepared statement. So you can do this:
dcl-s seqno Packed(5:0);
exec sql declare c2 cursor for s2;
stm = 'SELECT fld02 FROM testpf WHERE fld01 = ?';
exec sql prepare s2 from :stm;
seqno = 2;
exec sql open c2 using :seqno;
Also notice that I removed the declaration of the cursor to somewhere outside the logic flow as the declaration is not an executable statement. I see programs where the declare is in a subroutine that is called before a separate subroutine containing the open for the cursor. This is semantically incorrect. The DECLARE CURSOR statement is more correctly equivalent to an RPGLE dcl- statement. But because the SQL precompiler processes the source linearly, largely without regard to subroutines or sub-procedures, the requirement is for the DECLARE CURSOR to be physically before the OPEN in the source.
Generally I like to put my SQL Declares at the head of the program right after the SET OPTION statement which must be the first SQL embedded in the program. This is where I put the declares when I am using prepared statements. I also declare the statement name as well though this isn't strictly necessary. There is a little gotcha for this though, and that exists when using static SQL with locally scoped host variables. To deal with this, I declare static cursors a bit differently when using sub-procedures. The SQL precompiler recognizes that sub-procedures use locally scoped variables, so if you are declaring a static cursor with locally scoped host variables, the host variables and the cursor declaration must be in the same scope. That means I must declare my static cursors in the same sub-procedure as the open. I still declare the cursor up near the RPGLE dcl- statements to keep the declarations together.
Below query takes 20 secs to execute and i need to optimize it as much as i can. Please help me on this.
SELECT Distinct
qh.QuoteHeaderId, [dbo].[mpx2_Get_PhoneGrade](qh.QuoteHeaderId)
FROM
t_QuoteHeader QH
INNER JOIN
t_HandsetQuote h ON Qh.QuoteHeaderId = h.QuoteHeaderId
INNER JOIN
t_phoneAudit P ON ISNULL(h.InspectionPhoneAuditId, h.QuotePhoneAuditId) = p.PhoneAuditId
INNER JOIN
mpx2_vw_customers C ON qh.CustomerId = C.CustomerId
INNER JOIN
#ContactChannels CC ON C.ContactChannelId = CC.ContactChannelId
LEFT OUTER JOIN
t_HandsetQuoteAdditionalInfo_TRNX hqa ON hqa.hqid = h.HandsetQuoteId
WHERE
((#VirtualStatusId = 0 OR #VirtualStatusId = -2 OR
C.ContactChannelId NOT IN (1, 2, 13, 80)))
AND ((#VirtualStatusId = -2) OR
('Q'+ CAST(Qh.QuoteStatusId AS VARCHAR(3)) + 'S' + CAST(h.StockStatusId AS VARCHAR(3)) IN
(SELECT 'Q'+ CAST(QuoteStatusId AS VARCHAR(3)) + 'S' + CAST(StockStatusId AS VARCHAR(3)) FROM t_VirtualStatusMap WHERE (#VirtualStatusId IS NULL OR #VirtualStatusId IN (0,-1) OR VirtualStatusId = #VirtualStatusId))
)
)
AND ((qh.IsCancelled = 0 and #onlyOpenOrders = 1) OR #onlyOpenOrders = 0)
AND ((h.IsCancelled = 0 and #onlyOpenOrders = 1) OR #onlyOpenOrders = 0)
AND (qh.ConfirmedDate <= #CutOff)
Please help me to optimize it. This query is used in a stored procedure.
This is too long for a comment.
ORs in WHERE and ON clauses are very hard to optimize. Often with a query like this, it is better to construct the query based on the components and use dynamic SQL.
For instance, the condition on #OnlyOpenOrders would be included like this:
declare #sql varchar(max);
set #sql = ' . . .';
declare #where varchar(max);
set #where = 'where . . .';
if (#OnlyOpenOrders = 0) begin
set #where = #where + ' and qh.IsCancelled = 0 and h.IsCancelled = 0'
end;
set #sql = #sql + ' ' + #where;
exec sp_executesql #sql;
You need to have similar logic for all the variables you are using.
There are a couple of things, although as others have said without all the required information such as a full execution plan, and schemas of the tables involved it is mostly guidelines/guesswork;
1.) In this part, it would appear you build a string from QuoteStatusId and StockStatusId in order to compare them;
('Q'+ CAST(Qh.QuoteStatusId AS VARCHAR(3)) + 'S' + CAST(h.StockStatusId AS VARCHAR(3))
IN
(SELECT 'Q'+ CAST(QuoteStatusId AS VARCHAR(3)) + 'S' + CAST(StockStatusId AS VARCHAR(3))
FROM t_VirtualStatusMap
WHERE (#VirtualStatusId IS NULL
OR #VirtualStatusId IN (0,-1) OR VirtualStatusId = #VirtualStatusId)))
If you skipped building the strings, since they are comprised of the same two columns and just compared the two columns directly that may speed things up.
2.) Have you tried adding the index which it suggests in the picture you attached? Without seeing your schema and an execution plan it is hard to suggest appropriate ones but it might be worth adding the one suggested (right click the green writing and it will generate the code to add the suggested index). I would read up on indexes and ensure there is an appropriate index for the query to use. ConfirmedDate seems like an obvious one, as well as all the join keys.
3.) As Gordon suggested using dynamic sql or if you are not comfortable with that - maybe splitting the query out into a few queries and switching between each using an IF statement, could help SQL generate a good plan for each scenario, instead of trying to find a generic plan to work for all cases.
How do I make the query like this?
UPDATE DUA_DATA_FIL_AUD
SET REV = :rev,
SYS_UPDT_TS = :now
WHERE DUA_DATA_FIL_ID = 283
AND REV = 2524;
And so on for all the next 13 records and update all the corresponding columns?
Create a result of dynamic update queries using a select form the table you want to apply the updates on it's result set.
This shall create you a sort of script that you can copy and run in your command window to update the desired lines.
Hope this addresses what you really want :
SELECT 'UPDATE DUA_DATA_FIL_AUD SET REV = :rev, SYS_UPDT_TS = :now WHERE
DUA_DATA_FIL_ID =' || DUA_DATA_FIL_ID || 'AND REV =' || MAX(REV) || ';/'
FROM DUA_DATA_FIL_AUD GROUP BY DUA_DATA_FIL_ID,REV
I am looping through all my databases and aggregating the results into an aggregates database.
In my loop I call
master.dbo.xp_cmdshell osql C:\whatever.SQL
As the loop progresses, the cmdshell takes longer and longer to execute. If I stop the loop and run a single aggregate for one database it executes quickly.
Is there anything I can add to my external SQL script to make it run faster? Maybe something to commit and free the records before the next loop? Or should I add some kind of a pause after every loop?
I want to use an external SQL file because it contains many update statements and it's more manageable for me.
Here's how I loop:
Update dbFoo.dbo.tblBar set Processed = 0
Go
WHILE EXISTS ( SELECT ID FROM dbFoo.dbo.tblBar WHERE Processed = 0)
BEGIN
SELECT #aRow = MIN(tblBar.ID) FROM dbFoo.dbo.tblBar
SELECT #aFoo1 = Foo1 FROM dbFoo.dbo.tblBar WHERE ID = #aRow
SELECT #aFoo2 = Foo2 FROM dbFoo.dbo.tblBar WHERE ID = #aRow
SELECT #aFoo3 = Foo3 FROM dbFoo.dbo.tblWhatever WHERE Foo = #aFoo
EXEC RunPreAgg #Foo1 = #aFoo1, #Foo2 = #aFoo2, #Foo3 = #aFoo3, #RetVal = #aRetVal OUTPUT
SELECT returning = #aRetVal
UPDATE dbFoo.dbo.tblBar SET Processed = 1 WHERE ID = #aRow
END
Then the RunPreAgg stored procedure basically does this:
if db_id('db' + #Foo1 + '_' + #Foo2) is not null
BEGIN
--This bat file creates the SQL File
select #sql = 'master.dbo.xp_cmdshell '''+#path+'wwwRunPreAgg.bat ' + #Foo1 + ' ' + #Foo2 + ' ' + #Foo3 + ''''
exec( #sql )
--execute
select #sql = 'master.dbo.xp_cmdshell ''osql -E -o '+#path+'output\tmp'+#Foo1+'_'+#Foo2+'.txt -i '+#path+'tmp' + #Foo1 + '.SQL'''
exec( #sql )
--This erases the SQL File
select #sql = 'master.dbo.xp_cmdshell '''+#path+'wwwCleanup.bat ' + #Foo1 + ' ' + #Foo2 + ''''
exec( #sql )
Set #retval = 'Done!'
END
ELSE
BEGIN
Set #retval = 'Err: No DataBase'
END
The variable names are changed to protect the innocent. The code works fine, I just need to optimize.
If it is the loops performance that is causing you trouble, you might try reducing the number of selects. Normally I dislike Cursors, but your loop might benefit from one. You can select all the values you need for the loop into memory, then loop through those values without having to run 3 or 4 selects per loop (of course if the performance hit is occurring inside the RunPreAgg SP, then this won't help):
DECLARE cFoos CURSOR FOR
SELECT tblBar.ID, tblBar.Foo1, tblBar.Foo2, tblWhatever.Foo3
FROM dbFoo.dbo.tblBar
INNER JOIN dbFoo.dbo.tblWhatever
ON tblWhatever.Foo = tblBar.Foo
WHERE tblBar.Processed = 0;
OPEN cFoos;
FETCH NEXT FROM cFoos INTO #aRow, #aFoo1, #aFoo2, #aFoo3;
WHILE ##FETCH_STATUS = 0
BEGIN
EXEC RunPreAgg #Foo1 = #aFoo1, #Foo2 = #aFoo2, #Foo3 = #aFoo3, #RetVal = #aRetVal OUTPUT
SELECT returning = #aRetVal
UPDATE dbFoo.dbo.tblBar SET Processed = 1 WHERE ID = #aRow
FETCH NEXT FROM cFoos INTO #aRow, #Foo1, #Foo2, #Foo3;
END
CLOSE cFoos;
DEALLOCATE cFoos;