I have a "flat file" with structure as below:
machineCode,Key,Ip_Name_No,Share_Percent,Account_Name,Account_No
"ygh048GT",4767,534293748,"100.00","cderfgdsc Publishing International Ltd","160102040"
"xcd064HW",6380,65424090,"100.00","dascdfrgh snm skion","00090382478"
"000065AN",6402,65424090,"100.00","xcdertn,john sean","00090382478"
.....
The first row are the column headings. As can be seen, the fields are separated by a comma.
The requirement is to split the single string into separate fields.
This could be done by excel and then uploaded to a DB table using the data to columns option with comma as delimiter but the Account_Name field can contain commas within the values itself.
So, I came up with the below SQL. Question is, does this look correct ? Also, there must be some easier way to do this, any suggestions ?
WITH POS AS (
select
LOCATE_IN_STRING ( DATA , ',' , 2 ) - 1 AS TUNECODE_END ,
LOCATE_IN_STRING (DATA, ',' , LOCATE_IN_STRING ( DATA , ',' , 2 ) + 1) - 1 AS WORKKEY_END,
LOCATE_IN_STRING( DATA , ',' , (LOCATE_IN_STRING (DATA, ',' , LOCATE_IN_STRING ( DATA , ',' , 2 ) + 1) + 1) ) - 1 AS IPNN_END,
LOCATE_IN_STRING( DATA , ',' , (LOCATE_IN_STRING( DATA , ',' , (LOCATE_IN_STRING (DATA, ',' , LOCATE_IN_STRING ( DATA , ',' , 2 ) + 1) + 1) ) + 1) ) - 1 AS PERC_END,
CASE WHEN
SUBSTR ( DATA ,
(
LOCATE_IN_STRING ( DATA , ',' ,
(LOCATE_IN_STRING( DATA , ',' , (LOCATE_IN_STRING( DATA , ',' , (LOCATE_IN_STRING (DATA, ',' , LOCATE_IN_STRING ( DATA , ',' , 2 ) + 1) + 1) ) + 1) ) + 1 ) ) + 1),
1) <> '"'
THEN
LOCATE_IN_STRING ( DATA , ',' ,
(LOCATE_IN_STRING ( DATA , ',' ,
(LOCATE_IN_STRING( DATA , ',' , (LOCATE_IN_STRING( DATA , ',' , (LOCATE_IN_STRING (DATA, ',' , LOCATE_IN_STRING ( DATA , ',' , 2 ) + 1) + 1) ) + 1) ) + 1 ) ) + 1)) - 1
ELSE
LOCATE_IN_STRING ( DATA , ',' ,
(LOCATE_IN_STRING( DATA , ',' , (LOCATE_IN_STRING( DATA , ',' , (LOCATE_IN_STRING (DATA, ',' , LOCATE_IN_STRING ( DATA , ',' , 2 ) + 1) + 1) ) + 1) ) + 1 ) ) - 1
END AS ACNAME_END,
RRN(P) ROWN
FROM PLDWRK P
) SELECT
CAST ( SUBSTR( DATA , 1, TUNECODE_END ) AS CHAR(25))AS MACHINECODE ,
CAST ( SUBSTR( DATA , TUNECODE_END + 2 , WORKKEY_END - (TUNECODE_END + 1) ) AS DEC(12,0)) AS KEY,
CAST(SUBSTR( DATA , WORKKEY_END + 2, IPNN_END - (WORKKEY_END + 1) ) AS DEC(12, 0 )) AS IP_NN,
CAST (SUBSTR( DATA, IPNN_END + 2, PERC_END - (IPNN_END + 1)) AS CHAR(8))AS PERCENTAGE,
CAST (SUBSTR( DATA, PERC_END + 2, ACNAME_END - (PERC_END + 1)) AS CHAR(100)) AS ACCOUNT_NAME,
CAST (SUBSTR( DATA, ACNAME_END + 2 ) AS CHAR(30)) as ACCOUNT_NUMBER
FROM PLDWRK P JOIN POS ON ROWN = RRN(P)
It doesn't work properly for the last row, since one token contains , inside...
select
t.str
, regexp_substr (t.str, '[^,]+', 1, 1) as tok1
, regexp_substr (t.str, '[^,]+', 1, 2) as tok2
---, ...
, regexp_substr (t.str, '[^,]+', 1, 6) as tok6
from
(
values
(1, 'machineCode,Key,Ip_Name_No,Share_Percent,Account_Name,Account_No')
, (2, '"ygh048GT",4767,534293748,"100.00","cderfgdsc Publishing International Ltd","160102040"')
, (3, '"xcd064HW",6380,65424090,"100.00","dascdfrgh snm skion","00090382478"')
, (4, '"000065AN",6402,65424090,"100.00","xcdertn,john sean","00090382478"')
) t (id, str)
order by t.id
STR
TOK1
TOK2
TOK6
machineCode,Key,Ip_Name_No,Share_Percent,Account_Name,Account_No
machineCode
Key
Account_No
"ygh048GT",4767,534293748,"100.00","cderfgdsc Publishing International Ltd","160102040"
"ygh048GT"
4767
"160102040"
"xcd064HW",6380,65424090,"100.00","dascdfrgh snm skion","00090382478"
"xcd064HW"
6380
"00090382478"
"000065AN",6402,65424090,"100.00","xcdertn,john sean","00090382478"
"000065AN"
6402
john sean"
Lots of things are easier than raw SQL...
Why not simply use Copy From Import File (CPYFRMIMPF) that's what it's designed for.
CPYFRMIMPF FROMSTMF('/inbound/somedata.csv') TOFILE(MYLIB/MYTABLE) MBROPT(*REPLACE) RCDDLM(*CRLF) DTAFMT(*DLM) STRDLM(*DBLQUOTE) RMVCOLNAM(*YES)
You'll have to transfer the stream data into the IFS (where it really belongs) instead of a DB table.
IBM's Access Client Solutions (ACS) includes data transfer functionality that can understand .CSV files. This can be automated and can in fact run on either a PC or the IBM i itself.
Another great option would be an RPG program, back in 2008 Scott Klement wrote a CSV parser in RPG. He's since enhanced it to make it easier to use by taking advantage of RPG's DATA-INTO op-code.
Lastly it's 2023...node.js, PHP, Python are all available on the IBM i and all of them have libraries/packages to handle CSV and write to a DB table.
I am inserting a job step in Azure Elastic job agent. This is my code:
EXEC jobs.sp_add_jobstep #job_name='Sample T-SQL',
#command=N' declare #columns varchar(max)
select #columns = stuff(( select '],[' + [$Name]
from dbo.groups
order by '],[' + convert(varchar(max), [$_ClosingBalance]) desc
for xml path('')), 1, 2, '' + ']'
INSERT INTO dbo.Trial1
( [Bank Accounts],[Bank OD A/c],[Branch / Divisions],[Capital Account],[Cash-in-Hand],[Current Assets],[Current Liabilities],[Deposits (Asset)]
,[Direct Expenses],[Direct Incomes],[Duties & Taxes],[Fixed Assets],[Duties & Taxes1],[Duties & Taxes2],[Indirect Expenses],[Indirect Incomes],[Investments]
,[Loans & Advances (Asset)],[Loans (Liability)],[Misc# Expenses (ASSET)],[Provisions],[Purchase Accounts],[Reserves & Surplus],[Sales Accounts],[Secured Loans]
,[Stock-in-Hand],[Stock Transfer Outward],[Sundry Creditors],[Sundry Debtors],[Suspense A/c],[Duties & Taxes3]
,[Unsecured Loans]
)
Select * from (
Select [$_ClosingBalance], RowN = Row_Number() over (order by #columns) from dbo.Groups
) a
pivot (max([$_ClosingBalance]) for RowN in ([1],[2],[3],[4],[5],[6],[7],[8],[9],[10],[11],[12],[13],[14],[15],[16],[17],[18],[19],[20],[21],[22],[23],[24],[25],[26],[27],[28],[29],[30],[31],[32])) p
INSERT INTO dbo.Date ([Date]) VALUES (GETDATE())
INSERT INTO dbo.Final
SELECT *
FROM dbo.Trial1
INNER JOIN dbo.Date
ON dbo.Trial1.IdCol = dbo.Date.IDCol1'',
#credential_name='JobRun',
#target_group_name='DatabaseGroup5'
I am receiving the following error :
Msg 102, Level 15, State 1, Line 3
Incorrect syntax near ']'.
Msg 137, Level 15, State 2, Line 15
Must declare the scalar variable "#columns".
Msg 102, Level 15, State 1, Line 24
Incorrect syntax near ''
My code is ending where it should not. Please help.
after that you assigned #command ,you can write : print #command
then you can look printed script ,copy it and paste in query window.then probebly you will understand your mistake.
I am having an issue on joining a table to a result in a substring in SQL.
Here is my statement:
SELECT a.MEASR_COMP_ID
, f.D1_USAGE_ID
, f.SCH_SELECTION_DT
, g.MEASR_COMP_ID
, g.MSRMT_VAL
, i.USAGE_ID
, SUBSTR(dbms_lob.substr(i.USG_DATA_AREA, 4000, 1)
,INSTR(dbms_lob.substr(i.USG_DATA_AREA, 4000, 1)
,'<startReading>') + LENGTH('<startReading>'), 8)
"Start Reading"
, SUBSTR(dbms_lob.substr(i.USG_DATA_AREA, 4000, 1)
, INSTR(dbms_lob.substr(i.USG_DATA_AREA, 4000, 1)
, '<endReading>') + LENGTH('<endReading>'), 8) as "End Reading"
FROM C1_USAGE I
, CISADM.D1_USAGE#CCBMDM f
, CISADM.D1_MSRMT#CCBMDM g
, CISADM.D1_USAGE_PERIOD_SQ#CCBMDM a
WHERE f.USG_EXT_ID = i.USAGE_ID AND
f.D1_USAGE_ID = a.D1_USAGE_ID AND
a.MEASR_COMP_ID = g.MEASR_COMP_ID AND
i.USAGE_ID = '119993413555' AND
SUBSTR(dbms_lob.substr(i.USG_DATA_AREA, 4000, 1)
,INSTR(dbms_lob.substr(i.USG_DATA_AREA, 4000
,1), '<endReading>') + LENGTH('<endReading>'), 8)
as "End Reading" = g.MSRMT.VAL
It should be fine except that you should remove the part as "End Reading" from your condition and also consider using proper ANSI style JOIN syntax rather this puzzled syntax
I have this query which runs fine and with no problem!
SELECT [Q].sHost
, LEFT([Q].sDescription, Len([Q].sDescription) - 1) AS [sDescription]
FROM (
SELECT DISTINCT [Q2].sHost
, (
SELECT CONVERT(NVARCHAR(MAX), [Q1].[sDescription]) + N', ' AS [text()]
FROM (
SELECT (
CASE
WHEN (CHARINDEX('\', [sInstance]) = 0)
THEN [sInstance]
ELSE substring([sInstance], 0, CHARINDEX('\', [sInstance]))
END
) AS sHost
, [sDescription]
FROM [db_group].[dbo].[instanceCommentsList]
) AS [Q1]
WHERE ([Q1].sHost = [Q2].sHost)
FOR XML PATH('')
) [sDescription]
FROM (
SELECT (
CASE
WHEN (CHARINDEX('\', [sInstance]) = 0)
THEN [sInstance]
ELSE substring([sInstance], 0, CHARINDEX('\', [sInstance]))
END
) AS sHost
, [sDescription]
FROM [db_group].[dbo].[instanceCommentsList]
) AS [Q2]
) AS [Q]
when I save this query as a view, I get the following error
Error in WHERE clause near '('.
Error in WHERE clause near '='. Unable
to parse query text.
not sure where is the problem!
it's saves any way and it kind of works, I can use it in a simple select query but I get a red line under the name of the view and when I put the mouse over I get this message
The object name is invalid ....
and if I use it in a more complex query it does not work at all
EDIT:----------------------------------
after reading Cannot get FOR XML PATH to work thanks to SelectDistinct's comment
I fixed it but still get one more error!
Error in WHERE clause near '('.
Unable to parse query text.
here is the fixed code:
SELECT [Q].sHost
, LEFT([Q].sDescription, Len([Q].sDescription) - 1) AS [sDescription]
FROM (
SELECT DISTINCT [Q2].sHost
, (
SELECT CONVERT(VARCHAR(MAX), [Q1].[sDescription]) + ', ' AS [text()]
FROM (
SELECT (
CASE
WHEN (CHARINDEX('\', [sInstance]) = 0)
THEN [sInstance]
ELSE substring([sInstance], 0, CHARINDEX('\', [sInstance]))
END
) AS sHost
, [sDescription]
FROM [db_group].[dbo].[instanceCommentsList]
) AS [Q1]
WHERE ([Q1].sHost = [Q2].sHost)
FOR XML PATH(''), type
).value('.', 'varchar(max)') as [sDescription]
FROM (
SELECT (
CASE
WHEN (CHARINDEX('\', [sInstance]) = 0)
THEN [sInstance]
ELSE substring([sInstance], 0, CHARINDEX('\', [sInstance]))
END
) AS sHost
, [sDescription]
FROM [db_group].[dbo].[instanceCommentsList]
) AS [Q2]
) AS [Q]
Try WHERE [Q1].sHost = [Q2].sHost instead of WHERE ([Q1].sHost = [Q2].sHost) .
I am a newby at sql and can find the problem with my query, I am getting this error:
Msg 245, Level 16, State 1, Line 10
Conversion failed when converting the nvarchar value '00999-00-210312-11' to data type int.
Declare #Referenceid Int
SET #Referenceid = (
SELECT TOP 1 (CASE WHEN ad.ReferenceID LIKE '%NO ID%' THEN (RIGHT(MAX(ad.ReferenceID),2)+10)END)
FROM Campaign.dbo.Assets_DailyBulkImport AS ad
WHERE (ad.ReferenceID IS NOT NULL)
GROUP BY ad.ReferenceID
ORDER BY ad.ReferenceID DESC);
Select (c.Note + '' + '999' + N'-' + '00' + N'-' +
(SELECT REPLACE(CONVERT(VARCHAR(8), GETDATE(), 3), '/', '') AS MMDDYY) + N'-' + #Referenceid)
FROM Assets_DailyBulkImport AS a INNER JOIN
Controls_Profiles AS c ON a.Profile = c.Venture
where a.ReferenceID=#ReferenceID
it says that '00999-00-210312-11' is a string value you cannot convert that in integer value. as you can see - is part of data so it must be string i.e varchar its not integer value that the reason its giving error.
so to resolve you error you need to change this line
(SELECT REPLACE(CONVERT(VARCHAR(8), GETDATE(), 3), '/', '') AS MMDDYY) + N'-'
+ cast(#Referenceid as varchar(10))
in your code #Referenceid is integer value you cannot directly append it you first need to convert it like cast(#Referenceid as varchar(10))