Have a txt file and have to pass it to sql
A bulk insert
BULK INSERT table
FROM '\ \ 01cends5 \ TestBulk \ a.txt'
WITH (
DATAFILETYPE = 'char'
FIELDTERMINATOR = '|'
ROWTERMINATOR = '\ n ',
FIRSTROW = 1,
LASTROW = 15
)
But it do not take as a final line ROWTERMINATOR and probe everything and does not work
{CR} {LF}{LF}{CR}\ n\ r\ r \ n\ n \ r
My txt format is:
0 | 20276708598 | 119302 | 201101 | 000000 | 000000
It looks like something is wrong with '\r' translation to 0x0A, at least in my case.
http://dbaspot.com/sqlserver-programming/463913-bulk-insert-rowterminator-failing.html#post1509360 demonstrates how to do it with exec and specifying of rowterminator with char() function. It worked for me. Paste of sample code from the link above:
DECLARE #bulk_cmd varchar(1000)
SET #bulk_cmd = 'BULK INSERT [GRC].[dbo].[UP_040109]
FROM ''C:\TEMP\up\524.d.0''
WITH (ROWTERMINATOR = '''+CHAR(10)+''',FIELDTERMINATOR = ''\t'')'
EXEC(#bulk_cmd)
Related
I have an issue where I'm trying to remove all of the '.' from the string/filename below in SSMS apart from the last one which dictates file type.
EPC 14.10.14.pdf
Ideally I would like this string to appear as below:
EPC 141014.pdf
Any help would be appreciated
As a variable :
declare #doc varchar(30) = 'EPC 14.10.14.pdf'
declare #ext varchar(8) = right(#doc, charindex('.', reverse(#doc)));
set #doc = concat(replace(left(#doc,len(#doc)-len(#ext)),'.',''), #ext);
select #doc as doc;
doc
EPC 141014.pdf
As a table column :
create table test (
doc varchar(30) not null
);
insert into test (doc) values
('EPC 14.10.14.pdf'),
('FQD 15.11.15.jpeg');
select doc
, undotted_doc = concat(replace(left(doc, len(doc)-charindex('.', reverse(doc))),'.',''), right(doc, charindex('.', reverse(doc))))
from test;
doc
undotted_doc
EPC 14.10.14.pdf
EPC 141014.pdf
FQD 15.11.15.jpeg
FQD 151115.jpeg
Test on db<>fiddle here
Use replace,substring and len function
select replace(substring(#x,0,len(#x) - 3),'.','') + substring(#x,len(#x) - 3,len(#x))
EDIT:
If the name extension has a variable length, you can use the following query
select
CONCAT(
replace(substring(#x,0,len(#x) - CHARINDEX('.',TRIM(REVERSE(#x)))),'.','')
,
substring(#x,len(#x) - CHARINDEX('.',TRIM(REVERSE(#x))),len(#x))
)
Result
If you have extensions with different length (e.g. docx, xls), you need to find the index of the last occurrence of the . character using REVERSE() and CHARINDEX():
SELECT CONCAT(
REPLACE(SUBSTRING(SomeText, 1, LEN(SomeText) - CHARINDEX('.', REVERSE(SomeText))), '.', ''),
STUFF(SomeText, 1, LEN(SomeText) - CHARINDEX('.', REVERSE(SomeText)), '')
) AS FileName
FROM (VALUES
('EPC 14.10.14.pdf'),
('EPC 14.10.14.docx'),
('14.10.14.xlsx')
) t (SomeText)
Result:
FileName
----------------
EPC 141014.pdf
EPC 141014.docx
141014.xlsx
One more way.
SQL
SELECT fileName AS [Before]
, CONCAT(CONCAT(PARSENAME(fileName,4), PARSENAME(fileName,3), PARSENAME(fileName,2))
, '.', PARSENAME(fileName,1)) AS [After]
FROM (VALUES
('EPC 14.10.14.pdf'),
('EPC 14.10.14.docx'),
('14.10.14.xlsx'),
('csharp.10.14.cs')
) AS t(fileName);
Output
+-------------------+-----------------+
| Before | After |
+-------------------+-----------------+
| EPC 14.10.14.pdf | EPC 141014.pdf |
| EPC 14.10.14.docx | EPC 141014.docx |
| 14.10.14.xlsx | 141014.xlsx |
| csharp.10.14.cs | csharp1014.cs |
+-------------------+-----------------+
I've used the following script in the past without issue, so I'm not sure why it's causing me issues now.
Msg 7301, Level 16, State 2, Line 8
Cannot obtain the required interface ("IID_IColumnsInfo") from OLE DB provider "BULK" for linked server "(null)".
My code:
(
FORMAT = 'CSV',
FIELDQUOTE = '"',
FIRSTROW = 2,
FIELDTERMINATOR = ',', --CSV field delimiter
ROWTERMINATOR = '\n', --Use to shift the control to next row
TABLOCK
)
screenshot of setup and error
File Size: 112 MB
Rows: 322,190
Microsoft Server Management Studio v17.4
Can you try
ROWTERMINATOR = '\r\n'
or
ROWTERMINATOR = '0x0a'
Since you're using a CSV file the row terminator may be a line feed (LF), which 0x0a in the hexadecimal notation for. The example below accounts accounts for this type of row terminator.
BULK INSERT dbo.YourTable
FROM 'C:\FilePath\DataFile.csv'
WITH (
FORMAT = 'CSV',
FIRSTROW = 2,
FIELDQUOTE = '"',
FIELDTERMINATOR = ',',
ROWTERMINATOR = '0x0a',
TABLOCK
);
try removing the FORMAT= 'CSV' line
your file may not be RFC 4180 compliant.
this has worked for me and this error
Make sure there is not a byte-order mark (BOM) at the beginning of the file, which will cause this to fail with this error.
I have a SQL script and a ".csv" file. I want the SQL script to read the data from the ".csv" file instead of manually entering the data in the script. Is it possible?
....
.....
......
and SP_F.trade_id = SP_R.trade_id
and SP_R.iSINCode IN (here is where I can manually enter the data)
ps: I am new to SQL and I am still learning.
Here is good solution.
BULK INSERT CSVTest
FROM 'c:\csvtest.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
More explained:
1) We have csv file named test.csv with such content:
'JE000DT', 'BE000DT2J', 'DE000DT2'
1, 2, 3
2, 3, 4
4, 5, 6
2) We need to create table for this file in DB:
CREATE TABLE CSVTest ([columnOne] int, [columnTwo] int, [columnThree] int)
3) Insert your data with BULK INSERT. The columns count and type must match your csv.
BULK INSERT CSVTest
FROM 'C:\test.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
FIRSTROW = 2
)
4) Use your this table in yours subquery:
Select
SP_F.trade_id, -- as 'Trade ID',
SP_F.issuer_id, --as 'Issuer ID',
SP_R.iSINCode --as 'ISIN'
from t_SP_Fundamentals SP_F
JOIN t_SP_References SP_R ON SP_F.trade_id = SP_R.trade_id
where
(SP_F.issuer_id = 3608 or SP_F.issuer_id = 3607)
and SP_R.iSINCode IN (SELECT [columnOne] FROM CSVTest)
There is another solution with OPENROWSET statement, that allows direct reading from the file. But I strongly recommend you to use the solution above. Reading direct from the file in QUERY is not very great choose.
Trying to BULK import data in SQL server with below lines but getting error:
Msg 4866, Level 16, State 8, Line 3
The bulk load failed. The column is too long in the data file for row 1, column 96. Verify that the field terminator and row terminator are specified correctly.
Msg 7301, Level 16, State 2, Line 3
Cannot obtain the required interface ("IID_IColumnsInfo") from OLE DB provider "BULK" for linked server "(null)".
Is there anything wrong with my statements? As when I use import wizard it works fine.
BULK INSERT BICX.dbo.raw
FROM 'D:\NEW_CDR\NEW.txt'
WITH
(
FIRSTROW = 5,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
);
As you say the table contains 95 columns, and the error says column 96 is too long you have a problem with your row delimiter.
If your file came from a windows system it most likely is \r\n or you could try 0x0a if that doesn't work
BULK INSERT BICX.dbo.raw
FROM 'D:\NEW_CDR\NEW.txt'
WITH
(
FIRSTROW = 5,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\r\n'
);
or
BULK INSERT BICX.dbo.raw
FROM 'D:\NEW_CDR\NEW.txt'
WITH
(
FIRSTROW = 5,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '0x0a'
);
I have a text file with three columns of text (strings) per line. I want to create an SQL insert command by substituting each of the three strings into a skeleton SQL command. I have put place markers in the skeleton script and used SED s/placemarker1/first string/ but with no success. Is there an easier way to accomplish this task. I used pipes to repeat the process for 'second string' etc. I actually used awk to get the fields but could not convert to the actual values.
enter code here
for i in [ *x100* ]; do
if [ -f "$i" ]; then {
grep -e "You received a payment" -e "Transaction ID:" -e "Receipt No: " $i >> ../temp
cat ../temp | awk 'NR == 1 {printf("%s\t",$9)} NR == 2 {printf("%s\t",$9)} NR == 3 {printf("%s\n",$3)}' | awk '{print $2,$1,$3}' | sed 's/(/ /' | sed 's/)./ /' >> ../temp1
cat temp1 | awk 'email="$1"; transaction="$2"; ccreceipt="$3";'
cat /home/linux014/opt/skeleton.sql | sed 's/EMAIL/"$email"/' | sed 's/TRANSACTION/"$transaction"/' | sed 's/CCRECEIPT/"$ccreceipt"/' > /home/linux014/opt/new-member.sql
rm -f ../temp
} fi
done
I cannot figure out how to get the values instead of the names of the variables inserted into my string.
Sample input (one line only):
catdog#gmail.com 2w4e5r6t7y8u9i8u7 1111-2222-3333-4444
Sample actual output:
INSERT INTO users (email,paypal_tran,CCReceipt) VALUES ('"$email"','"$transaction"','"$ccreceipt"');
Preferred output:
INSERT INTO users (email,paypal_tran,CCReceipt) VALUES ('catdog#gmail.com','2w4e5r6t7y8u9i8u7','1111-2222-3333-4444');
awk '{print "INSERT INTO users (email,paypal_tran,CCReceipt) VALUES"; print "(\x27"$1"\x27,\x27"$2"\x27,\x27"$3"\x27);"}' input.txt
Converts your sample input to preferred output. It should work for multi line input.
EDIT
The variables you are using in this line:
cat temp1 | awk 'email="$1"; transaction="$2"; ccreceipt="$3";'
are only visible to awk and in this command. They are not shell variables.
Also in your sed commands remove those single quotes then you can get the values:
sed "s/EMAIL/$email/"
You can try this bash,
while read email transaction ccreceipt; do echo "INSERT INTO users (email,paypal_tran,CCReceipt) VALUES ('$email','$transaction','$ccreceipt');"; done<inputfile
inputfile:
catdog#gmail.com 2w4e5r6t7y8u9i8u7 1111-2222-3333-4444
dog#gmail.com 2dsdsda53563u9i8u7 3333-4444-5555-6666
Test:
sat:~$ while read email transaction ccreceipt; do echo "INSERT INTO users (email,paypal_tran,CCReceipt) VALUES ('$email','$transaction','$ccreceipt')"; done<inputfile
INSERT INTO users (email,paypal_tran,CCReceipt) VALUES ('catdog#gmail.com','2w4e5r6t7y8u9i8u7','1111-2222-3333-4444')
INSERT INTO users (email,paypal_tran,CCReceipt) VALUES ('dog#gmail.com','2dsdsda53563u9i8u7','3333-4444-5555-6666')
You can write a small procedure for this
CREATE PROCEDURE [dbo].[appInsert]--
#string VARCHAR(500)
AS
BEGIN
DECLARE #I INT
DECLARE #SubString VARCHAR(500)
SET #String = 'catdog#gmail.com 2w4e5r6t7y8u9i8u7 1111-2222-3333-4444'
SET #I = 1
SET #String = REPLACE(#String, ' ', '`~`')
WHILE #I > 0
BEGIN
SET #SubString = SUBSTRING (REVERSE(#String), 1, ( CHARINDEX( '`~`', REVERSE(#String)) - 1))
SET #String = SUBSTRING(#String, 1, LEN(#String) - CHARINDEX( '`~`', REVERSE(#String)) - 2 )
print REVERSE(#SubString) + ' === ' + #String
SET #i = CHARINDEX( '`~`', #String)
END
END