Example table contains the fields Id (the Identity of the table, an integer); Name (a simple attribute that allows null values, it's a string)
I'm trying a CSV that contains this:
1,
1,""
1,''
None of them gives me a empty string as the result of the bulk insertion. I'm using SQL Server 2012.
What can I do?
As far as I know, bulk insert can't insert empty string, it can either keep null value or use default value with keepnulls option or without keepnulls option. For your 3 sample records, after insert database, it should be like:
| id | name
| 1 | NULL
| 1 | ""
| 1 | ''
The reason is, the bulk insert will treat your first row, second column value as null; for other 2 rows, will take the second column value as not null, and take it as it is. Instead of let Bulk Insert to insert empty string value for you, you can let you table column having default value as empty string.
Example as following:
CREATE TABLE BulkInsertTest (id int, name varchar(10) DEFAULT '')
Bulk Insert same CSV file into table
BULK INSERT Adventure.dbo.BulkInsertTest
FROM '....\test.csv'
WITH
(
FIELDTERMINATOR ='\,',
ROWTERMINATOR ='\n'
)
SELECT * FROM BulkInsertTest
The result will be like following: (The first row in your CSV will get an empty string)
| id | name
| 1 |
| 1 | ""
| 1 | ''
Please bear in mind that the specified DEFAULT value will only get inserted if you are not using the option KEEPNULLS.
Using the same example as above, if you add the option KEEPNULLS to the BULK INSERT, i.e.:
BULK INSERT BulkInsertTest
FROM '....\test.csv'
WITH
(
FIELDTERMINATOR ='\,',
ROWTERMINATOR ='\n',
KEEPNULLS
)
will result in the default column value being ignored and NULLs being inserted fro empty strings, i.e:
SELECT * FROM BulkInsertTest
will now give you:
id name
1 NULL
1 ""
1 ''
There does not seem to be a good reason to add KEEPNULLS this in your example, but I came across a similar problem just now, where KEEPNULLS was required in the BULK INSERT.
My solution was to define make the column [name] in the staging table BulkInsertTest NOT NULL but remember that the DEFAULT column value gets ignored and an empty string gets inserted instead.
See more here : Keep Nulls or UseDefault Values During Bulk Import (SQL Server)
Related
For example, I have a column with type int.
The raw data source has integer values, but the null values, instead of being empty (''), is 'NIL'
How would I handle those values when trying to Bulk Insert into MSSQL?
My code is
create table test (nid INT);
bulk insert test from #FILEPATH with (format="CSV", firstrow=2);
the first 5 rows of my .csv file looks like
1
2
3
NIL
7
You can replace the nil with " (empty string) directly in your data source file or insert the data into a staging table and transform it:
BULK INSERT staging_sample_data
FROM '\\data\sample_data.dat';
INSERT INTO [sample_data]
SELECT NULLIF(ColA, 'nil'), NULLIF(ColB, 'nil'),...
Of course if your field is for example a numeric, the staging table should have a string field. Then, you can do as Larnu offers: 'TRY_CONVERT(INT, ColA)'.
*Note: if there are default constraints you may need to check how to keep nulls
I have the following table:
CREATE TABLE tab (
id SERIAL PRIMARY KEY,
code TEXT NOT NULL,
data TEXT
)
In some cases, I'd like to insert a new row ensuring that the code column is generated by the id column. In other cases the code is provided by the user.
For example:
INSERT INTO tab(code, data) VALUES ('code: ' || id::TEXT, 'my data');
The expected result is something like:
id
code
data
1
code: 1
abc
2
code: 2
xyz
INSERT INTO tab(code, data) VALUES ('user.provided.code', 'my data');
The expected result is something like:
id
code
data
1
code: 1
abc
2
code: 2
xyz
3
user.provided.code
xyz
Is it possibile in one statement?
It sounds like you want to default the coder to something based on the id. Unfortunately, this doesn't work in Postgres:
create table tab (
id integer primary key generated always as identity,
code text not null default ('code '||id::text),
data text
);
One option is a single statement that does both an insert and update:
with i as (
insert into tab (code, data)
values ('', 'my data'
returning *
)
update tab
set code = 'code: ' || id::TEXT
where tab.id in (select i.id from i);
Another is to use a trigger that assigns the value.
Use INSERT INTO .. SELECT as follows:
INSERT INTO tab(code, data)
select 'code: ' || id::TEXT, 'my data' from tab;
Note: In newly added data(above insert), you are missing to add id column data or I think it is auto generated.
I am importing data into SQL Server table using bulk insert.
BULK INSERT MySampleDB.dbo.Sample
FROM ''' + #location + '''
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = ''","'',
ROWTERMINATOR = ''\n'',
TABLOCK
)
I need to do check condition like if the column value whether it is an integer. If it is not an integer then I need to skip that entire record from inserting.
use case when script to understand its an integer or not.
look this
In my thought, you can not do this in the bulk insert but you can this help of the following the steps in the below.
Step-1 : First of all, you can add a varchar data type column to your table so that you can map CSV column to this column.
Step-2 : You can update the int column through the TRY_CONVERT function
CREATE TABLE Bulk_InsertTable
(ColImport VARCHAR(100) , ColINT INT)
GO
CREATE VIEW Viev_Bulk_InsertTable
AS SELECT ColImportVarchar
FROM Bulk_InsertTable
GO
BULK INSERT Viev_Bulk_InsertTable FROM 'C:\Test.csv'
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
TABLOCK
)
UPDATE Bulk_InsertTable
SET ColINT = TRY_CONVERT(INT, ColImport);
+--------------+-----------+
| ColImport | ColINT |
+--------------+-----------+
| 669165933 | 669165933 |
| 871543967AAA | NULL |
| 871543967AAA | NULL |
| 514321792 | 514321792 |
| 115456712 | 115456712 |
+--------------+-----------+
I've a csv file in hdfs directory /user/bzhang/filefortable:
123,1
And I use the following to create an external table with presto in hive:
create table hive.testschema.au1 (count bigint, matched bigint) with (format='TEXTFILE', external_location='hdfs://192.168.0.115:9000/user/bzhang/filefortable');
But when I run select * from au1, I got
presto:testschema> select * from au1;
count | matched
-------+---------
NULL | NULL
I changed the comma to the TAB as the delimeter but it still returns NULL. But If I modify the csv as
123
with only 1 column, the select * from au1 gives me:
presto:testschema> select * from au1;
count | matched
-------+---------
123 | NULL
So maybe I'm wrong with the file format or anything else?
I suppose the field delimiter of the table is '\u0001'.
You can change the ',' to '\u0001' or change the field delimiter to ',' , and check your problem was solved
This seems like a trivial question. And it is. But I have googled for over a day now, and still no answer:
I wish to do a bulk insert where for a column whose datatype is varchar(100), I wish to insert an empty string. Not Null but empty. For example for the table:
create table temp(columnName varchar(100))
I wish to insert an empty string as the value:
BULK INSERT sandbox..temp FROM
'file.txt' WITH ( FIELDTERMINATOR = '|#', ROWTERMINATOR = '|:' );
And the file contents would be row1|:row2|:|:|:. So it contains 4 rows where last two rows are intended to be empty string. But they get inserted as NULL.
This question is not the same as the duplicate marked question: In a column, I wish to have the capacity to insert both: NULL and also empty-string. The answer's provided does only one of them but not both.
Well instead of inserting empty string explicitly like this why not let your table column have a default value of empty string and in your bulk insert don't pass any values for those columns. Something like
create table temp(columnName varchar(100) default '')