I have a comma delimited text file with the structure
field1 field2 field3 field4
1 2 3 4
I wrote the following script to bulk insert the text file, but I wanted to leave out column 3
create table test (field1 varchar(50),field2 varchar(50),field4 varchar(50))
go
bulk insert test
from 'c:\myFilePath'
with
(fieldterminator=',',
rowterminator='\n'
)
The insert worked fine, but the results of the insert made field4 look like
field3,field4, so the field 3 was actually just concatenated onto field4. The flat files I'm working with are several gigs and can't be easily modified. Is there a way to use bulk insert but have it ignore the columns that aren't declared in the create table statement?
The easiest way is to create a view that has just the columns you require.
Then bulk insert into that view.
Example:
create table people (name varchar(20) not null, dob date null, sex char(1) null)
--If you are importing only name from list of names in names.txt
create view vwNames as
select name from people
bulk insert 'names.txt'
You can use a format file to do this:
http://msdn.microsoft.com/en-gb/library/ms178129.aspx
http://msdn.microsoft.com/en-gb/library/ms179250.aspx
Or if you want a slightly cheekier way, just import it all and drop a column afterwards. ;)
you cant ignore a field while doing bulk insert , insted of doing that .. Load all 4 column and drop the colum which you dont want
create table test (field1 varchar(50),field2 varchar(50), field3 varchar(50),field4 varchar(50))
go
bulk insert test
from 'c:\myFilePath'
with
(fieldterminator=',',
rowterminator='\n'
)
ALTER TABLE test DROP column [field3]
You can create a temporary table and insert any data there. After that, you can do whatever you want with it.
CREATE TABLE #TmpTable(
[CategoryId] [int] IDENTITY(1,1) NOT NULL,
[ParentCategoryId] [int] NULL,
[CategoryTypeId] [int] NOT NULL,
[Name] [varchar](255) NOT NULL,
[Code] [varchar](255) NOT NULL,
[AlternateName] [varchar](900) NOT NULL
)
BULK INSERT [dbo].[#TmpTable]
FROM 'C:\tmp\Categories.csv'
WITH
(
FIELDTERMINATOR = ',', --CSV field delimiter
ROWTERMINATOR = '0x0a' --Use to shift the control to next row
)
-- original table insert
INSERT INTO [dbo].[Categories]
(
CategoryId,
ParentCategoryId,
CategoryTypeId,
Name,
Code,
AlternateName,
IsDeleted,
SystemCreated
)
SELECT
CategoryId,
ParentCategoryId,
CategoryTypeId,
Name,
Code,
AlternateName,
0, -- custom value missed by file
GETDATE() -- custom value missed by file
FROM #TmpTable
-- remove tmp table
DROP TABLE #TmpEventCategories
Related
I need to insert values into a table if a value with a matching ID does not exist like in this thread: SQL - Insert Where Not Exists
But I need to make sure if an other thread executes a query in exactly the same time, I won't get two the same rows.
This is my table:
CREATE TABLE [dbo].[Localizations]
(
[id] [int] IDENTITY(1,1) NOT NULL,
[name] [nvarchar](50) NOT NULL,
[regionId] [int] NOT NULL
) ON [PRIMARY]
This my current query which inserts a new localization row if a localization row with regionId = x doesn't exist (unfortunately it works incorrectly - now I have duplicates in my database):
-- firstly I execute something like that (my real queries are more complex):
DECLARE #id int = (SELECT [id] FROM [dbo].[Localizations] WHERE regionId = 1);
-- secondly I execute something like that (my real queries are more complex):
IF (#id IS NULL)
BEGIN
INSERT INTO [dbo].[Localizations]
VALUES ('Test', 1);
END
It caused that now I have MANY rows with the same regionId, I can't remove them now, they are used in different tables :( :( :( Because of that, I can't create the unique constraint on the regionId column because I have duplicates :( :(
Could you tell me if the following query doesn't create duplicates with the same regionId if many threads execute that query in the same time? I have read this thread:
SQL - Insert Where Not Exists
but I am not sure and I don't want to insert more duplicates :(
INSERT INTO [dbo].[Localizations] (name, regionId)
SELECT 'Test', 1
WHERE NOT EXISTS (SELECT *
FROM [dbo].[Localizations]
WHERE regionId = 1)
After you remove the duplicates and add a unique constraint you can change your batch to prevent sessions from attempting to insert duplicates like this:
BEGIN TRANSACTION;
DECLARE #id int = (SELECT [id] FROM [dbo].[Localizations] WITH (UPDLOCK,HOLDLOCK) WHERE regionId = 1);
-- secondly I execute something like that (my real queries are more complex):
IF (#id is null)
BEGIN
INSERT INTO [dbo].[Localizations] VALUES('Test', 1);
END
COMMIT TRANSACTION;
This will force the first query to take and hold an update lock on the row or empty range, which will ensure that the INSERT will succeed, and any other session running this code will block until the transaction is committed.
You already know the answer, you should remove duplicities and add unique constraint. Until then, your data are broken.
If you want just a patch for new data, you can create unique filtered index on regionId, where you filter on regionId > lastDuplicitValue. But if you do not care about duplicities you already have, why care about the new ones?
If I want to do something relatively complicated - something usually done by a stored procedure. Is it possible to make it automatic using a VIEW?
My specific case:
I want output table = input table A + some rows input table B. In a stored procedure, I can make a copy of table A and then INSERT INTO it, but it's not allowed in a view.
Simplified example:
input table is [test_album], and output table = input table + singer Prince.
--create test data
IF OBJECT_ID('[dbo].[test_album]', 'U') IS NOT NULL
DROP TABLE [dbo].[test_album]
CREATE TABLE [test_album] (
id int not null identity(1, 1) primary key,
singer VARCHAR(50) NULL,
album_title VARCHAR(100) NULL
)
INSERT INTO [test_album] (singer, album_title)
VALUES ('Adale', '19'),
('Michael Jaskson', 'Thriller')
--this can be executed as sql code or in stored proc
SELECT *
INTO [result_table]
FROM [test_album]
INSERT INTO [result_table] ([singer])
VALUES ('Prince')
select *
from [result_table]
--id singer album_title
--1 Adale 19
--2 Michael Jaskson Thriller
--3 Prince NULL
----as expected
But I can do this INSERT INTO inside a view.
Real-life case:
additional singers are in a table [extra_singers]
[test_album] may have many other columns (or schema may change) so it's ideal not to type all column names in the code.
--create test data
IF OBJECT_ID('[dbo].[test_album]', 'U') IS NOT NULL
DROP TABLE [dbo].[test_album]
IF OBJECT_ID('[dbo].[extra_singers]', 'U') IS NOT NULL
DROP TABLE [dbo].[extra_singers]
IF OBJECT_ID('[dbo].[result_table]', 'U') IS NOT NULL
DROP TABLE [dbo].[result_table]
CREATE TABLE [test_album] (
id int not null identity(1, 1) primary key,
singer VARCHAR(50) NULL,
album_title VARCHAR(100) NULL,
many_other_columns VARCHAR(100) NULL
)
INSERT INTO [test_album] (singer, album_title)
VALUES ('Adale', '19'),
('Michael Jaskson', 'Thriller')
CREATE TABLE [extra_singers] (
[id] int not null identity(1, 1) primary key,
[name] VARCHAR(50) NULL )
INSERT INTO [extra_singers] ([name])
VALUES ('Prince'),
('Taylor Swift')
--append [extra_singers] to [test_album]
--this can be executed as sql code or in stored proc
SELECT *
INTO [result_table]
FROM [test_album]
INSERT INTO [result_table] ([singer])
SELECT [name]
FROM [extra_singers]
Is there an alternative to this (that is automatic)?
any help's appreciated. Thank u-
a partial solution I can think of:
create view test_view as
select *
from [test_album]
union all
select 3 as id,
'Prince' as singer,
NULL as album_title
but you have to know all the column names in [test_album] and you can't let column [id] do auto-increment
So you may be misunderstanding what a view does, or what an insert is. A view is simply a wrapper around a single select query. It contains exactly one select statement, and nothing else. An insert permanently adds a row of data to a persisted table. The example you gave where you just union the row you want seems valid enough. And certainly if it's the same row you want every time, you would not want to be inserting (or even trying to insert) that row into the underlying table each time
This raises a couple questions though.
If you're always going to be unioning the same single row every time, why not jut add that row to the table?
If, lets say, you don't want that row in the underlying table, cool. But if it's always the same static values, why do you need to include it in the view? Can't it just be assumed it's there?
If it can't be assume to always be the same, you certainly don't want to be changing the VIEW body every time you need it to change. So if it is going to change and you don't want to insert it into the base table, maybe make a second table containing the values you want appended to the base table in the view. Then union the base table and the "extra values" table together instead of a single, hard coded row constructor.
Is there any way of how to prevent inserting data in specified columns in table and use only the default (constraint) values?
E.g. I have columns:
LogInsert (DF GETDATE())
LogUser (DF ORIGINAL_LOGIN())
both defined with DEFAULT constraint. I do not want to allow users to insert into those columns, but use default values here instead when inserting new row.
This should raise an error.
INSERT INTO T1
( C1
,C2
,LogInsert
,LogUser
)
VALUES ( 'A'
,'B'
,'20160101 10:53'
,'domain\user'
);
User should be able to perform the following script without error.
INSERT INTO T1
( C1, C2 )
VALUES ( 'A', 'B' );
You could always give your users a view to work against instead of a table. You can then either choose to hide the columns completely or (as here) make them computed so that they cannot insert a value into the column, via the view:
create table dbo._T1 (
ID int IDENTITY(1,1) not null,
Inserted datetime2 constraint DF__T1_Inserted DEFAULT (SYSDATETIME()) not null,
ABC varchar(10) not null,
constraint PK__T1 PRIMARY KEY (ID)
)
go
create view dbo.T1
with schemabinding
as
select
ID,
COALESCE(Inserted,SYSDATETIME()) as Inserted,
ABC
from dbo._T1
go
insert into dbo.T1 (ABC) values ('abc')
go
insert into dbo.T1 (ABC,Inserted) values ('def',SYSDATETIME())
Results:
(1 row(s) affected)
Msg 4406, Level 16, State 1, Line 19
Update or insert of view or function 'dbo.T1' failed because it contains a derived or constant field.
All of the users queries continue to just use T1. It just happens to be a view rather than a table.
In the above, the view uses COALESCE(Inserted,SYSDATETIME()). It doesn't really matter what's used here, and it doesn't need to match e.g. the default definition. All that's important is that some computation is performed on the Inserted column so that it becomes a read-only column in the view.
You can create a Check Constraint on the table, for example the below
CREATE TABLE [dbo].[T1]
(
[C1] VARCHAR(50)
,[LogInsert] DATETIME DEFAULT GETDATE()
,[LogUser] VARCHAR(500) DEFAULT ORIGINAL_LOGIN()
)
ALTER TABLE [T1] WITH CHECK ADD CONSTRAINT [CK_T1_LogInsert] CHECK ([LogInsert] = GETDATE())
ALTER TABLE [T1] WITH CHECK ADD CONSTRAINT [CK_T1_LogUser] CHECK ([LogUser] = ORIGINAL_LOGIN())
INSERT INTO [dbo].[T1] ([C1]) VALUES ('A')
INSERT INTO [dbo].[T1] ([C1]) VALUES ('B')
INSERT INTO [dbo].[T1] ([C1]) VALUES ('C')
SELECT * FROM [dbo].[T1]
--Will Fail
INSERT INTO [dbo].[T1] ([C1],[LogInsert]) VALUES ('D','2016-11-11 00:00')
INSERT INTO [dbo].[T1] ([C1],[LogUser]) VALUES ('D','Not Your UserName')
OR
You can force the user to only insert using a stored procedure and not allow that as a parameter, this can be done with a table variable too for bulk inserts.
A bit of an odd requirement this one.
There is a column on a database (SQL) that is nullable.
I have a constrain in place that populates the column is null/default is provided and it populates from a sequence.
Is it possible to put a constraint in place that ignores any data provided by the insert statement and always puts in the next sequence value?
my current table/constraint is:
CREATE TABLE [dbo].[testmembership](
[id] [int] NOT NULL,
[name] [nvarchar](50) NOT NULL,
[membershipno] [nvarchar](50) NULL
) ON [PRIMARY]
alter table testmembership
add constraint DF_mytblid
default
'PREFIX-'+cast((next value for membershipseq) as nvarchar(50))
for membershipno
If I do the following:
insert into testmembership (id,name,membershipno) values (12,'test',default)
it yeilds the correct sequence generated value.
However, I want it to still have that value from the sequence even if i call this:
insert into testmembership (id,name,membershipno) values (12,'test','ignoreme')
I don't think you can do what you want using a constraint, but you can define a trigger that replaces the normal insert behaviour using the instead of option:
Something like this might work:
create trigger tr on testmembership instead of insert as
insert testmembership (id, name, membershipno)
select id, name, 'PREFIX-' + cast((next value for membershipseq) as nvarchar(50))
from inserted;
I have one table, which has a filed. When I insert in that table with the Empty value like '' or null, it should get converted to 'DUMMY-VALUE'.
--Have one table;
CREATE TABLE TEST ( FIELD1 VARCHAR2(50) );
--When I insert ''
INSERT INTO TEST(FIELD1) VALUES('');
SELECT * FROM TEST
--Expected Result 'DUMMY-VALUE' but not NULL
I can apply NVL('','DUMMY-VALUE') in INSERT statement but I am allowed to change CREATE statement only.
For Now, I am handing this with TRIGGER but , wondering if there is any alternative in 11g however I know about DEFAULT ON NULL for oracle 12c.
You can do like this:
create table TEST (FIELD1 VARCHAR2(50) default 'DUMMY-VALUE' );
and when you want to insert
you should insert without that field if the values is NULL or empty
Try this:
CREATE TABLE TEST (FIELD1 VARCHAR2(50) DEFAULT 'DUMMY-VALUE');
then use the DEFAULT keyword in the INSERT statement:
INSERT INTO TEST (FIELD1) VALUES (DEFAULT);
SQLFiddle here
Share and enjoy.