I am trying to import a csv file to insert data into an existing table on my database. I go through the wizard and when it comes to select source tables and views for the destination, there are none to choose from. It just thinks I am trying to create a new table.
Any suggestions? Thanks!
Skip the wizard and use just BULK INSERT, here is an example:
BULK
INSERT CSVTest
FROM 'c:\csvtest.txt'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
Full example : SQL SERVER – Import CSV File Into SQL Server Using Bulk Insert – Load Comma Delimited File Into SQL Server
Cyril's answer is along the right track. However, a true CSV-compliant solution now exists since SQL Server 2017 (14.x) CTP 1.1.
Use this
BULK INSERT destinationtable
FROM 'filepath'
WITH
(
FORMAT = 'CSV'
)
GO
There is an issue though. If your data uses NULL to indicate nulls, then you'll need to remove them as MSSQLSMS will not accept NULL as a valid NULL value. Do a search/replace for ,NULL to ,.
For example (4 columns):
1,NULL,test,"Escaped text with comma, this works"
must be formatted like this:
1,,test,"Escaped text with comma, this works"
See SQL won't insert null values with BULK INSERT for information on NULL insertion problems.
You can go to https://learn.microsoft.com/en-us/sql/t-sql/statements/bulk-insert-transact-sql?view=sql-server-2017 for more information.
We do have multiple options:
using dts wizard
by programming
to know dts path or code in c# go through it
http://sqlcopy.blogspot.in/2012/07/bulk-sql-to-sql-sql-to-csv-csv-to-sql.html (dead link)
Related
I am trying to look for a way to import excel data into a temporary table without using OLEDB since I don't have OLEDB installed in SQL Server and I can not install it due to security restrictions.
I am aware of below mentioned ways of doing it but this is not going to help me
Begin Tran
If OBJECT_ID('tempdb..#tblUserImport') IS NOT NULL
Begin
Drop table #tblUserImport
end
Create Table [dbo].[#tblUserImport]
(
id nvarchar(max) NULL,
Name nvarchar(max) NULL,
Job_Title nvarchar(max) NULL,
Work_Email nvarchar(max) NULL
)
INSERT INTO [dbo].[#tblUserImport]
SELECT id, Name, Job Title, Work Email
FROM
OPENROWSET('Microsoft.ACE.OLEDB.12.0','Excel12.0;HDR=YES;Database=C:\Users\Desktop\Book2.xlsx', 'SELECT * FROM [Sheet1$]');
select * from [#tblUserImport]
Rollback Tran
I will get the below mentioned error if I execute the openrowset.
The OLE DB provider "Microsoft.ACE.OLEDB.12.0" has not been registered.
Is it possible to achieve it using Stored Procedure or any other way?
Here are 3 options:
1.Do the import from a computer you have admin rights on
It sounds like you don't have the ability to install OLE or ODBC data providers on the SQL Server machine. But you don't have to run the import from the same machine. As long as you have valid credentials and a working network path to your SQL server, you can run the import from any computer. So you could install the Microsoft ACE OLEDB 12.0 data provider driver on another PC along with SQL Server Management Studio, copy the Excel file there, and then do the import through the wizard.
Try an alternate data provider driver
There are alternate data provider drivers for Excel sources that may already be installed in your environment. E.g. the Jet OLEDB 4.0 driver mentioned at https://www.connectionstrings.com/excel/. Connection string:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\MyExcel.xls;
Extended Properties="Excel 8.0;HDR=Yes;IMEX=1";
Use Excel formulas to build INSERT statements
You can use the venerable Excel formula trick alluded to in RBell's answer: Assuming the fields you specified in your question are in A1 through D1, add this formula to cell E1:
="INSERT INTO #tblUserImport VALUES ('"&SUBSTITUTE(A1, "'", "''")&"', '"&SUBSTITUTE(B1, "'", "''")&"', '"&SUBSTITUTE(C1, "'", "''")&"', '"&SUBSTITUTE(D1, "'", "''")&"');"
You then copy this formula down through all rows of your data. Excel will automatically alter the cell references for each row. Note that the SUBSTITUTE() function handles single-quotes in the data that might otherwise break the SQL syntax. Then you simply copy and paste the resulting text out into your SQL window, and run it.
super mcguyver way using table variables. use the concat function on your excel sheet to prepare the data for the insert part.
declare #temp1 table (id int identity(1,1),name varchar(255),job_title varchar(255),work_title varchar(255),work_email varchar(255));
insert into #temp1 (name,job_title,work_title,work_email)
values
('John','Electrician','level 3 lightning wizard','john#ex.com'),
('amy','Java Developer','Cyber Coffee Slinger','amy#ex.com');
Its quite a bit of work , but feasible
I am assuming you have read-only access to prod db with temp table creation and insert into temp table statements.
No access to sql wizards or any elevated privileges.
Below 4 steps we have to preform in our dev sql box. Then generate the create #temptable script and auto generated insert statements and run them in production temp database
save the excel as csv file to dev machine
create a #temptable/ ##temptable ( because usually developers wont have create table permissions in production database )
Note : if the table has more columns use wizard to generate the auto crate table statement.
How ?
Answer is : Right click on Database --> Task --> Import Data --> on sql server import and export wizard, select the data source as flat file source and browse the csv file what you created in step1. --> Next --> on the choose a destination select destination as sql server native client 11.0 --> next --> on select source table and views screen choose edit mappings --> on column mappings screen click edit sql button which gives auto generated create table statement.
Copy the auto generated script --> edit the name to ##TempTable_CSV --> Create the temp table.
Now table is ready. For data ,
Use Bulk insert by giving the csv path
BULK INSERT ##TempTable_CSV
FROM 'C:\Users<username>\Downloads\temp.csv'
WITH ( FIRSTROW = 2, FORMAT = 'CSV');
To auto generate the insert statements from the temp table ( for example assuming the temp table has id , varchar columns )
select 'insert ##TempTable_CSV values ('+Cast(Id as varchar(5))+','''+SomeVarcharColumn+''') from ##TempTable_CSV
which will generate all the insert into statements
Note : Make sure to handle null values
Finally you can copy the create temp table script from step 2 + auto generated insert statements from step 4 , run it in prod temp db.
Enjoy :-)
I need to export a table, that contains a XML column (this xml my contain any special characters, so I can not use them as column delimiters) into a text file.
I am using SQL Server 2014. The XML column may contain special characters like #, |, ,, ?, tab, <cr> many things that could be used as delimiters.
I want to export the whole table. The XML is not structured internally. The max length of the column is around 6000 characters. The table has around 700k rows. And the destination is the same table in a SQL Server 2012 (lower version than the origin), they are in different networks.
I am trying to export it as a .txt file with || as column delimiter. But when I try to import this file into the destination table, it says that the text was truncated and could not be imported.
What's the best way I can do this?
All your answers you gave in your comments give me the idea that you are going the wrong way... The best approach should be linked servers:
Read here: https://msdn.microsoft.com/en-us/library/ff772782.aspx?f=255&MSPPError=-2147217396
Further information here: https://msdn.microsoft.com/en-us/library/ms190479.aspx?f=255&MSPPError=-2147217396
Try this in your SS2014
USE [master]
GO
EXEC master.dbo.sp_addlinkedserver
#server = N'YourLowerServer',
#srvproduct=N'SQL Server' ;
GO
This you need to get access:
EXEC master.dbo.sp_addlinkedsrvlogin
#rmtsrvname = N'YourLowerServer',
#locallogin = NULL ,
#useself = N'True' ;
GO
If this is done you can use the INSERT INTO from one server directly to the other server. Try this in your SS2014:
INSERT INTO YourLowerServer.YourDatabase.dbo.TableName(col1,col2,...)
SELECT col1,col2,... FROM dbo.TableName
If you want to get rid of your linked server after this operation use sp_dropserver (read here: https://msdn.microsoft.com/en-us/library/ms174310.aspx)
Hope this helps...
Im trying to run the following to import a large volume of sales data in a text file into a database. When i run the following i get the error: "Invalid object name 'PetDatabase.Sales'
BULK INSERT PetDatabase.Sales
FROM 'C:\Temp\P1.txt'
WITH
(
FORMATFILE = 'C:\Temp\PetSales.Fmt'
);
Can anyone see whats causing my problem? I do have the tables within a folder; however, when i tried PetsDatabase.Tables.Sales it made no difference.
Ignore this answer. It was written when the question was tagged with mysql. Leaving the answer here to keep the comments.
--
Try using LOAD DATA INFILE instead.
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Make sure PetDatabase.Sales exists in your text file.
Swap for whichever row and field terminator delimiters you're using. Here I'm using delimiters from a comma separated file
BULK INSERT PetDatabase
FROM 'c:\temp\p1.txt'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
--Check the content of the table.
SELECT *
FROM PetDatabase
GO
--Drop the table to clean up database.
SELECT *
FROM PetDatabase
GO
Also, make sure the following doesn't apply to you:
If a SQL Server user is logged in using Windows Authentication, the user can read only the files accessible to the user account, independent of the security profile of the SQL Server process.
When executing the BULK INSERT statement by using sqlcmd or osql, from one computer, inserting data into SQL Server on a second computer, and specifying a data_file on third computer by using a UNC path, you may receive a 4861 error.
To resolve this error, use SQL Server Authentication and specify a SQL Server login that uses the security profile of the SQL Server process account, or configure Windows to enable security account delegation.
Is PetDatabase is schema name or database name?
If it is database name, then include schema name also like this if your schema name is dbo.
PetDatabase.dbo.Sales
I have several files (they are XML but that's not important) that need to be inserted into an existing SQL table (i.e. I didn't design it.) The table looks like this.
ReportType
ID (int) <- identity
Name (varchar(32))
TransformXSLT (nvarchar(max))
Normally I would do:
INSERT INTO ReportType (Name, TransformXSLT)
VALUES ('template name', '<lots><of><xml><goes><here>...</lots>')
Is there any way to do:
INSERT INTO ReportType (Name, TransformXSLT)
VALUES ('template name', {filename})
I'm using SQL Server Management Studio and Eclipse+Maven to manage the files.
BULK INSERT or OPENROWSET(BULK…) are the usual options from T-SQL
After comment...
...FROM OPENROWSET(BULK N'C:\Text1.txt', SINGLE_BLOB);
and the "Bulk Exporting or Importing SQLXML Documents" section here
Sorry, I've not actually tried this but MSDN says you can
Have you tried using the SQL Server Import and Export Wizard?
Go into SQL Server Management Studio. In Object Explorer, right click the database then Tasks > Import Data....
This will let you import data as a one off exercise, or let you save the resulting SSIS package and re-run it.
Give it a go.
We have a SQL Server table for user settings. Originally the settings were domain objects which had been serialized as XML into the table but we recently begun serializing them as binary.
However, as part of our deployment process we statically pre-populate the table with predefined settings for our users. Originally, this was as simple as copying the XML from a customized database and pasting it into an INSERT statement that was ran after the database was built. However, since we've moved to storing the settings as binary data we can't get this to work.
How can we extract binary data from a varbinary column in SQL Server and paste it into a static INSERT script? We only want to use SQL for this, we don't want to use any utilities.
Thanks in advance,
Jeremy
You may find it easier to store a template value in a config table somewhere, then read it into a variable and use that variable to fill your inserts:
DECLARE #v varbinary(1000)
SELECT #v = templatesettings from configtable
INSERT INTO usertable VALUES(name, #v, ....)
From SQL Server 2008 onwards you can use Tasks > Generate Scripts and choose to include data. That gives you INSERT statements for all rows in a table which you can modify as needed.
Here's the steps for SQL 2008. Note that the "Script Data" option in SQL 2008 R2 is called "Types of data to script" instead of "Script Data".
I presume you're OK with utilities like Query Analyzer/Mangement Studio?
You can just copy and paste the binary value returned by your select statement (make sure that you are returning sufficient data), and prefix it with "0x" in your script.
If I understand you correctly, you want to generate a static script from your data. If so, consider performing a query on the old data that concatenates strings to form the SQL statements you'll want in the script.
First, figure out what you want the scripted result to look like. Note that you'll need to think of the values you're inserting as constants. For example:
INSERT INTO NewTable VALUES 'value1', 'value2'
Now, create a query for the old data that just gets the values you'll want to move, like this:
SELECT value1, value2
FROM OldTable
Finally, update your query's SELECT statement to produce a single concatenated string in the form of the output you previous defined:
SELECT 'INSERT INTO NewTable VALUES ''' + value1 + ''', ''' + value2 + ''''
FROM OldTable
It's a convoluted way to do business, but it gets the job done. You'll need a close attention to detail. It will allow a small (but confusing) query to quickly output very large numbers of static DML statements.
David M's suggestion of using the 0x prefixing works but i had to add an extra 0 at the end of varbinary data that i was trying to insert.
See the stackoverflow entry below to see the issue with additional 0 that gets added when converting to varbinary or saving to varbinary column
Insert hex string value to sql server image field is appending extra 0