Create table, and import data from csv or txt file - sql

I have a CSV file that contains stock quotes. I am new when it comes to SQL, but I have done a lot of research and come up with a code that I thought should work. But it doesn't. I get errors all the way....
USE ShakeOut
GO
CREATE TABLE CSVTest1
(Ticker varchar(10),
dateval smalldatetime),
timevale time(),
Openval varchar(10),
Highval varchar(10),
Lowval varchar(10),
Closeval varchar(10),
Volume varchar(10),
)
GO
BULK
INSERT CSVTest1
FROM 'c:\TEST.txt'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
--Check the content of the table.
SELECT *
FROM CSVTest1
GO
--Drop the table to clean up database.
DROP TABLE CSVTest1
GO
My CSV file has timevalue as 03:15:00 PM, and I'm not sure how to set that up in the table. The other values I think are aproxmately right, here's a sample of my csv file:
5/1/2009,9:30:00 AM,18.21,18.45,18.21,18.32,32163
5/1/2009,9:35:00 AM,18.33,18.34,18.27,18.29,36951
5/1/2009,9:40:00 AM,18.29,18.38,18.25,18.37,53198
5/1/2009,9:45:00 AM,18.38,18.4,18.28,18.285,49491
And here is my error messages in SQL Management Studio:
Msg 102, Level 15, State 1, Line 4 Incorrect syntax near ','. Msg 208,
Level 16, State 82, Line 3 Invalid object name 'CSVTest1'. Msg 208,
Level 16, State 1, Line 3 Invalid object name 'CSVTest1'. Msg 3701,
Level 11, State 5, Line 3 Cannot drop the table 'CSVTest1', because it
does not exist or you do not have permission.
I would really appreciate help here, my head is about to explode after all these hours without any progression. I've tried MySQL too, didn't work there either.
As I'm new, I might need it explained to the details.

It appears you have an extraneous comma in the CREATE TABLE statement. There is a comma following the final column prior to the closing paren. Perhaps it is valid in some implementations, but you might try removing it. Change it to:
Volume varchar(10)
Ah - and it appears there is an extraneous closing parent in the second column definition. Change it to:
dateval smalldatetime,
And the time column:
timevale time,
Ultimately, it appears you should probably just try to get the CREATE TABLE statement syntax correct, then start adding the other parts.

There is no need for a comma after the last column definition: Volume varchar(10),.
I assume timevale should be timeval.
time() should just be time.
Also, I'm probably being picky but you have capitalised the first letter of all the column names except the first two - won't cause an error but probably better to have a consistent naming convention. I would capitalise the 'v' in val and write the whole word too.
The CSV data needs revising too - you need to specify EVERY column, even if it is null. See my example data (the new lines at the end of each row are for illustration purposes only).
1234567890,2012-08-25,22:15,anytext,ornum,for,varchar,columns <-new line
abcd123456,2010-05-20,00:01,anything,in,these,varchar,columns <-new line
abcd123456,2010-05-20,00:01,anything,in,,,columns <-new line
This works:
CREATE TABLE CSVTest1 (
Ticker varchar(10) NULL,
DateValue smalldatetime NULL,
TimeValue time NULL,
OpenValue varchar(10) NULL,
HighValue varchar(10) NULL,
LowValue varchar(10) NULL,
CloseValue varchar(10) NULL,
Volume varchar(10) NULL)
GO
BULK INSERT CSVTest1
FROM 'C:\TEST.txt'
WITH (FIELDTERMINATOR = ',', ROWTERMINATOR = '\n')
GO
Your CSV file needs to have a new line for each record you want to insert, as specified by the ROWTERMINATOR = '\n' and a comma between each field as specified by FIELDTERMINATOR = ','.
EDIT:
By the way if you are using SQL Server Management Studio (SSMS) you can create the table through the user interface and then:
Right click on the table
Script Table as
CREATE To
New Query Editor Window

Related

IN operator with SELECT clause fails

The resultant error is:
Msg 207, Level 16, State 1, Line 9
Invalid column name 'Email'.
Code:
-- Bulk insert data from csv file into server temp table
BULK INSERT vwTemporaryIT_USE_ONLY_Import FROM 'C:\Bulk\b_email.csv'
WITH (
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
Go
-- Set the flag in db for all records imported from csv
UPDATE [APTIFY].[dbo].[Person]
SET
[IT_Use_Only] = 1
WHERE
[Email] IN
(Select [Email] From vwTemporaryIT_USE_ONLY_Import)
Go
I can see that the vwTemporaryIT_USE_ONLY_Import table is being populated with the data from the CSV fine, but is seems the following statement is failing for some reason:
WHERE
[Email] IN
(Select [Email] From vwTemporaryIT_USE_ONLY_Import)
I am certainly not an expert at this and I may not have setup the table or view correctly, as I recently added the Email column to both. But they have matching datatype of nvchar(100) not null. I have also tried it as null. I'm not even sure if IN handles nvchar such is the level of my SQL expertise. Any clues what I'm doing wrong?
Actually no! In the Person table it was called Email1. I have changed the code to:
WHERE [Email1]
IN
(Select [Email] From wTemporaryIT_USE_ONLY_Import)
...and now it works fine. Thanks for your help!!

SQL Server Insert Statement From a CSV File

I'm trying to follow the example on this page:
http://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/
The database is on another machine and I'm connecting to it using SQL Server Management Studio 2012.
Here's the insert statement
BULK
INSERT CSVTest
FROM '\\BPWKS\network share\csvtest.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
When I go to check the content of the table, I get the following error:
Error: Msg 208, Level 16, State 82, Line 1
Invalid object name 'CSVTest'.
Table Definition
Create Table CSVTest
(
ID INT,
FirstName VARCHAR(40),
LastName VARCHAR(40),
BirthDate SMALLDATETIME
)
GO
I've also tried importing the local file at 'C:\csvtest.csv'
Resolution appears to be
Missing Table initially
Missing file or permissions to access
file.

SQL Server BULK INSERT error

This is my source data in CSV format:
4,23,2AY5623,7235623
4,23,2GP1207,1451207
4,23,2GQ6689,4186689
Table:
CREATE TABLE [dbo].[Table1](
[idCodeLevel] [int] NOT NULL,
[idFirm] [int] NOT NULL,
[valCodeFrom] [varchar](15) NOT NULL,
[valCodeTo] [varchar](15) NOT NULL
) ON [PRIMARY]
This the code I am using to bulk import:
USE Test
GO
TRUNCATE TABLE Table1
GO
BULK INSERT Table1
FROM 'C:\Temp\test.csv'
WITH (
FIELDTERMINATOR = ',',
MAXERRORS=0,
ROWTERMINATOR = '\n'
)
GO
Error I am getting is:
Msg 4864, Level 16, State 1, Line 2
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 1 (idCodeLevel).
Can you please someone tell me why is it failing?
I googled and found out that I might have to use the format .fmt. But how can I convert a csv file to fmt. I have seen code to create fmt file from sql table.
Thanks a lot for your help!
Does the csv have a row at the top of field names? If so you'll need to add "FIRSTROW = 2" to your bulk statement. If not, try creating a new table that is all VARCHAR fields, then check the data: you probably have something strange in your data that you aren't expecting, like a non-printing character. Import as text and then try something like "SELECT ISNUMERIC([FIELD1]) FROM NEWTABLE".
use the sql import wizard to import data from external file.
Right click on database--->task--->import----> specify the flat file as source and select the destination server.
for more information please visit Import CSV data to SQL

OUPUT for insert from table variable gives: "The multi-part identifier "k.CustomerName" could not be bound."

I have seen many people here on stack overflow with this error message and all get it in another situation. I could not find my own situation among the already existing questions. So I hope someone can help me with this. I use SQL server 9 with SQL management studio 10.
--import the customers.
CREATE TABLE #AllCustomers(CustomerName NVARCHAR(100), CustomerNr NVARCHAR(16));
BULK INSERT #AllCustomers
FROM 'C:\allcustomers.txt'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
);
DECLARE #DefinitionId int;
--get the id of the definition for which I have to set values.
SELECT #DefinitionId = pkDefinitionId
FROM dbo.Definitions
WHERE Name = 'DEFINITION-OF-MY-ITEM';
DECLARE #TempA TABLE (CustomerName NVARCHAR(255), CustomerNr NVARCHAR(16));
--reduce the set of all customers to only the customer for whom I have to insert.
WITH MyView AS
(SELECT kciv.CustomerNr
FROM dbo.CustomerItems kciv
INNER JOIN dbo.DefinitionToItem civ ON civ.pkDefinitionToItemId = kciv.pkCustomerItemId
WHERE civ.fkDefinitionId = #DefinitionId)
INSERT INTO #TempA
SELECT k.CustomerName , k.CustomerNr
FROM #AllCustomers k
WHERE k.CustomerNr NOT IN (SELECT CustomerNr FROM MyView);
--used to store the generated primairy keys I need for creating relations.
DECLARE #ItemIds TABLE (CustomerName NVARCHAR(255), CustomerNr NVARCHAR(16), pkItemId int);
DECLARE #DefinitionToItemIds TABLE (CustomerName NVARCHAR(255), CustomerNr NVARCHAR(16), pkDefinitionToItemId int);
--insert the default values.
INSERT INTO dbo.Items
OUTPUT k.CustomerName, k.CustomerNr, inserted.pkItemId
INTO #ItemIds (CustomerName, CustomerNr, pkGenericValueId)
SELECT 2, 1, null, null, 1, null
FROM #TempA k;
--couple the values to the definition.
INSERT INTO dbo.DefinitionToItem
OUTPUT gvd.CustomerName, gvd.CustomerNr, inserted.pkDefinitionToItemId
INTO #DefinitionToItemIds
SELECT 1, 0, #DefinitionId, gvd.pkItemId
FROM #ItemIds gvd;
--couple the 'coupling' to the customers.
INSERT INTO dbo.CustomerItems
SELECT civd.pkDefinitionToItemId, civd.CustomerName, civd.CustomerNr
FROM #DefinitionToItemIds civd;
I get four errors when running the query, all on the two output lines near the end of the code sample.
Msg 4104, Level 16, State 1, Line 64 The multi-part identifier "k.CustomerName" could not be bound.
Msg 4104, Level 16, State 1, Line 64
The multi-part identifier "k.CustomerNr" could not be bound.
Msg 4104, Level 16, State 1, Line 69
The multi-part identifier "gvd.CustomerName" could not be bound.
Msg 4104, Level 16, State 1, Line 69
The multi-part identifier "gvd.CustomerNr" could not be bound.
I have checked for typos but couldn't find any (I might have introduced some here though while changing some of the names to remove the context). I can't find out why this is going wrong. I've looked at MSDN, but I can't find anything wrong.
Extra info:
The database schema is as follows:
The Items table contains "values" (pkItemId, bunch of other columns)
The Definition table contains "definitions" (pkDefinitionId, Name, bunch of other columns)
The DefinitionToItem table matches the values to definitions (pkDefinitionToItemId, fkDefinitionId, fkItemId)
The CustomerItems table links a customer to a DefinitionToItemId (pkDefinitionToItemId, CustomerName, CustomerNr).
What I need to achieve is to insert default values (i.e. "2, 1, null, null, 1, null" linked to definitino 'DEFINITION-OF-MY-ITEM') into the items database for a given set of customers. Some might already have a value for that definition and then I should skip them (hence the #TempA).
So I insert the value into Items, then insert the coupling between definition and items in DefinitionToItem and lastly couple the customer to the DefinitionToItem by inserting into the DefinitionToItem table.
If there if a better way to achieve this than through what I'm doing, then I'm open to suggestions.
I think the approach you are taking is complicating the scenario.
Firstly, the OUTPUT clause is not going to work the way you need it to work here, because you can only use columns inserted, and you want to use columns from the source table, but ones that does not get inserted. (CustomerName as an example).
There are two ways I suggest you can go about this:
First Approach. Change you query. Use OUTPUT, but output an id field. Then, after your first insert into ITEMS, join your new table with your source table. NOW you will have access to all fields and still a way to identify which records should be inserted.
Second approach. Drop your temp tables. Use a simple insert statement with a WHERE _ NOT IN clause. This takes away the complexity and still achieves the goal of not inserting duplicates, just on a closer level.
Use:
INSERT INTO dbo.Items
OUTPUT INSERTED.CustomerName, INSERTED.CustomerNr, INSERTED.pkItemId
INTO #ItemIds (CustomerName, CustomerNr, pkGenericValueId)
SELECT 2, 1, null, null, 1, null
FROM #TempA k;
INSERT INTO dbo.DefinitionToItem
OUTPUT INSERTED.CustomerName, INSERTED.CustomerNr, INSERTED.pkDefinitionToItemId
INTO #DefinitionToItemIds
SELECT 1, 0, #DefinitionId, gvd.pkItemId
FROM #ItemIds gvd;
You might need to change the column names, because they need to come from the table being inserted into rather than the column they were sourced from.
Your table #AllCustomers does not contain field CustomerName in this query
SELECT k.CustomerName , k.CustomerNr
FROM #AllCustomers k

SQL BULK INSERT FROM errors

I'm attempting to insert a CSV file into an Microsoft SQL Server Management Studio database like this:
BULK INSERT [dbo].[STUDY]
FROM 'C:\Documents and Settings\Adam\My Documents\SQL Server Management Studio\Projects\StudyTable.csv'
WITH
(
MAXERRORS = 0,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
But I am getting errors:
Msg 4863, Level 16, State 1, Line 2
Bulk load data conversion error (truncation) for row 1, column 9 (STATUS).
Msg 7399, Level 16, State 1, Line 2
The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error.
Msg 7330, Level 16, State 2, Line 2
Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
Unfortunately, I cannot provide the contents of StudyTable.csv, to protect the privacy of the company we're working for.
EDIT I can vouch for the validity of the csv file though. It was exported from OpenOffice's version of Excel, and I went through and made sure it was valid.
EDIT2
Here's a dummy version of the CSV file:
1234,,,1234,1234,,"asdf","asdf","Z","asd",7/1/2010 12:23,8/5/2010 13:36,9/4/2010 13:36,"(asdf,1661,#1234,F,T)","F",,,"F",,"68866",1234,1234,1234,"F"
Here's a create script for the STUDY table:
CREATE TABLE [dbo].[STUDY]
(
[STUDY_ID] INT IDENTITY(1,1) NOT NULL,
[PARENT_ID] INT,
[GROUP_ID] INT,
[WORKFLOW_NODE_ID] INT,
[STUDY_TEMPLATE_ID] INT,
[INSPECTION_PLAN_ID] INT,
[NAME] VARCHAR(255),
[DESCRIPTION] VARCHAR(4000),
[STATUS] VARCHAR,
[OLD_STATUS] VARCHAR,
[CREATED_ON] DATE,
[COMPLETED_ON] DATE,
[AUTHORIZED_ON] DATE,
[EVENTS] VARCHAR,
[NEEDS_REVIEW] CHAR,
[HAS_NOTES] CHAR,
[HAS_AUDITS] CHAR,
[STUDY_PART] CHAR,
[STUDY_TYPE] VARCHAR,
[EXTERNAL_REFERENCE] VARCHAR,
[CREATED_BY] INT,
[COMPLETED_BY] INT,
[AUTHORISED_BY] INT,
[ARCHIVED_CHILD_COMPLETE] CHAR
)
It sounds like the data in your STATUS column in the .csv file is longer than the definition for the field in your SQL Table.
Check the definition on that field and make sure the data you have in the .csv will fit (both length and type).
The first error is likely because a column is too big to fit the database. First thought is does the CSV file contain column headers?
The second error is very generic.
Do any of your fields contain a comma as part of their data - for example 37, High Street which isn't encased in quotes? I've come unstuck lots of times in the past because of "issues" in the data (the data is valid, just not quite what's expected). If your CSV file is a few thousand rows or more - what happens if you try importing just the first hundred (or ten) rows?