Reading data from CSV file and insert to SQL using vb.net - sql

Hi i have test folder in that daily we copy one csv file we dont use any code to copy file we just drag and drop from local. CSV file have 11 columns but i want only 3 columns data in sql. so i created 3 columns in sql. My aim is to read file from folder and insert those 3 columns data to sql. I will run the task daily using task scheduler if file found in folder it need to import data to sql

This will do it for you. Simply choose the fields (columns) you want to load.
Protected Sub uploadButton_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles uploadButton.Click
' declare CsvDataReader object which will act as a source for data for SqlBulkCopy
Using csvData = New CsvDataReader(new StreamReader(fileUpload.PostedFile.InputStream, True))
' will read in first record as a header row and
' name columns based on the values in the header row
csvData.Settings.HasHeaders = True
' must define data types to use while parsing data
csvData.Columns.Add("varchar") ' First
csvData.Columns.Add("varchar") ' Last
csvData.Columns.Add("datetime") ' Date
csvData.Columns.Add("money") ' Amount
' declare SqlBulkCopy object which will do the work of bringing in data from
' CsvDataReader object, connecting to SQL Server, and handling all mapping
' of source data to destination table.
Using bulkCopy = New SqlBulkCopy("Data Source=.;Initial Catalog=Test;User ID=sa;Password=")
' set the name of the destination table that data will be inserted into.
' table must already exist.
bulkCopy.DestinationTableName = "Customer"
' mappings required because we're skipping the customer_id column
' and letting SQL Server handle auto incrementing of primary key.
' mappings not required if order of columns is exactly the same
' as destination table definition. here we use source column names that
' are defined in header row in file.
bulkCopy.ColumnMappings.Add("First", "first_name") ' map First to first_name
bulkCopy.ColumnMappings.Add("Last", "last_name") ' map Last to last_name
bulkCopy.ColumnMappings.Add("Date", "first_sale") ' map Date to first_sale
bulkCopy.ColumnMappings.Add("Amount", "sale_amount") ' map Amount to sale_amount
' call WriteToServer which starts import
bulkCopy.WriteToServer(csvData)
End Using ' dispose of SqlBulkCopy object
End Using ' dispose of CsvDataReader object
End Sub ' end uploadButton_Click

Related

Query all tables in a sql data base with Python

I am using python with sqlite3, and I have problem in reading out tables in a data base created by myself.
The data base is created using the below code snippet
con = sqlite3.connect(folderdir + dbFileName)
#Create a cursor
c = con.cursor()
#Below code creates the data base. I later need to extract all tablename in
my data base
c.execute('CREATE TABLE IF NOT EXISTS ' + tablename + ' (Bord text, Date
text, Time text, ID integer, Menu text, Qty integer, Price real,
Time_kitchen text, TableActive integer)')
c.execute('INSERT INTO ' + tablename + ''' (Bord, Date, Time, TableActive)
VALUES(?,?,?,?)''',
(tablename, date, tabletime, 1))
con.commit()
con.close()
If above code is called many times with different tablename variables, I get a database.db file containing many tablenames. My problem is, even using the sqlite_master, I cannot read out all the tablenames. I use the below code to read out
import sqlite3
#I have verified that I can connect successfully to the database file
con = sqlite3.connect(dbFilePath + "/" + dbFileName + ".db")
c = con.cursor()
c.execute("SELECT ALL Name FROM sqlite_master")
rows = c.fetchall()
for row in rows:
print(row)
The print returned nothing although I know for sure that I have a database with multiple tablenames in. What am I doing wrong?
When the file whose name is given to connect() does not exist, SQLite will happily create a new, empty database for you.
Ensure that the file name is correct; and it should not be a relative file name to avoid a dependency on the current directory.

Alter Table Add Column - Record too large error VBA SQL

I am trying to add a column to an existing Access 2007 database and name it the value in a predetermined cell ("B3" in this case).
Sub InsertField()
Dim nfield As String, ntable As String, wsr As Worksheet, objAccess As Object
Set wsr = Sheets("Sheet1")
wsr.Select
ntable = "MyTable"
nfield = wsr.Range("B3").Value
Set objAccess = New Access.Application
' open access database
Call objAccess.OpenCurrentDatabase( _
"C:\Users\user\Desktop\MyDatabase.accdb")
' add field
objAccess.CurrentProject.Connection.Execute ("ALTER TABLE " + ntable + " ADD COLUMN " + nfield + " CHAR")
End Sub
This code works on an empty test Access database, but when I run it on my production database (~330MB) I get the following error on the "ALTER TABLE" line:
Run-time Error '-2147467259 (80004005)': Record is too large.
References are identical between the databases.
This code worked on the production database for a time so I feel like I hit some limit, but I cannot find any limit in the specifications that I meet or exceed.
Thanks in advance for the help!
Do you know if you have UnicodeCompression turned on for the table? With that property on there is a limit of 4000 bytes per row on the table. Adding that CHAR column might extend an existing row past that limit, while in any empty database none of the rows will have gone over that limit.

Query to return field names in a table within Microsoft Access

I thought there was a way you can do this in ms access whereby I can run a query on a table called 'Employees' for example and it will return the field names i.e. EmpID, FirstName, Surname titles,not the values of the field e.g. 13, john, doe.
Is this possible in ms access?
Regards,
forestgump
You will be needing a Recordset object to play with this. Not sure how you intent to use. But this code will print it to the immediate window.
For more info on how to use Recordset - http://www.utteraccess.com/wiki/index.php/Recordsets_for_Beginners
Public Sub printFieldName(inputTable As String)
'Takes a TableName as input and prints the field names
' of that Table.
Dim tmpRS As DAO.Recordset
Dim fCtr As Long
Set tmpRS = CurrentDB.OpenRecordset("SELECT * FROM " & inputTable)
For fCtr = 0 To tmpRS.Fields.Count - 1
Debug.Print tmpRS.Fields(fCtr).Name
Next
Set tmpRS = Nothing
End Sub
Usage would be,
printFieldName "TransactionTable"
FirstName
LastName
manTeam
probEnd
department
If you just want a list, open the table, select the first line, copy it and paste it into Excel. The field names and the first record will be pasted. Delete the first record. Copy the row with the field names and paste, transpose to get a list. Obviously not usable if you need the info dynamically but helpful if you need a list to work with.

Exporting SQL table to Excel - column names are not exporting, only the data

After I connect to SQL Management Studio via Excel VBA, I have the following code (PART OF IT) that pulls the data from a table in SQL.
With BalanceSheet
' Assign the Connection object.
.ActiveConnection = cnPubs
' Extract the required records.
.Open "select * from Analytics.dbo.BalSheetKeyLineItemsFinal"
Sheet1.Range("A13").CopyFromRecordset BalanceSheet
' Tidy up
.Close
End With
HOWEVER, the column names from the table do not show up. Is there a way I can get the column names to be included when my data is exported to Excel??
Thanks!
A recordset contains the data you have queried against, so this is what you would expect to happen. Since you're using .Open and .ActiveConnection, I assume you're using ADO: In which case, you can get the field names from the recordset object's Fields collection:
For i = 0 to .Fields.Count - 1
Sheet1.Range("A12").Offset(, i) = .Fields(i).Name
Next
This will write the field titles across row 12 of your Sheet1.

How can i copy data table records of different field name based on mapping list evaluating condition on source data table?

I have a Source Data table of type System.Data.DataTable from which i have to generate Destination Data Table of mapped column(Consider same SqlDBType for mapped columns.)
I have a list of MappingInfo class in which each Source Datatable Column mapped with New Column Name which will be in destination data table.
Public Class MappingInfo
Public Property SourceFieldName As String
Public Property DestinationFieldName As String
End Class
I have to evaluate a condition in source datatable to allow row data copy in destination table.
I did this using following code snippet:
''Prepare destination table.
For Each oMapping In oMappingInfo
DestinationDataTable.Columns.Add( _
New DataColumn(oMapping.DestinationFieldName))
Next
For Each oRow In SourceDataTable.Rows ''Copy data.
If oRow("IsActive") Then
oDataRow = DestinationDataTable.NewRow
For Each oMapping In oMappingInfo
oDataRow(oMapping.DestinationFieldName) = _
oRow(oMapping.SourceFieldName)
Next
DestinationDataTable.Rows.Add(oDataRow)
End If
Next
The main drawback is that here i have minimum 40k records in source datatable and data is not possible to fetch from database as all changes with data committed only when user save his work. The generated destination table is been assigned as data source to grid control and to report for preview.
How can i achieve this efficiently using Linq or do anyone please suggest me best way to achieve this requirement.
I've not tried this, so I can't say for sure that it'll be faster, but it seems to me that you'd get much better speed using something like the following:
Dim l_destinationTable As DataTable
' This creates a copy of the structure and content
l_destinationTable = SourceTable.Copy()
For Each l_column As DataColumn in l_destinationTable.Columns
Dim l_columnMap = oMappingInfo.FirstOrDefault( _
Function (c) c.SourceFieldName = l_column.ColumnName )
If l_columnMap IsNot Nothing Then
' Rename the column if it is mapped
l_column.ColumnName = l_columnMap.DestinationFieldName
Else
' Drop the column if it is not mapped
l_destinationTable.Columns.Remove( l_column )
End If
Next
NOTE: This method will fail if an unmapped column is part of a relationship or another column's expression depends on this column. Also, if you are swapping the name of two columns (for example, A will be named B and B will be named A) then you will get an exception as two columns may not have the same name at the same time.