Import CSV file to SQLITE database via VBA - Access - sql

I have a project in Access VBA where I need to automate the import of the CSV file (MAX 15MB) into linked SQLITE db. All done on a local machine.
Something similar as for MS SQL (as written below) that would be achievable via query or VBA?
strConn = "DRIVER=ODBC Driver 13 for SQL Server;Server=" & serv & ";Database=" & databs & ";Uid=" & usern & ";Password=" & pwd & ";"
cn.Open strConn
Strsql1 = "BULK INSERT dbo.SEARCHUSRTABLE2 FROM '" & insertfile & "' WITH(FIELDTERMINATOR = ',',ROWTERMINATOR = '\n');"
Set rs = cn.Execute(Strsql)

Ok, we need a few things.
First up, we need from Access:
Browse to csv file – select
Import the csv file to a table.
Now, export to a SQLite.
But, we missing a few questions here.
First up, is the SQLite database the same database and in the same location all the time?
Have you installed the SQLite ODBC driver, and have you been able to link to SQLite from Access?
So, we kind of need the above all sorted out. Especially the connection to SQLite.
And it not clear if you plan/want to create a new table for each csv import, or you going to clear out a table, and re-fill?
Lets assume the following:
You have a linked table to SQLite working from Access.
You can click on this linked table in Access, and you can see/view and even edit the data FROM Access, but the database and table is of course a linked table to SQLite.
If all the above is working?
And ALL of the above so far takes ZERO code. So you “need” to get the above part working.
And we assume that each csv import is to create a new table in the SQLite database?
Then the code would look like this:
Paste this sub into a standard access code module.
Save it, and then with cursor anywhere in the code, hit f5 to run.
Sub CsvImportToSQL()
Dim strCsvFile As String
Dim f As Object
Set f = Application.FileDialog(3)
f.Filters.Clear
f.Filters.Add "Csv file", "*.csv"
If f.Show = False Then
' no file selected - quite
Exit Sub
End If
Dim strFromTable As String
strFromTable = f.SelectedItems(1)
Dim strTable As String
' get only name of file without extension for Access table name
strTable = Mid(strFromTable, InStrRev(strFromTable, "\") + 1)
strTable = Left(strTable, InStr(strTable, ".") - 1)
strTable = InputBox("Select table = " & strFromTable, "Inport to Access table", strTable)
If strTable = "" Then
' user hit cancel - exit
Exit Sub
End If
' transfer the table into access
DoCmd.TransferText acImportDelim, , strTable, strFromTable, True
' ok, now ask for table name to export to in SQLite
Dim strSQLiteTable As String
strSQLiteTable = strTable
strSQLiteTable = InputBox("Table name to export for SQLite", "SQL lite table name", strSQLiteTable)
If strSQLiteTable = "" Then
' user cancel - don't transfer - exit
Exit Sub
End If
' now transfer table to SQL site
Dim strCon As String
strCon = CurrentDb.TableDefs("Hotels").Connect
' (replace above Hotels with a KNOWN WORKING LINKED table to SQLite)
DoCmd.TransferDatabase acExport, "ODBC Database", strCon, acTable, strTable, strSQLiteTable
MsgBox "table exported to SQLITE"
End Sub

Related

Access VBA Export to New CSV Run-time Error 3011

I have code that exports a dynamic query to a csv. First I save the sql to an existing query, updating it's sql. Then I use that as the query defs, provide saved specs, and want to export to a dynamically constructed csv file name. The csv doesn't exist yet, so I am creating it. I thought doing a docmd.transfertext acExportDelim would create the file.
I end up getting the following error:
Run-time error '3011':
The Microsoft Access database engine could not find the object 'filename.csv'.
Make sure it is spelled correctly. If 'filename.csv' is not a local object,
check your network connection or contact the server administrator.
The "filename.csv" in the error, is exactly what I construct in the below code, and the name of the csv file I want to be created by this process.
Private Sub cmdExportList_Click()
Dim fsql As String
Dim f As Form
Dim RS As DAO.Recordset
Dim MyDate As String
Dim args As String
Set f = Forms!frmMyLists.Form.frmMyLists_SubResults.Form
args = Nz(Forms!frmMyLists.Form.cboMyList.Value, "00")
fsql = "SELECT * " & _
"FROM tblVFileImport "
fsql = fsql & "WHERE tblVFileImport.ListName = '" & args & "' "
fsql = fsql & "ORDER BY tblVFileImport.ID"
CurrentDb.QueryDefs("qryListExportSpecs").SQL = fsql
MyDate = Format(Now(), "yyyymmdd")
If IsDev = True Then
vFile = "C:\LocalPath\VFileData\ExportList\" & args & MyDate & ".csv"
Else
vFile = "\\Server\Share\ExportList\" & args & MyDate & ".csv"
End If
DoCmd.TransferText acExportDelim, "MyListExportSpecs", "qryListExportSpecs", vFile, True
Application.FollowHyperlink vFile
MsgBox "All Set! " & args & " tagged list is now open in Excel. Have Fun!", vbOKOnly, "Export Complete"
End Sub
Right now, I am in my dev copy (IsDev = True) so I'm just going to that local path. I tried creating a csv with the name first, but if I don't write anything to it (which I don't want to, because I'm wanting to transfer the text to it) then it goes away when I close it. I am wondering if it does that because it's 0 bytes. Does anyone know what I am doing wrong? Thanks!
Edit:
I found that if I tested by creating the csv before running the code, so it would already exist, the code was deleting it. I researched that and found the export specs can sometimes mess things up. I removed the export specs and now it works!
So this is :
DoCmd.TransferText acExportDelim, "", "qryListExportSpecs", vFile, True

How to insert data from Excel into a database

I'm looking for a solution to import data from an Excel file into a database (for example: MS Access file).
I can get the idea as well as the structure but because I'm new to something like this it is really hard to finish the work.
Below is my code, which should do these:
Select database
Select import files
Create connection using ADODB
I'm stuck here, using Insert statement to import but how? Because the values to import would be a value of a variable run from the very first cell of Excel files till the end of it.
Please ignore the comment because I'm using my native language to easier understand in future
Sub Import_Data()
'Khai bao bien lien quan den Database
Dim connection As ADODB.connection
Dim record As ADODB.Recordset
'Khai bao cau lenh Query
Dim SQLstr As String
'Khai bao connection string
Dim connectionstring As String
Dim dbstring As String
'Duong dan den file import
Dim fdlg As FileDialog
Application.ScreenUpdating = False
'Chon datafile
With Application.FileDialog(msoFileDialogFilePicker)
.AllowMultiSelect = False
.Filters.Clear
.Filters.Add "Access files", "*.accdb, *.mdb"
If .Show = True Then
datapath = .SelectedItems(1)
Else
Exit Sub
End If
End With
'Chon file import
Set fdlg = Application.FileDialog(msoFileDialogFilePicker)
With fdlg
.AllowMultiSelect = False
.Filters.Clear
.Filters.Add "Excel files", "*.xls,*.xlsx,*.xlsm"
If .Show = True Then
importstring = .SelectedItems(1)
Else
Exit Sub
End If
End With
'Connect to Database
Set connection = New ADODB.connection
With connection
.Provider = "Microsoft.ACE.OLEDB.12.0"
.connectionstring = "Data Source=" & datapath & ";"
.ConnectionTimeout = 30
.Open
If .State = adStateOpen Then
MsgBox "welcome " & Environ("Username") & " ket noi den database"
Else: Exit Sub
End If
End With
Dim a, c As Integer
Dim b, d As Integer
Dim ImpWb As Workbook
Dim ImpWs As Worksheet
Set ImpWb = Application.Workbooks.Open(importstring)
ImpWb.Application.Visible = True
Set ImpWs = ImpWb.Worksheets("Sheet1")
With ImpWs.UsedRange
a = .Rows.Count
b = .Columns.Count
For c = 2 To a
For d = 1 To b
SQLstr = "Insert into Test values(" & Cells(c, d).Value & ")"
connection.Execute SQLstr
Next d
Next c
End With
ImpWb.Close
connection.Close
Set ImpWs = Nothing
Set ImpWb = Nothing
Set connection = Nothing
End Sub
From the comments above it's become clearer now that the issue you face is how to build the SQL statements inside VBA. However, you are very much closer than you think.
This function below will build an insert statement for each row of data provided the columns in the Excel tab is exactly equal to and in exact order as the fields in the SQL table. It also will combine all the inserts into one statement so you can execute all the rows at once.
With ImpWs.UsedRange
a = .Rows.Count
b = .Columns.Count
For c = 2 To a
SQLstr = SQLstr & "INSERT INTO TEST VALUES("
For d = 1 To b
SQLstr = SQLStr & .Cells(c, d).Value & iif(d <> b, ",","")
Next d
SQLstr = SQLStr & ");"
Next c
End With
The other caveat is that assumes all fields are numerical. If you have some text fields you will have to add ' before and after each field, like ' & .Cells(c, d).Value & & '. If you know the columns that need text, you can modify the input with a Select Case on column number as needed.
Hi you recognize or discovered already the magic connection.execute function. Here is the music playing. All its need is a valid SQL string to insert. That's (nearly) all.
SQL Strings can be fired single by single or a bunch separated by a ";" at the end
INSERT INTO table1 (column1,column2 ,..)
VALUES
(value1,value2 ,...),
(value1,value2 ,...),
...
(value1,value2 ,...); --<<<<<<<<<<< see the ";"
In your case you has to define the affected columns in the first line and then just all the values for a line inside the braces. If this do not work - something is wrong with the SQL string. You can use access which has a nice SQL builder to create a sql template or maybe use a online sql tester to see if your sql is valid. For testing purposes i suggest to just add one line first. And another hint: The values has to be escaped ! So add first something trivial like "abc" or 1234 just to check out the principle function. Life datas might have things like this >"< inside or "," or other things which could be seen as sql statement. Which will end up like it will do nothing at all. So read the manual how access escaped such poisoned text fragments ;) And implement a function to avoid that. SQLITE has a nice tutorial and example page. SQLITE is nearly like access. To get a overview i would read there first and the M$ documentation a biut later. Have fun :)
There is also this approach using SQL
Sub test()
Dim c As ADODB.Connection
Dim s As String
Dim strSQL As String
s = "Provider=Microsoft.ACE.OLEDB.12.0;" & _
"Data Source=c:\…...\TestDBs\API_TEST.accdb;" & _
"Persist Security Info=False;"
Set c = New ADODB.Connection
c.ConnectionString = s
c.Open
' Excel workbook, sheet name test1
strSQL = "INSERT INTO tblDestination ( Firstname )" & _
"SELECT [test1$].FirstName " & _
"FROM [EXCEL 12.0;HDR=YES;IMEX=2;DATABASE=C:\Databases\csv\test1.xlsx].[test1$];"
c.Execute strSQL
c.Close
End Sub

Referencing a linked table

I'm having an issue with my databases. I have multiple access databases that share the same pool of users and passwords. They reference the table of users and passwords by a linked table to the 'master' database (the backend that holds the tables for users and passwords). I also have a table that stores the current users and the databases that they are logged into. The problem I'm encountering is that my logout method is not actually logging them out. This is what it is roughly:
'this code is run on click of exit button
Public Sub logout(UserName As String, database As String)
On Error Resume Next
Dim dbMine As DAO.database
Set dbMine = CurrentDb
Dim qr As String
qr = "DELETE * FROM tblCurrentUsers WHERE username = '" & UserName & "' AND Database = '" & database & "' ;"
'debug.print qr
dbMine.Execute qr
Application.Quit
End Sub
The problem is, the records don't seem to be deleting. Do I need to set my database object to the source table instead of referencing the linked table that exists in the database on which the code is run? If so, do I just reference that database by relative path?
Add dbFailOnError option when executing query to catch the error details. See what error you get; that could help you in resolving of your issue.
Public Sub logout(UserName As String, database As String)
On Error GoTo mError:
Dim dbMine As DAO.database
Set dbMine = CurrentDb
Dim qr As String
qr = "DELETE * FROM tblCurrentUsers WHERE username = '" & UserName & "' AND Database = '" & database & "' ;"
'debug.print qr
dbMine.Execute qr, dbFailOnError
Application.Quit
Exit Sub
mError:
MsgBox "Error: " & Err.Description
End Sub
Try using
Set dbMine = DBEngine.Workspaces(0).Databases(0)
instead of
Set dbMine = CurrentDb
I've had similar issues when using CurrentDb. According to http://msdn.microsoft.com/en-us/library/office/bb237861(v=office.12).aspx,
The CurrentDb method creates another instance of the current database, while the DBEngine.Workspaces(0).Databases(0) syntax refers to the open copy of the current database.
Perhaps there's some subtle difference in the way DAO or Access handles the "new instance of the current database".

Link multiple tables from one database

I have actually this little function working as supposed:
Function createAttached(strTable As String, strPath As String, strBaseTable As String) As Boolean
On Error GoTo CreateAttachedError
Dim tdf As TableDef
Dim strConnect As String
Dim fRetval As Boolean
Dim myDB As Database
DoCmd.SetWarnings False
Set myDB = CurrentDb
Set tdf = myDB.CreateTableDef(strTable)
With tdf
.Connect = ";DATABASE=" & strPath
.SourceTableName = strBaseTable
End With
myDB.TableDefs.Append tdf
myDB.TableDefs.Refresh
fRetval = True
DoCmd.SetWarnings True
CreateAttachedExit:
createAttached = fRetval
Exit Function
CreateAttachedError:
If Err = 3110 Then
Resume CreateAttachedExit
Else
If Err = 3011 Then
Resume Next
Else
If Err = 3012 Then
Set tdf = myDB.TableDefs(strTable)
tdf.Connect = ";DATABASE=" & strPath
tdf.RefreshLink
fRetval = True
GoTo CreateAttachedExit
End If
End If
End If
End Function
This code works fine, I can call the function as many times as I want to add links to tables from another database. However, I have about 30 tables to import from the same database and this script restarts from scratch everytime I call it. Because the database is located on another server, it takes about 1 min to link the 30 tables.
Is there anything I could do with that function to make it work faster when I need to link multiple tables from the same database? I would like it to work with multiple strTable and strBaseTable in parameters instead of one (maybe arrays?), but I don't know how to do this.
Thank you.
You can loop through the tabledefs collection in the external database or use a table of tables that lists all the tables to be connected and the external database.
Dim db As Database
Dim ThisDb As Database
Set ThisDb = CurrentDB
sDb = "z:\docs\test.accdb"
Set db = OpenDatabase(sDb)
For Each tdf In db.TableDefs
''Connect
If Left(tdf.Name, 4) <> "MSys" Then
If IsNull(DlookUp("Name","MsysObjects","Type In (1,4,5,6) And Name='" _
& tdf.Name & "'")) Then
DoCmd.TransferDatabase acLink, "Microsoft Access", _
sDb, acTable, tdf.Name, tdf.Name
Else
If ThisDb.TableDefs(tdf.Name).connect <> vbNullString Then
ThisDb.TableDefs(tdf.Name).connect = ";DATABASE=" & sDb
ThisDb.TableDefs(tdf.Name).RefreshLink
End If
End If
End If
Next
A table of tables would work in a similar fashion in that you can select the database (select distinct) from the table and loop through that recordset attaching the tables in a further selection (select table where database ...)
What you could also do is set up the database connect in a different sub as well as the database disconnect (which I might be missing here?).
So then the code that calls this script normally (I assume you have a loop) will look like this:
call function/sub that opens the connection to DATABASE= strPath
your loop that calls Function createAttached
call function/sub that closes the connection to DATABASE= strPath
That way you avoid the repetitive (usually time consuming) connection to the external database

How to export data from excel file and import into to sql database?

Is there a method to do this ? I have like 6000 rows in excel, and it would take months to import it into the sql database.
For example there are : Column A and Column B in excel and I'd like to import it into Column A and Column B into an sql table.
Thanks!
Here is an approach through VBA that I used for a book database. Not too tough as long as you can establish a connection to the DB. Obviously you will have to tweek it a bit for it to work with your database and your data. In particular you will have to adjust the sql string to connect you your database (change "test.yourdatabase")
Sub addBook(title As String, author As String, link As String, foundBy As String)
' for this code to work, you need a reference to the MS Activex
' data access objects
' Tools|References... microsoft activex data objects 2.8 (or newer)
Dim cn As New Connection
Dim cs As String
Dim un as String
Dim pw as String
Dim sql As String
cs = "DRIVER=SQL Server;SERVER=websql.org,5901;DATABASE=websql" 'connection string
un = username 'this is the username for the database
pw = password 'this is the password for the database account
cn.Open cs, Range("un").Value, Range("pw").Value
sql = "insert into test.yourdatabase(searchtitle, searchAuthor, link, foundBy, foundTime) values(" & _
"'<TITLE>', " & _
"'<AUTHOR>', " & _
"'<LINK>', " & _
"'<FOUNDBY>', " & _
"getdate());"
'replace statements are for correcting any ' that occure in the titles or names
sql = Replace(sql, "<TITLE>", Replace(title, "'", "''"))
sql = Replace(sql, "<AUTHOR>", Replace(author, "'", "''"))
sql = Replace(sql, "<LINK>", Replace(link, "'", "''"))
sql = Replace(sql, "<FOUNDBY>", Replace(foundBy, "'", "''"))
'Debug.Print sql
cn.Execute sql
cn.Close
End Sub
If you are using MySQL, it provides a direct way to export data into the database: https://dev.mysql.com/doc/mysql-for-excel/en/mysql-for-excel-export.html
If you are using SQLite/PostGres, there is a commercial offering from Devart to do the same:
https://www.devart.com/excel-addins/postgresql.html
https://www.devart.com/excel-addins/sqlite.html