Create Access table from text file - vba

I need to create an access (2007) table from a text file. I know ahead of time what columns should exist, but from time to time the vendors slip up and submit a text file that contains an incorrect number of columns. So I don't want to specify the columns in advance. I want to load all data as text into whatever columns exist. Then I will do QC.
The columns are pipe delimited and there are over 200 columns per record. There are no column headers, but there is one line of header text for the file, and one line at the end that states how many records there are. There may be anywhere from 1 to over 5,000 records in a text file. Records are identified with CRLF (windows).
Here is what I have so far, and it works (in that it reads the file and places the correct information in the recordset (columns and records), and I can count the number of records), except that the SELECT INTO gives me an error:
Sub OpenTextADO(strFileName As String, strPath As String)
Dim cn As ADODB.Connection
Dim rs As ADODB.Recordset
Dim fld As ADODB.Field
Dim recs As Integer
Dim strRecord As String
Dim strSQL As String
recs = 0
Set cn = New ADODB.Connection
If Right(strFileName, 3) = "txt" Then
'cn.Open "DRIVER={Microsoft Text Driver (*.txt; *.csv)};" & "DBQ=" & strPath & "\" 'need schema.ini file
cn.Open "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & strPath & "\;Extended Properties='text;HDR=No;FMT=Delimited(|)'" 'need schema.ini file
End If
Set rs = New ADODB.Recordset
rs.Open "SELECT * INTO TESTTEXT FROM [" & strFileName & "]", cn, adOpenStatic, adLockOptimistic, adCmdText
'Do Until rs.EOF
' For Each fld In rs.Fields
' strRecord = strRecord & "|" & fld.Value
' Next fld
' strRecord = strRecord & vbCr
' recs = recs + 1
' rs.MoveNext
'Loop
'Debug.Print strRecord
'recs = rs.RecordCount
rs.Close
Set rs = Nothing
MsgBox "Text was opened and there are " & recs & " records in the table."
cn.Close
Set cn = Nothing
End Sub
NOTE: I included both the OLEDB version and the text driver version - they both seem to operate identically. I also created a schema.ini file that looks like this:
[test.txt]
Format=Delimited(|)
ColNameHeader=False
Both drivers seem to need this to desregard column headers, despite the "HDR=No" in the OLEDB version.
The error I get is: "Cannot update. Database or object is read-only".
I appreciate any help.

Could you do a sequential read of the text file, using the count of pipe-delimited fields in the first data line of the file to create a table with the proper number of columns, then just write subsequent lines into that table? I just threw the following together, but it seems to work.
Public Function import_txt_to_db(strFile As String) As Boolean
On Error GoTo ErrHandle
Dim strLine As String
Dim intFileNum As Integer
Dim blnFirstLine As Boolean
blnFirstLine = True
Dim varArray As Variant
intFileNum = FreeFile
Open strFile For Input Access Read As intFileNum
Do While Not EOF(intFileNum)
Line Input #intFileNum, strLine
varArray = Split(strLine, "|")
If blnFirstLine = True Then
'Use count of fields in first line to determine # of columns to create
Dim intColCount As Integer
intColCount = UBound(varArray)
Dim strQry As String
strQry = "CREATE TABLE tblImport ("
Dim intCtr As Integer
For intCtr = 1 To intColCount + 1
strQry = strQry & "[COLUMN_" & intCtr & "] TEXT(255),"
Next intCtr
strQry = Left(strQry, Len(strQry) - 1) & ")" 'get rid of terminal comma
CurrentDb.Execute strQry
blnFirstLine = False
End If
Dim strQry2 As String
strQry2 = "INSERT INTO tblImport VALUES('" & Replace(strLine, "|", "','") & "')"
CurrentDb.Execute strQry2
Loop
Close #intFileNum
import_txt_to_db = True
Exit Function
ErrHandle:
import_txt_to_db = False
End Function
I did a simple test with the folowing five-line text file
Thomas|Jefferson|Virginia
Bill|Clinton|Arkansas
Jimmy|Carter|Georgia
Lyndon|Johnson|Texas
George|Washington|Virginia
After running the code, here's my (simple) table:

Related

can i import csv to mdb

the code first changes the dbf file to csv.
Sub FindFiles()
Dim strDocPath As String
Dim strCurrntFile As String
Dim Fname As String
strDocPath = "Y:\Eilat\Shapes\"
'strCurrentFile = Dir(strDocPath & "*.*")
strCurrentFile = Dir(strDocPath & "111.dbf")
Workbooks.Open FileName:=strDocPath & strCurrentFile
Fname = Left$(strCurrentFile, Len(strCurrentFile) - 4) & ".csv"
ActiveWorkbook.SaveAs FileName:=strDocPath & Fname, FileFormat:=xlCSVMSDOS, CreateBackup:=False
ActiveWorkbook.Close (True)
Dim filepath As String
Dim sqlinsert As String
Dim sqlvalue As String
Dim sqlquery As String
Dim sqlwhere As String
'Set db = CurrentDb
directory = "Y:\Eilat\Shapes\"
FileName = "111.csv"
Set rs = CreateObject("ADODB.Recordset")
strcon = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & directory & ";" _
& "Extended Properties=""text;HDR=Yes;FMT=Delimited"";"
'strSQL = "SELECT * FROM " & FileName
'rs.Open strSQL, strcon
'rs.MoveFirst
Dim strTextLine As String
Dim aryMyData() As String
Open directory & FileName For Input As #1
Do While Not EOF(1) ' Loop until end of file.
Line Input #1, strTextLine ' Read line into variable.
aryMyData = Split(strTextLine, ",") 'Split text into array by comma
' (the csv length changes for example this is the columns in csv but it can be longer. EHANDLE,UseCode,UseCode2,Descriptio,Gush,Helka,Owner,OwnerID,Holder,HolderID,Floor,PhysicalNo,Date,Area,Comments,Address,StreetName,HouseNo,Telephone,Fax,Email,Manager,Business,SerialNo,MeasuredBy,Height,BlockNo,Mapkey,User1,User2,user3 is the columns in csv but it can be longer.
strSQL = "??(what sql statement need??)
(dont know how to do import to mdb "Y:\Eilat\Arnona\Eilat.mdb")
Debug.Print strSQL
DoCmd.RunSQL strSQL
Loop
Close
End Sub
With the assumptions:
Your code properly opens the CSV file and reads line by line
All your table fields are of type String
Your table has fields Field_01, Field_02, Field_03 into which you
want to import columns 1, 2 and 3 of the CSV file
You can use
DoCmd.RunSQL "INSERT INTO MyTable (Field_01, Field_02, Field_03) VALUES ('" & aryMyData(0) & "','" & aryMyData(1) & "','" & aryMyData(2) & "'")
and expand as needed for all your fields and columns.
This is a very basic example that assumes your table is specifically prepared for the CSV file you are importing. If you expect the table's design to change to accommodate whatever CSV file you are importing that is way more complicated.

Unable to Select First Column of .csv with ADODB in VBA [duplicate]

I would like to query an UTF-8 encoded CSV file using VBA in Excel 2010 with the following database connection:
provider=Microsoft.Jet.OLEDB.4.0;;data source='xyz';Extended Properties="text;HDR=Yes;FMT=Delimited(,);CharacterSet=65001"
All CSV files start with the BOM \xEF\xBB\xBF and the header line. Somehow the BOM isn't recognized correctly and the first column header gets read as "?header_name", i.e. a question mark gets prepended. I have tried different CharacterSets and I have also tried to use Microsoft.ACE.OLEDB.12.0, but everything was without success so far.
Is this a known bug or is there any way to get the right first column header name without changing the encoding of the source files?
The following procedure extracts the entire CSVfile into a new Sheet, clearing the BOM from the Header. It has the Path, Filename and BOM string as variables to provide flexibility.
Use this procedure to call the Query procedure
Sub Qry_Csv_Utf8()
Const kFile As String = "UTF8 .csv"
Const kPath As String = "D:\StackOverFlow\Temp\"
Const kBOM As String = "\xEF\xBB\xBF"
Call Ado_Qry_Csv(kPath, kFile, kBOM)
End Sub
This is the Query procedure
Sub Ado_Qry_Csv(sPath As String, sFile As String, sBOM As String)
Dim Wsh As Worksheet
Dim AdoConnect As ADODB.Connection
Dim AdoRcrdSet As ADODB.Recordset
Dim i As Integer
Rem Add New Sheet - Select option required
'With ThisWorkbook 'Use this if procedure is resident in workbook receiving csv data
'With Workbooks(WbkName) 'Use this if procedure is not in workbook receiving csv data
With ActiveWorkbook 'I used this for testing purposes
Set Wsh = .Sheets.Add(After:=.Sheets(.Sheets.Count))
'Wsh.Name = NewSheetName 'rename new Sheet
End With
Set AdoConnect = New ADODB.Connection
AdoConnect.Open "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=" & sPath & ";" & _
"Extended Properties='text;HDR=Yes;FMT=Delimited(,);CharacterSet=65001'"
Set AdoRcrdSet = New ADODB.Recordset
AdoRcrdSet.Open Source:="SELECT * FROM [" & sFile & "]", _
ActiveConnection:=AdoConnect, _
CursorType:=adOpenDynamic, _
LockType:=adLockReadOnly, _
Options:=adCmdText
Rem Enter Csv Records in Worksheet
For i = 0 To -1 + AdoRcrdSet.Fields.Count
Wsh.Cells(1, 1 + i).Value = _
WorksheetFunction.Substitute(AdoRcrdSet.Fields(i).Name, sBOM, "")
Next
Wsh.Cells(2, 1).CopyFromRecordset AdoRcrdSet
End Sub
The only solution for this problem I found is to use Schema.ini file.
my test csv file
Col_A;Col_B;Col_C
Some text example;123456789;3,14
Schema.ini for my test csv file
[UTF-8_Csv_With_BOM.csv]
Format=Delimited(;)
Col1=Col_A Text
Col2=Col_B Long
Col3=Col_C Double
This Schema.ini file contains the name of the source csv file and describes my columns. Each column is specified by its name and type but you can specify more informations. This file must be located in the same folder as your csv file. More info here.
Finally the VBA code which reads the csv file. Note that HDR=No. This is because the columns headers are defined in the Schema.ini.
' Add reference to Microsoft ActiveX Data Objects 6.1 Library
Sub ReadCsv()
Const filePath As String = "c:\Temp\StackOverflow\"
Const fileName As String = "UTF-8_Csv_With_BOM.csv"
Dim conn As ADODB.Connection
Dim rs As New ADODB.Recordset
Set conn = New ADODB.Connection
conn.Open "Provider=Microsoft.Jet.OLEDB.4.0;Data Source='" & filePath & _
"';Extended Properties='text;HDR=No;FMT=Delimited()';"
With rs
.ActiveConnection = conn
.Open "SELECT * FROM [" & fileName & "]"
If Not .BOF And Not .EOF Then
While (Not .EOF)
Debug.Print rs.Fields("Col_A") & " " & _
rs.Fields("Col_B") & " " & _
rs.Fields("Col_C")
.MoveNext
Wend
End If
.Close
End With
conn.Close
Set conn = Nothing
End Sub
Output
Some text example 123456789 3,14

VBA, Import CSV split by ";" to sheet

I am trying to import a CSV file split by semicolon ";" into an excel object so I can use it later on.
Ideally i would like to use ADO, DAO or ADODB so I can also run SQL queries on the object, and get sum of specific fields, or total number of fields and so on.
So far i've gotten the code below, but it does not split the data by ";", so it all comes back as 1 field instead of multiple fields that can be handled.
Sub Import()
Dim conn As New ADODB.Connection
Dim rs As New ADODB.Recordset
Dim f As ADODB.Field
Dim csvName, csvPath
csvPath = ActiveWorkbook.path
csvName = "fileName.csv"
conn.Open "DRIVER={Microsoft Text Driver (*.txt; *.csv)};DBQ=" & csvPath & ";"
rs.Open "SELECT * FROM " & csvName, conn, adOpenStatic, adLockReadOnly, adCmdText
Debug.Print rs.Fields
While Not rs.EOF
For Each f In rs.Fields
Debug.Print f.Name & "=" & f.Value
Next
Wend
End Sub
Can anyone give me an idea how I can also split the data by ";" and query it using SQL query? Or a different object that I could load a CSV into and query certain columns.
Here's example:
Public Sub QueryTextFile()
Dim rsData As ADODB.Recordset
Dim sConnect As String
Dim sSQL As String
' Create the connection string.
sConnect = "Provider=Microsoft.ACE.OLEDB.12.0;" & _
"Data Source=C:\Files\;" & _
"Extended Properties=Text;"
' Create the SQL statement.
sSQL = "SELECT * FROM Sales.csv;"
Set rsData = New ADODB.Recordset
rsData.Open sSQL, sConnect, adOpenForwardOnly, _
adLockReadOnly, adCmdText
' Check to make sure we received data.
If Not rsData.EOF Then
' Dump the returned data onto Sheet1.
Sheet1.Range("A1").CopyFromRecordset rsData
Else
MsgBox "No records returned.", vbCritical
End If
' Clean up our Recordset object.
rsData.Close
Set rsData = Nothing
End Sub
The only answer I found that was usable was to create an ini file in the current folder, and enter the delimiter in the ini file.
iniPath = activeworkbook.path & "\"
iniName = "schema.ini"
iniPathName = iniPath & iniName
If Not fso.FileExists(iniPathName) Then
fso.CreateTextFile (iniPathName)
End if

Inherited MS Access Database, Tracking Sources of Queries

I have just inherited a database at my new company. Old DB owner left no good documentation and queries very hard to keep track of. Looking for programmatic answer to track sources of fields in every query (what table it come from). Prefer something can be exported to Excel to study, Access visualization is no good. Am familiar with VBA.
This is pretty messy but could save you time collecting each query's SQL code. The following code exports all SQL stored in the QueryDefs collection into a text file. I have it splitting the code with a space delimiter, but a comma might be preferable. The data will not be normalized, I don't have the time to go to that level of complexity. Just make sure to update strPath before you execute. Hopefully this helps.
Sub PullQuerySQL()
Dim dbs As Database
Dim i As Integer
Dim fso As Object
Dim oFile As Object
Dim varParse() As String
Dim element As Variant
Dim strPath As String
strPath = ".txt"
Set dbs = CurrentDb()
Set fso = CreateObject("Scripting.FileSystemObject")
Set oFile = fso.CreateTextFile(strPath)
For i = 0 To dbs.QueryDefs.Count - 1
oFile.WriteLine dbs.QueryDefs(i).Name
varParse = Split(dbs.QueryDefs(i).SQL, " ")
For Each element In varParse
oFile.WriteLine element
Next element
Next i
oFile.Close
Set oFile = Nothing
Set fso = Nothing
Set dbs = Nothing
End Sub
I have been through this with many inherited databases. I find it extremely helpful to create an Access table with the fields and the tables/queries that they come from. Try this code below. It will prompt you for the name of the query that you are looking to "map" as I call it. It will then create a new table named "queryName Definitions".
Option Compare Database
Public Sub MapQuery()
Dim strQueryName As String
Dim rst As DAO.Recordset
Dim fld As Field
Dim strSource As String
Dim strField As String
Dim strValue As String
Dim strSQL1 As String
Dim strSQL2 As String
Dim booExists As Boolean
strQueryName = InputBox("Please enter the name of the query that you are looking to map")
Set rst = CurrentDb.OpenRecordset(strQueryName)
On Error GoTo error1
booExists = IsObject(CurrentDb.TableDefs(strQueryName & " Definitions"))
DoCmd.DeleteObject acTable, strQueryName & " Definitions"
continue:
strSQL1 = "CREATE TABLE [" & strQueryName & " Definitions]" & " (FieldName CHAR, SourceName CHAR);"
DoCmd.RunSQL (strSQL1)
DoCmd.SetWarnings False
For Each fld In rst.Fields
strField = fld.Name
strSource = fld.SourceTable
Debug.Print strValue
strSQL2 = "INSERT INTO [" & strQueryName & " Definitions]" & "(FieldName, SourceName) VALUES(""" & strField & """, """ & strSource & """);"
DoCmd.RunSQL (strSQL2)
Next fld
error1:
If Err.Number = 3265 Then
Resume continue
Else
MsgBox Err.Description
End If
DoCmd.SetWarnings True
Exit Sub
DoCmd.SetWarnings True
End Sub

How can creating dbf file, and define encoding in Notepad, or VBA

what is DBF4 (dBase IV)(*.dbf) file fundamental format? And how can create these file in a same word editor as Notepad with typing?(Update:, or excel VBA?)
What is that's format specifications as:
Delimiter (Same as: , or tab or etc)
Separator (may Same as above!) (If these two are not synonymy)
Row End character: (Same as vbCrLf)
Defining headers of columns(fields).
Code-Page of encoding: (same as: Unicode - 1256 or etc)
and others...
Please present an algorithm for creating this DB file format that made us able to create a same file easily by a VBA method which creates a text file.
(Update Or using built-in VBA or its references methods.)
I using below for creating text file.
Sub CsvExportRange(rngRange As Object, strFileName As String, strCharset, strSeparator As String, strRowEnd As String, NVC As Boolean) 'NVC: _
Null Value Control (If cell contain Null value, suppose reached end of range), d: delimiter
Dim rngRow As Range
Dim objStream As Object
Dim i, lngFR, lngLR As Long 'lngFR: First Row, lngLR: Last Row
lngFR = rngRange.SpecialCells(xlCellTypeVisible).Rows(1).row - rngRange.Rows(1).row + 1
lngLR = rngRange.End(xlDown).row - rngRange.Rows(1).row + 1
Set objStream = CreateObject("ADODB.Stream")
objStream.Type = 2
objStream.Charset = strCharset
objStream.Open
For i = lngFR To lngLR
If Not (rngRange.Rows(i).EntireRow.Hidden) Then
If IIf(NVC, (Cells(i + rngRange.Rows(1).row - 1, _
rngRange.SpecialCells(xlCellTypeVisible).Columns(1).column).Value = vbNullString), False) Then Exit For
objStream.WriteText CsvFormatRow(rngRange.Rows(i), strSeparator, strRowEnd)
End If
Next i
objStream.SaveToFile strFileName, 2
objStream.Close
End Sub
Function CsvFormatRow(rngRow As Variant, strSeparator As String, strRowEnd As String) As String
Dim arrCsvRow() As String
ReDim arrCsvRow(rngRow.SpecialCells(xlCellTypeVisible).Cells.Count - 1)
Dim rngCell As Range
Dim lngIndex As Long
lngIndex = 0
For Each rngCell In rngRow.SpecialCells(xlCellTypeVisible).Cells
arrCsvRow(lngIndex) = CsvFormatString(rngCell.Value, strSeparator)
lngIndex = lngIndex + 1
Next rngCell
CsvFormatRow = Join(arrCsvRow, strSeparator) & strRowEnd
End Function
Function CsvFormatString(strRaw, strSeparator As String) As String
Dim boolNeedsDelimiting As Boolean
Dim strDelimiter, strDelimiterEscaped As String
strDelimiter = """"
strDelimiterEscaped = strDelimiter & strDelimiter
boolNeedsDelimiting = InStr(1, strRaw, strDelimiter) > 0 _
Or InStr(1, strRaw, chr(10)) > 0 _
Or InStr(1, strRaw, strSeparator) > 0
CsvFormatString = strRaw
If boolNeedsDelimiting Then
CsvFormatString = strDelimiter & _
Replace(strRaw, strDelimiter, strDelimiterEscaped) & _
strDelimiter
End If
End Function
(Forgotten source)
Because I reached this: I should create a dbf file from my Excel Range by hand! After searching founded web sources.
Updated:
How can declare encoding of DBF?
About encoding that needed, considerable ones is Commonplace in this issue is Iran System encoding.
How can I store data with suitable encoding as Iran System in DB table records?
we have joy .... lol
this test code creates a dbf file from data in excel worksheet
creates a table and inserts one record
Sub dbfTest()
' NOTE: put this test data at top of worksheet (A1:F2)
' Name Date Code Date2 Description Amount
' frank 11/12/2017 234.00 11/20/2018 paint $1.34
' ref: microsoft activex data objects
Dim path As String
Dim fileName As String
filePath = "C:\database\"
fileName = "test"
Dim dc As Range
Dim typ As String
Dim fieldName As String
Dim createSql As String
createSql = "create table " + fileName + " (" ' the create table query produces the file in directory
Dim a As Variant
For Each dc In Range("a1:e1")
fieldName = dc.Value
a = dc.offset(1).Value
Select Case VarType(a)
Case vbString: typ = "varchar(100)"
Case vbBoolean: typ = "varchar(10)"
Case vbInteger: typ = "int"
Case vbLong: typ = "Double"
Case vbDate: typ = "TimeStamp"
Case Else: typ = "varchar(5)" ' default for undefined types
End Select
createSql = createSql + " [" + fieldName + "]" + " " + typ + ","
Next dc
createSql = Left(createSql, Len(createSql) - 1) + ")"
Debug.Print createSql
Dim conn As ADODB.connection
Set conn = CreateObject("ADODB.Connection")
conn.Open "DRIVER={Microsoft dBase Driver (*.dbf)};" & "DBQ=" & filePath ' both work
' conn.Open "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & filePath & ";Extended Properties=dBASE IV"
Dim cmd As ADODB.Command
Set cmd = CreateObject("ADODB.Command")
cmd.ActiveConnection = conn
cmd.CommandText = createSql
cmd.Execute
Dim insertSql As String
insertSql = "insert into " + fileName + " values("
For Each dc In Range("a2:e2")
insertSql = insertSql + "'" + CStr(dc.Value) + "',"
Next dc
insertSql = Left(insertSql, Len(insertSql) - 1) + ")"
Debug.Print insertSql
cmd.CommandText = insertSql
cmd.Execute
conn.Close
Set conn = Nothing
End Sub
my research has concluded. the Iran System encoding is actually ascii, it is not unicode. it uses ascii values to represent some of the Persian alphabet.
the problem with converting from unicode to Iran System encoding is that any letter is written completely differently depending where in the word it is positioned. you have "isolated", "initial", "medial" and "final" forms of most of the letters.
it is like upper and lower case on steroids ... lol
ref: https://www.math.nmsu.edu/~mleisher/Software/csets/IRANSYSTEM.TXT
so additional process would be needed to convert unicode text in excel into an equivalent Iran System encoding string before storing in database.
the code creates a table with one text field and stores 3 records
Sub dbfTestWork()
' ref: microsoft activex data objects
Dim filePath As String
Dim fileName As String
filePath = "C:\database\"
fileName = "test"
Dim conn As ADODB.Connection
Set conn = CreateObject("ADODB.Connection")
conn.Open "Driver={Microsoft dBase Driver (*.dbf)};Dbq=" + filePath + ";"
'conn.Open "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & filePath & ";Extended Properties=dBASE IV;"
Dim fil As String
fil = filePath & fileName & ".dbf"
If Not Dir(fil, vbDirectory) = vbNullString Then Kill fil ' delete file if it exists
Dim cmd As ADODB.Command
Set cmd = CreateObject("ADODB.Command")
cmd.ActiveConnection = conn
cmd.CommandText = "create table test ([testTextData] char(20))"
cmd.Execute
Dim nFileNum As Integer
nFileNum = FreeFile ' Get an available file number from the system
Open filePath & fileName & ".dbf" For Binary Lock Read Write As #nFileNum ' Open the file in binary mode. Locks are optional
Put #nFileNum, 30, CByte(1) ' set language driver id (LDID) 0x01 = ascii encoding
Close #nFileNum
' Debug.Print Range("e2").Value
Dim aaa As String
aaa = StrConv(Range("e2").Value, vbUnicode)
' Debug.Print aaa
Dim cmdStr As String
cmdStr = "insert into test values ('"
Dim ccc As Variant
For Each ccc In Array("ac", "92", "9e", "20", "93", "a1", "fe", "a4") ' one of these two should store
cmdStr = cmdStr & Chr(CDec("&h" & ccc)) ' "good morning" in persian
Next ccc
cmdStr = cmdStr & "');"
cmd.CommandText = cmdStr
cmd.Execute
cmdStr = "insert into test values ('"
For Each ccc In Array("a4", "fe", "a1", "93", "20", "9e", "92", "ac")
cmdStr = cmdStr & Chr(CDec("&h" & ccc))
Next ccc
cmdStr = cmdStr & "');"
cmd.CommandText = cmdStr
cmd.Execute
cmd.CommandText = "insert into test values ('abc123');"
cmd.Execute
conn.Close
Set conn = Nothing
End Sub
'