I have a tool which reads dBase files and uploads the contents to SQL Server, part of a system to import shapefiles. It works but now we have a requirement to import files that include non-English characters (Norwegian in this case, could be other languages later) and they're being corrupted.
The dBase files are being read using an OleDbDataAdapter. Stepping through the code I can see that the text is wrong as it is read in. I'm assuming it's something to do with code pages or Unicode but I have no idea how to fix it.
A dBase Reader application tells me the DBFs are in code page 1252 - I don't know if this is correct. My upload tool runs on Win7 with English (UK) regional settings.
Examples:
ÅSGARD in DBF becomes +SGARD in VB.Net & SQL Server.
RINGHORNE ØST in DBF becomes RINGHORNE ÏST in VB.Net & SQL Server.
The code that reads the DBF:
dbfConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & strPath & ";Extended Properties=dBASE IV"
Cnn.ConnectionString = dbfConnectionString
Cnn.Open()
strSQL = "SELECT * FROM [" & strDBF & "]"
DA = New OleDb.OleDbDataAdapter(strSQL, Cnn)
DS = New DataSet
DA.Fill(DS)
If DS.Tables(0).Rows.Count > 0 Then
dtDBF = DS.Tables(0)
Else
dtDBF = Nothing
End If
Data is read like: Name = dtDBF.Rows(index)("NAME_1")
Is there a way to tell OleDbDataAdapter what code page to use or a better way to read dBase files from VB.Net?
Try adding this to your DSN:
CollatingSequence=Norwegian-Danish
You might also be able to use:
CollatingSequence=International
Check whether the shapefile contains codepage information. There are two places to look
Look in the language driver ID (LDID), which is found in the header of the shapefile’s DBF table (in the 29th byte).
Look for an associated separate file with extension .cpg.
If the code page is not specified in those locations, it defaults to the codepage on the PC that generated the shapefile. You will just have to know that :(
I've never used it, but maybe Shape2SQL takes care of this for you? Or shp2text? I believe the PostGIS shapefile loader handles code pages: maybe you could import into PostGIS and then export in another format??
Old question, but this may answer it for future readers...
You might try adding a property setting in your connection string:
Locale Identifier=1044
This property (and a list of values including this one) is documented for ADO in conjunction with Jet 4.0's OLDB Provider but I have no reason to believe it isn't also supported by ADO.Net. This value (1044) is Norwegian/Danish.
Untested, but something else to try.
Related
Dim cat As New Catalog()
Dim con As New OleDbConnection()
Dim cmd As New OleDbCommand
Dim ds1 As New DataSet
Dim conn As ADODB.Connection
' Open the Access database.
conn = New Connection
conn.ConnectionString =
"Provider=Microsoft.Jet.OLEDB.4.0;" &
"Data Source=" + openExcel + "\Test" + ".mdb; Persist Security Info=False"
con.ConnectionString =
"Provider=Microsoft.Jet.OLEDB.4.0;" &
"Data Source=" + openExcel + "\Test" + ".mdb; Persist Security Info=False"
conn.Open()
xlWorkSheet1.Columns(5).Insert
Dim cellValue As String = ""
Dim newValue As String = ""
Dim sh1 As String = ""
Dim qty As String = ""
Dim matchText As String = ""
Dim sql As String = ""
con.Open()
sh1 = LTrim$(xlWorkSheet1.Cells(i, 1).Text)
sql = "SELECT Num_ber, Q_ty FROM good WHERE Na_me LIKE 'staff%' And Ty_pe = 'ORD'"
Dim cmd As New OleDbCommand(sql, con)
Dim myReader As OleDbDataReader = cmd.ExecuteReader()
conn.Execute(sql)
myReader = cmd.ExecuteReader() ' HERE'S THE PROBLEM
xlWorkSheet1.Cells(1, 5) = myReader.GetString(0)
xlWorkSheet1.Cells(1, 11) = myReader.GetString(1)
myReader.Close()
conn.Close()
conn = Nothing
**I wanted to retrieve a specific value from mdb and then write it to excel.
Here's my code, I got this error so many times and I can't find it out. Can anybody help me? Thanks.**
[1]: https://i.stack.imgur.com/6Fuvg.png
Ok, you first have to decide when usng .net what "provider" you are going to use, AND THEN decide what kind of data objects you want to use.
You can use a oracle provider, a sql server provider, or in this case, since we using Access, then we can use EITHER oleDB, or ODBC. Either choice is fine. Most use oleDB providers for Access, but often ODBC is a good choice, especially if down the road you plane to swap out Access for say SQL server.
What the above means in plain English?
You don't want to adopt the external ADODB code and library. That code is NOT .net, and thus you REALLY but REALLY do not want to write your .net code that way. ADODB was written LONG before .net, and is what we call un-managed code (non .net). I strong, but strong suggest you do NOT add a reference to ADODB to your project, and I beyond strong recommend you avoid introduction of a non .net library for doing this!!! We certainly can adopt the oleDB provider in .net, but we will NOT have a direct reference to JET/ACE (the access database engine) in our applcation. As noted, there are some exceptions to this suggesting, but they don't apply to you and here.
Next up:
The design pattern in .net is to create the connection, get the data, and CLOSE the connection. This "pattern" will then be 100% sure that the data base is always closed, and you NEVER have to worry about if the connection is open, closed, or even if you forgot to close the connection!!! So, do this correct, and some "open" connection will never bite you, or will you have to worry about this issue.
You can in some operations keep the connection open for performance, but lets learn to walk before we run so to speak.
next up:
We certainly do NOT want to place and have connection strings all over in our code. Not only is this going to wear out your keyboard, but if you ever need to change the connection, then you going to have to hunt down all that code.
Best to let Visual Studio save that connection in ONE location, and MORE important MANAGE this for you!!!
Next up:
Do you ONLY need to work with mdb files, or do you plan/need to work with accDB files? This is a HUGE issue, and one that you cannot ignore.
next up:
Are you going to use the x32 bit version of the Access database system, or the x64 bit version?
Since your example posted code uses JET (access data engine for mdb files ONLY x32 bit version)?
Then this ALSO means you MUST (and I repeat MUST) force your .net project to run as x32 bits. You cannot use "any cpu", and you cannot use x64 bits, you MUST choose x86 bit size for your .net project. Failure to do so will result in the project not working.
Ok, with the above information?
First up, force/set/be 100% sure your project is set to run as x32 bits.
That setting is this one:
and remove the reference you have to ADO if you created one.
Ok,
next up:
Create the connection to the database.
Project ->properties.
This here:
And then:
and then
Now, you can browse, and select the access mdb file.
But, you MUST not skip the next step - you have to choose JET (older, mdb files), or choose the newer ACE (for accDB format files).
So, this:
now this:
As noted, you choose JET or ACE here.
now, we have this and you can use test connection.
BUT BE VERY careful!!!!
If you are using vs2022, then keep in mind vs2022 is the FIRST version of VS that is now x64 bits. As a result, vs can't pass the test connection!!! Your connection is in fact ok, but will fail with vs2022.
If you using a previous version of VS (before 2022), then the test connection button should work. and you see this:
Ok, now that we have a valid working conneciton setup, we can now write code, and we will NOT use ADODB!!!!
The typical code pattern to read and load a data table (like a access VBA recordset) will be like this:
Now, I became RATHER tired of writing that same using block over and over. So, in a global module, I have this code now:
Public Function MyRst(strSQL As String) As DataTable
Dim rstData As New DataTable
Using conn As New OleDbConnection(My.Settings.AccessDB)
Using cmdSQL As New OleDbCommand(strSQL, conn)
conn.Open()
rstData.Load(cmdSQL.ExecuteReader)
End Using
End Using
Return rstData
End Function
So, now with the above handy dandy helper routine?
Your code becomes this:
Private Sub Button2_Click(sender As Object, e As EventArgs) Handles Button2.Click
Dim Sql As String =
"SELECT Num_ber, Q_ty FROM good WHERE Na_me LIKE 'staff%' And Ty_pe = 'ORD'"
Dim rstData As DataTable = MyRst(Sql)
Debug.Print("Na_me is " & rstData.Rows(0).Item("Na_me"))
End Sub
Or, display all return rows from that data table
Debug.Print("Na_me is " & rstData.Rows(0).Item("Na_me"))
For Each OneRow As DataRow In rstData.Rows
Debug.Print("na_me = " & OneRow("Na_me"))
Next
So, you really don't need (or want) a reader anyway. Just load the results into a nice clean, easy to use data table, and from that you can loop the table, grab rows, or do whatever you want.
I have a directory with the following files:
Data.dat
Data.ldb
Program.exe
prog.UDL
I want to access the data in Data.dat which is 82mb large
Any solution?
I tried to run the program but it throws an error that it cannot connect to database. I was wondering if there is a way to access the data.dat file and view the data. When I try to open it with a text editor it looks like this:
I managed to view the file with MDB Viewer Plus and this is what I'm getting; a list of tables with this sort of encrypted data:
I looked around a bit and stumbled across this Visual Basic code here.
Set cat = CreateObject("ADOX.Catalog")
Set cn = CreateObject("ADODB.Connection")
cn.Open "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=C:\Docs\db1.dat;"
Set cat.ActiveConnection = cn
For Each t In cat.Tables
strList=t.Name & vbcrlf & strList
Next
MsgBox strList
Here's the Forum Link
If you just want to view any readable content from the file without any structure, a *nix environment includes a program called 'strings' which will output the printable character strings from a file and filter out the rest.
I you're running Windows, sysinternals provides the same utility:
https://learn.microsoft.com/en-gb/sysinternals/downloads/strings
I am trying to directly import a text file into Postgres, but using ODBC. I realize that this is a bit of an odd thing to do, but ODBC does a good job of fixing/ignoring errors in the text files and Postgres' Copy command is very, very picky. I use Copy when I can and ODBC where I can't.
I am currently doing this in two steps. ODBC Import to Access and then from Access to Postgres but I recently learned over on MSDN I may be able to do this in one step but am having trouble with the SQL.
Here is the code I am using:
Dim TextConnection As New System.Data.OleDb.OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=c:\PathToTextFile;Extended Properties=""Text;HDR=No""")
Dim TextCommand As System.Data.OleDb.OleDbCommand
TextCommand = New System.Data.OleDb.OleDbCommand("SELECT * INTO [ODBC;Driver={PostgreSQL};" & _
" Server=server;Database=database;Uid=UserName;Pwd=Password;].[TableName] FROM [textfile.txt]", TextConnection)
TextCommand.ExecuteNonQuery()
I am getting this error: Query input must contain at least one table or query.
I am not where to go from here in debugging this. It also just might now be possible and that would be good to know.
I want to write a program which will replace my current paper based record. My current paper record is basically many column and rows with different width, height, and other properties. I know how to write a VB program that can save the information, but I don't know how to make the VB program to generate a xls datasheet to which would exactly like my paper record.
Would someone please give me the information about that?
Thanks :)
I would recommend http://epplus.codeplex.com/releases/view/42439.
It is very easy to use and integrates flawlessy in vb.net.
I am not providing code as a sample because the samples which are included in the package are very good.
As a hint: Internally I would use a Data-Table to store your values and then use a separate module to load/store it to excel.
An excel file could be thougth as a simple database where each sheet is a different table.
Assuming you have Excel on your machine, you could create an empty XLS file and then use OleDB to fill the worksheets.
Sub WriteToExcel()
Dim con As String con = "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=C:\temp\test.xls;" & _
"Extended Properties='Excel 8.0;HDR=No;'"
Using c as OleDbConnection = new OleDbConnection(con))
c.Open()
Dim commandString as String = "Insert into [Sheet1$] (F1, F2, F3) " & _
"values('Column1Text', 'Column2Text', 'Column3Text')"
Using cmd As OleDbCommand = new OleDbCommand(commandString))
cmd.Connection = c
cmd.ExecuteNonQuery()
End Using
End Using
End Sub
other options include OpenXml (which I'd have thought is the "recommended" way to do it but which brings with it a learning curve) or at the other end of the scale (in terms of crudeness) write your data in a comma-separated (csv) format and manually import it into Excel
I'm using the following code (System.Data.SQLite within VB.net):
Dim SQLconnect As New SQLite.SQLiteConnection()
Dim SQLcommand As SQLiteCommand
SQLconnect.ConnectionString = "Data Source=vault.db;"
SQLconnect.Open()
SQLcommand = SQLconnect.CreateCommand
SQLcommand.CommandText = "INSERT INTO entries VALUES ('" & appinput.Text & "','" & userinput.Text & "','" & passinput.Text & "')"
SQLcommand.ExecuteNonQuery()
SQLcommand.Dispose()
SQLconnect.Close()
Me.Hide()
I get an error back that says it can't find the table "entries"
I know the table exists because I can write to it via the command line through sqlite and through Autoit and can see it and edit it in the SQLite browser when I open the database. I don't understand why VB can't see it (I know it sees and opens the database file just fine). Any ideas?
Most likely your problem is with relative paths (directories).
sqlite will create a database file if it does not exist so you will never get a "db file not found message". The first indication of an incorrect path is "table missing".
My personal experience is that although it goes against my programmers instinct is to alway use an absolute (fully qualified) path/file name for an sqlite database.
If you put in the full file location like "/var/myapp/vault.db" you should be OK.
If this is likly to move around store pick up the file name from a properties/config file -- 'config file not found' is much easier to deal with than "table not found".
Argh! There are 3 big issues in that code. Please update it like this to fix two of them:
Using cn As New SQLite.SQLiteConnection("Data Source=vault.db;"), _
cmd As New SQLiteCommand("INSERT INTO entries VALUES (#AppInput, #UserInput, #PassInput)", cn)
cmd.Parameters.AddWithValue("#AppInput", appinput.Text);
cmd.Parameters.AddWithValue("#UserInput", userinput.Text);
cmd.Parameters.AddWithValue("#PassInput", passinput.Text);
cn.Open()
cmd.ExecuteNonQuery()
End Using
This will prevent sql injection by parameterizing your query instead of substituting values directly and prevent db locking issues by making sure your connection is disposed properly, even if an exception is thrown.
The third problem is that you should NEVER store plain-text passwords in your database (or anywhere else for that matter). Go read up on how to hash values in .Net and hash and salt your password before storing or comparing it.
Once you've done that re-test your code to see if you still get the same errors reported as before. We need to make sure this didn't solve the problem or introduce something new. Then we can start addressing the missing table issue, perhaps by checking your connection string.
I had a similar error with SQLite (via .Net) refusing to believe the table existed, even though direct access confirmed it was there. The error could be produced only on one individual machine and not others. Hard coding the path didn't seem to fix the problem. The fix was either to run the program as Administrator or change the DB file to be available to Everyone. Apparently the .Net assembly raises a missing table error when the actual problem is access restrictions.