I need to open foxpro free tables in vb.net using the oledb connection.
But... I only need to get the column names. I don't really need to 'select' anything.
I am trying to dynamically browse through all our free tables and set up a listing of every column from every file and xref that to another free table that contains a description of each column.
I have a working model now, but it requires that I do...
SELECT TOP 1 FROM "File" ORDER BY 1
But on the largest table, it takes over two minutes just to read in the first record and there are over 250 tables. Overall, it takes between 15 and 20 minutes.
Or, is there another way to only get the first record of the table without using 'ORDER BY'?
Here's what I have so far. "File" is passed in as a parameter.
It would contain info like "C:\data\table1.dbf"
Dim filePath As String
filePath = IO.Path.GetDirectoryName(file)
myOledbConnection = New OleDbConnection("Provider=VFPOLEDB.1;Data Source=" & filePath & ";Collating Sequence=MACHINE")
myOledbCommand = New OleDbCommand
myOledbDataAdapter = New OleDbDataAdapter
Dim fields, from, order As String
fields = "select top 1 *"
from = " from " & file
order = " order by 1"
myOledbCommand.CommandText = fields & from & order
myOledbCommand.Connection = myOledbConnection
myOledbDataAdapter.SelectCommand = myOledbCommand
myOledbDataAdapter.Fill(dt)
I then take the datatable (dt) and loop through to get the column information.
I would like it to be as quick as Visual Studio is when I create a dataset and load all tables from the directory through the wizard. It is able to very quickly find all the column information without reading in the data from the table.
Let me know if you need more information.
Thanks.
Why do you need to get any records at all? You should be able to say:
SELECT * FROM "File" where 1 = 0
This will give you an empty result set, will also give you metadata on the projection returned.
You might also want to look into the GetOleDbSchemaTable method on the OleDbConnection class, as it will allow you to get information about the schema of the database without having to perform a query.
You can also use the Microsoft ADO Extensions for Data Definition Language and Security through COM interop (mxADOX.dll) to get the schema information as well.
I have not tried this/. But, it looks like the way to go.
Specifically the "GetSchema" method on OleDbConnection instance.
http://msdn.microsoft.com/en-us/library/ms254934(VS.80).aspx
Related
One of the requirements of a project that I have is to allow users to import or copy and paste in bulk a few hundred rows from excel to access. However, there is a reasonable chance due to human error that there will be some data validation issues between the imported data and the table structure/referential integrity rules. I would like to be able to identify exactly the field/s and record/s where these issues are occuring so that I can point them out to the user for correction.
As such the standard error essages like 'you cannot add or change a record because a related record is required in...' or 'data type mismatch in criteria or expression' are not descriptive enough to the exact location of the problem so even if I catch them I can't really give a better descriptor anyway
I am debating importing to a completely free text temporary table, then looping an insert to move one row at a time from the temp table to the properly validated table and using dbfailonerror to catch issues on individual records that need correction (the user needs to correct them I can't do this through code)
My question is whether this is a reasonable approach, is there a better/easier way, or a way to get a more specific error from access rather than using a loop?
Thanks
There are 2 ways to do this. I'm not sure what method you are using to do the import but if is as simple as copying rows from the excel sheet to the table Access will generate a Paste_errors table that will show the rows it couldn't import. The wizard will do the same thing but I think its prone to crashing.
The way I typically do it is actually have the end user use an excel template with a VBA backend that does the uploading. You can check each value conditionally and give a better descriptive alert and/or shuttle any defective rows to a temporary table for you to review.
You can do this the opposite way and do the import through Access VBA but that would be more coding since you would have to create an Excel object in code, open the sheet, etc.
I setup a Quickbooks export of a accounts receivable table by creating a User DSN on the local machine pointing to the Access file, opening an ADO recordset and looping through the rows one column at a time applying logic to each row.
Quickbooks would ask for an existing file to dump the data into so I made that a template file on the network. It sounds like your users may have to enter directly into the spreadsheet so you would have to distribute the template but the results are the same.
Example of Looping through the sheet and validating rows
Create a DSN file to the database and store it on a shared drive, this way you don't have to hardcode the string and if you need to change something, you only have to change the dsn file instead of redistributing templates.
Public Sub Upload
'Declare the main recordset that the records will be uploaded to.
Dim rstUpload as New Adodb.Recordset
'Declare a utility recordset object that you can reuse to check referential tables
Dim rstCheck as New Adodb.recordset
'Declare a utility command object that you can reuse to check referential tables
Dim SQLCommand as New Adodb.Command
'Declare the connection object to the database
Dim dataConn as New Adodb.Connection
'A tracking flag if you find something in a row that won't upload
Dim CannotUpload as Boolean
'Open the connection to the access database
dataConn.Open "\\Server\Share\Mydatabase.dsn" 'Your dsn file'
Set SQLCommand.ActiveConnection = DataConn
rst.Open "yourTable", dataConn, adOpenDynamic, adLockBatchOptimistic
For i = 1 to 100 ' Rows
*You may want to do a pass through the rows so you can get an accurate count, usually just loop through the rows until a column that must have data is blank. If your users are skipping rows that is going to be a problem.
rstUpload.AddNew
'Set the tracking Flag to False indicating you can upload this row, this will be changed if any field cannot be validated
CannotUpload = False
'First Column/Field: 'Non critical field, any value will do
rstUpload("Field1").Value = Range(i,1).Value '
'Second Column/Field has a referential integrity constraints
'Run a query against the table that has the values you are trying to validate.
SQLCommand.CommandText = "Select IDField From YourTable where ID = " & Range(i,2).Value
Set rstCheck = SQLCommand.Execute
'Check if any records were returned, if none than the value you are checking is invalid
If rstCheck.RecordCount > 0 Then 'we matched the value in the cell
rstUpload ("Field2").Value = Range(i,2).Value
else
'Design a flagging method, maybe highlight the cell in question
CannotUpload = True
End if
....continue through your columns in the same fashion, reusing the SQLCommand Object and the Utility recordset when you need to lookup something in another table
'After last column
If CannotUpload = False Then
'Nothing triggered the flag so this row is good to go
rstUpload.Update
Else
'Cannot Upload is true and this row won't be uploaded
rstUpload.Cancel
End If
Next i
dataconn.Close
set dataConn = Nothing
set rstUpload = Nothing
set rstCheck = Nothing
set SQLCommand = Nothing
I sometime extract plenty of data from a very old database (MS Access). The serialized output is stored as YAML files and these files are locally used by other scripts to speed up the process.
Sometimes we to an update of the local files by extracting the (possible) new data from the database. This extraction is quite long and I would like to avoid it if the content of the concerned tables is the same as the last extraction.
Is it possible to get a sort of signature of the state of a table, or part of the table ?
In other words this would help understanding my question:
signature = db.GetSignature('SELECT * FROM foo where bar = 1')
if local_foo.signature != signature:
local_foo = db.Extract('SELECT * FROM foo where bar = 1')
What solutions could I use?
Using triggers
If you have the luxury of controlling the insert/update/delete functionality of the original Access database, the best/safest solution would be to implement database triggers to enable tracking. That way you could easily at least store a "last modified" value or keep a table that is responsible for storing extensive tracking information.
Unfortunately Access doesn't support triggers (unless you're using 2010+, see below), but you could implement triggers using VBA in the database.
Access 2010 introduced data macros, but I don't think that's an option here!
Using the scripting language
If you can't use database triggers, perhaps you could use a workflow like this:
Execute query and get entire result (single collection)
Turn the query result/collection into a JSON string (e.g. json.dumps() in Python)
Get a hash of the JSON string (e.g. hashlib.sha1() & hashObject.hexdigest() in Python)
Compare hash against the last stored hash of the query result.
Using VBA
To keep things database-side (to avoid transferring data), it could be useful to try generating the hash using VBA in the Access database.
You could use the hashing algorithm code mentioned in this SO post: https://stackoverflow.com/a/14749855
Example:
Using this SHA1 code: https://gist.github.com/anonymous/573a875dac68a4af560d
Dim d As DAO.Database
Dim r As DAO.Recordset
Dim s As String
Set d = CurrentDb()
Set r = d.OpenRecordset("SELECT foo, bar, baz FROM foobar")
s = ""
While Not r.EOF
s = s & r!foo & "," & r!bar & "," & r!baz & ";"
r.MoveNext
Wend
r.Close
Set r = Nothing
Set d = Nothing
s = SHA1TRUNC(s)
I am using the below code to import a CSV file to my Access DB. I just have a couple of questions.
Con.Open()
Dim strSqlCommand = "SELECT F1 AS id, F2 AS firstname " &
"INTO MyNewTable " &
"FROM [Text;FMT=Delimited;HDR=No;CharacterSet=850;DATABASE=" & GlobalVariables.strDefaultDownloadPath & "].Airports.csv;"
Dim sqlCommand = New System.Data.OleDb.OleDbCommand(strSqlCommand, Con)
sqlCommand.ExecuteNonQuery()
Con.Close()
How can I change the Character Set to UTF-8? If I enter utf8 instead of 850 I get an error.
Also, the first line of my CSV file contains the column names. Can I amend the above code to take that in to account?
Regards,
Andrew
You could run into trouble trying to import and select all at once, for one thing you may not want to leave converting data types up to Access. For that, you will need 2 connections and SQL string to select from one another to insert into the other.
The connection string will need to look something like this:
"Provider=Microsoft.Jet.OLEDB.4.0; Data Source=C:\Temp\Tmp;Extended Properties='TEXT;HDR=Yes;FMT=Delimited;CharacterSet=ANSI'"
Note that just the path is listed and the Extended Properties are enclosed in ticks. If the first line has headers/field names then HDR=Yes will skip them in the result set. One of the benefits of having field names as the first line is that OleDB will use them as column names (no need for F1 As foo, F2 As bar; in fact that will fail because they have been renamed from F1, F2...).
The SQL to read from the CSV:
"SELECT * FROM filename.csv"
There are several ways to process it. You could use a reader to read a row at a time to INSERT them into the Access database. This is probably simpler: get all the data from the CSV into a DataTable and use it to INSERT into Access:
Private myDT As DataTable ' form level variable
...
Dim csvStr As String = "Provider=Microsoft.Jet.OLEDB.4.0; Data Source=C:\Temp\Tmp;Extended Properties='TEXT;HDR=Yes;FMT=Delimited;CharacterSet=ANSI'"
Dim csvSQL = "SELECT * FROM Capitals.csv" ' use YOUR file name
Using csvCn = New OleDbConnection(csvStr),
cmd As New OleDbCommand(csvSQL, csvCn)
Using da As New OleDbDataAdapter(cmd)
myDT = New DataTable
da.Fill(myDT)
End Using
End Using
For Each r As DataRow In myDT.Rows
'ToDo: INSERT INTO Access
Next
The Connection, Command and DataAdapter are all resources, so they are in USING blocks to dispose of them when we are done with them. myDT will have a collection of Rows, each with a collection of Items representing the fields from the CSV. Just loop thru the rows adding the desired items to the Access DB.
You will very likely have to do same data type conversion from String to Integer or DateTime etc.
As for the question about UTF8 - you can use the Codepage identifier. If you leave it off the connection string it will use whatever is in the Registry which may also work. For UTF8 use CharacterSet=65001.
The company that I work in has a new project. two of the employees will visit each store to record some information. The project is simple, We just want to insert that data, which has been collected by the two employee, to a sql database.
Unfortunately, the two employees will just have a laptop, which is offline. So, I can't use a synchronization technique to insert the data.
My solution is to allow the two employee to write the data in an excel sheet and then I try to find a way to insert that excel sheet into the sql database.
My question
is it possible to insert the excel sheet into the sql database?
is there any better solutions please?
Note
The database is very very simple, it is just a one table with three columns. ID, StoreName, and StoreProductsNumber
Or you can schedule query like this to run on server and auto sync:
INSERT INTO database..table(ID,StoreName,StoreProductsNumber)
SELECT
1 AS ID,
LTRIM(RTRIM([StoreName])),
LTRIM(RTRIM([StoreProductsNumber])),
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0', 'Excel 12.0;Database=C:\folder\excelfile.xls','select * from [sheet1$]')
Try This
public string GetDataTableOfExcel(string file_path)
{
using (OleDbConnection conn = new OleDbConnection())
{
DataTable dt = new DataTable();
string Import_FileName = Server.MapPath(file_path);
//Import_FileName = System.IO.Path.GetDirectoryName(file_path);
string fileExtension = Path.GetExtension(Import_FileName);
if (fileExtension == ".xlsx")
conn.ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + Import_FileName + ";" + "Extended Properties='Excel 12.0 Xml;HDR=YES;'";
using (OleDbCommand comm = new OleDbCommand())
{
comm.CommandText = "Select * from [Sheet1$]";
comm.Connection = conn;
using (OleDbDataAdapter da = new OleDbDataAdapter())
{
da.SelectCommand = comm;
da.Fill(dt);
}
}
}
}
Now Your Data in DataTable. You can create insert query from datatable's data.
file_path is excel file's full path with directory name.
Through import functionality of SQL server u can achieve it...
http://www.mssqltips.com/sqlservertutorial/203/simple-way-to-import-data-into-sql-server/
This can be achieved using Import/Export tool. I will try and add a few stwps that can help you through the process.
Using SSMS, In the Object Explorer pane, right click on the database > Tasks > Import
Select Data Source as Microsoft Excel and Click on Browse to search and select your excel file. Fill the appropriate fields and click next
Destination will be populated as you invoked the tool from the target DB. Click Next
Select copy data from one or more tables or vies radio button and click next
Now select the Excel Tab that contains the required data and under the source and select the target table from the Destination column. Click on Edit mapping and verify if the columns are mapped to the correct destination tables. Click Next
Click finish to run the package and load data into SQL Server
You could also be interested in viewing this video that talks about the exact same thing.
http://www.youtube.com/watch?v=GG2N2iI0W3M
Hope this helps.
I am trying to store and retrieve data that is entered into Excel into Access. I am an Access newbie and already have an Excel program on the front end, leading me to drive the import from Excel. I have successfully figured out how to save my Excel Named Range into the desired Access table, but now I need to figure out how to pull specific data from that Access table back into Excel. I know I can simply use the Get External Data feature from Excel to import the entire Access table into Excel, but I need to be able to only import a specific portion of the table into Excel based upon a predetermined set of parameters. Is this possible to do?
As a background to the program, basically it stores data from part number runs. Not only do I need to save new part runs into an Access database (which I figured out), but I also need to be able to pull previous part number runs from Access back into Excel to perform further analysis. That is why I need to figure out how to import only a specific portion of the table. I'm not sure what code, if any, I can post since I'm basically looking for code from scratch; but if there's any code you think you need from my Excel program I'm happy to provide it. Thanks for your help. Below is the code:
Sub GetSpecData()
Application.ScreenUpdating = False
'*******************************************************************************
'Open the connection to the data source.
Set Connection = New ADODB.Connection
Connection.Open "Provider=Microsoft.Jet.OLEDB.4.0; " & "Data Source=\\Server2013\network_shared\QC SHARED\Databases\P&Q_Tracking_Data_Storage.mdb;"
'*******************************************************************************
'Create the new RecordSet.
Set Recordset = New ADODB.Recordset
With Recordset
'Define the appropriate Filter(s) and notify the user of the selection criteria.
Src = "Select * from Raw_Data where Tag = 'GHI' "
Src = Src & "or Tag = 'DEF' "
Src = Src & "or Tag = 'LMN'"
.Open Source:=Src, ActiveConnection:=Connection
'Write the field names.
For ODCol = 0 To .Fields.Count - 1
Tracking.Sheets("Selected Past Data").Range("B7").Offset(0, ODCol).Value = .Fields(ODCol).Name
Next
'Write the recordset.
Tracking.Sheets("Selected Past Data").Range("B7").Offset(1, 0).CopyFromRecordset Recordset
End With
Set Recordset = Nothing
Connection.Close
Set Connection = Nothing
'*******************************************************************************
'Create and format the table from the Recordset.
With Tracking.Sheets("Selected Past Data")
DataLastRow = .Range("A" & Rows.Count).End(xlUp).row
.ListObjects.Add(xlSrcRange, Range("B7:M" & DataLastRow), , xlYes).Name = "INC2tbl"
.ListObjects("INC2tbl").ShowTotals = True
End With
Application.ScreenUpdating = True
'*******************************************************************************
End Sub
After googling the suggestion in the comment below, I have a couple questions. First, the code above seems to filter access data by three keys: GHI, DEF, and LMN. Am I interpreting that correctly? Second, where it says "Select * from Raw_Data where Tag = 'GHI' ", since that's in quotes, that's not actual code that will be executed, correct? That's simply a prompt or something like it, correct?
SELECT * FROM TABLE; OUTPUT TO TABLEName.EXL
FORMAT sql;
First, the code above seems to filter access data by three
keys: GHI, DEF, and LMN. Am I interpreting that correctly?
Second,
where it says "Select * from Raw_Data where Tag = 'GHI' ", since
that's in quotes, that's not actual code that will be executed,
correct? That's simply a prompt or something like it, correct?
First and second questions relate closely, so I'll answer as one. The basic idea here is that there is a connection to the database you defined (in the Connection.Open statement). The connection itself does nothing then that; establish a connection. The actual communication with the database is done in a specific language, named SQL (there are many dialects, but for simplicity sake, call it SQL for now). So your code in in the VBA language, but the communication with the database is done in SQL. The VBA code has to produce SQL statements (and those are strings, so you need the quotes). That's what the Src variable holds; and SQL statement. In this case:
"Select * from Raw_Data where Tag = 'GHI' or Tag = 'DEF' or Tag = 'LMN'"
I think this is not the place to teach you SQL, but there is plenty information on the net about it. What this statement does is:
select all columns from table "Raw_Data", but only those rows of data that has 'GHI', 'DEF' or 'LMN' in the Tag column.
So to receive all the data from table "part_numbers", you would have to use:
"SELECT * FROM part_numbers;"
And if you need to import only columns "col_1" and "col_2", you would use:
"SELECT col_1,col_2 FROM part_numbers;"
And if you need to import only columns "col_1" and "col_2", and only rows with "part_id" lower than 1000, you would use:
"SELECT col_1,col_2 FROM part_numbers WHERE part_id < 1000;"
That's it really. Now if your recordset (think of it as an array that holds the data you asked for) has queried the database, you can loop it to parse the data. Something like:
Set Recordset = Connection.Execute(Src)
Do until Recordset.EOF
debug.print Recordset!col_1
Recordset.movenext
loop
Again, google something like: "vba excel adodb access", and you'll hit plenty examples.
After researching SQL code formatting more I have made more sense of the subject. Thanks for the help.