I have created a sheet to extract data from a Microsoft SQL database to produce a customer report between 2 date StartDate and EndDate.
I have been playing with a few things but have not been successful in anyway. I have searched but have not been able to find anything that was what I was after or able to understand.
The problem I believe is data type of the date I am using in Excel and trying to pass it to the SQL query. I understand I need to convert this in some way to make this possible and correct.
If I manually enter dates into the query it works fine. But not practical for customer use
I am not experience with this and am just! stubbing my way through it. If someone would be so kind to me with this would be much appreciated.
Below is the code I am trying to use
Sub DataExtract()
'
DataExtract Macro
'
' Create a connection object.
Dim cni96X As ADODB.Connection
Set cni96X = New ADODB.Connection
' Set Database Range
' Provide the connection string.
Dim strConn As String
Dim Lan As Integer
Dim OS As Integer
Dim PointID As String
' Set Variables
Lan = Range("Lan").Value
OS = Range("OS").Value
PointID = Range("PointID").Value
StartDate = Range("StartDate").Value
EndDate = Range("EndDate").Value
'Use the SQL Server OLE DB Provider.
strConn = "PROVIDER=SQLOLEDB;"
'Connect to 963 database on the local server.
strConn = strConn & "DATA SOURCE=(local);INITIAL CATALOG=i96X;"
'Use an integrated login.
strConn = strConn & " INTEGRATED SECURITY=sspi;"
'Now open the connection.
cni96X.Open strConn
' Create a recordset object.
Dim rsi96X As ADODB.Recordset
Dim rsi96X1 As ADODB.Recordset
Set rsi96X = New ADODB.Recordset
Set rsi96X1 = New ADODB.Recordset
With rsi96X
' Assign the Connection object.
.ActiveConnection = cni96X
' Extract the required records1.
.Open "SELECT ModuleLabel, originalAlarmTime FROM LastAlarmDetailsByTime WHERE (os = " & OS & " And theModule = N'" & PointID & "'AND AlarmCode = N'DI=1' And lan = " & Lan & " And originalAlarmTime BETWEEN N'" & StartDate & "' AND N'" & EndDate & "') ORDER BY originalAlarmTime DESC"
' Copy the records into sheet.
Range("PointLabel, TimeCallInitiated").CopyFromRecordset rsi96X
With rsi96X1
.ActiveConnection = cni96X
' Assign the Connection object.
.Open "SELECT originalAlarmTime FROM LastAlarmDetailsByTime WHERE (os = " & OS & " And theModule = N'" & PointID & "'AND AlarmCode = N'CDI1' And lan = " & Lan & " And originalAlarmTime BETWEEN N'" & StartDate & "' AND N'" & EndDate & "')ORDER BY originalAlarmTime DESC"
' Copy the records into sheet.
Sheet1.Range("TimeCallEnded").CopyFromRecordset rsi96X1
' Tidy up
.Close
I hope this makes sense.
You cannot specify the data types, the Access database engine (formerly Jet) must guess. You can influence its guesswork by changing certain registry settings (e.g. MaxScanRows) and including IMEX=1 in the connection string. For more details, see this knowledge base article.
Here's something I wrote on the subject many years ago (if you google for "ONEDAYWHEN=0" you can see it has been widely read though perhaps not carefully enough!):
The relevant registry keys (for Jet 4.0) are in:
Hkey_Local_Machine/Software/Microsoft/Jet/4.0/Engines/Excel/
The ImportMixedTypes registry key is always read (whether it is
honored is discussed later). You can test this by changing the key to
ImportMixedTypes=OneDayWhen and trying to use the ISAM: you get the
error, "Invalid setting in Excel key of the Engines section of the
Windows Registry." The only valid values are:
ImportMixedTypes=Text
ImportMixedTypes=Majority Type
Data type is determined column by column. 'Majority Type' means a
certain number of rows (more on this later) in each column are scanned
and the data types are counted. Both a cell's value and format are
used to determine data type. The majority data type (i.e. the one with
the most rows) decides the overall data type for the entire column.
There's a bias in favor os numeric in the event of a tie. Rows from
any minority data types found that can't be cast as the majority data
type will be returned with a null value.
For ImportMixedTypes=Text, the data type for the whole column will be:
Jet (MS Access UI): 'Text' data type
DDL: VARCHAR(255)
ADO: adWChar ('a null-terminated Unicode character string')
Note that this is distinct from:
Jet (MS Access UI): 'Memo' data type
DDL: MEMO
ADO: adLongVarWChar ('a long null-terminated Unicode string value')
ImportMixedTypes=Text will curtail text at 255 characters as Memo is
cast as Text. For a column to be recognized as Memo, majority type
must be detected, meaning the majority of rows detected must contain
256 or more characters.
But how many rows are scanned for each column before is decided that
mixed and/or what the majority type is? There is a second registry
key, TypeGuessRows. This can be a value from 0-16 (decimal). A value
from 1 to 16 inclusive is the number of rows to scan. A value of zero
means all rows will be scanned.
There is one final twist. A setting of IMEX=1 in the connection
string's extended property determines whether the ImportMixedTypes
value is honored. IMEX refers to 'IMport EXport mode'. There are three
possible values. IMEX=0 and IMEX=2 result in ImportMixedTypes being
ignored and the default value of 'Majority Types' is used. IMEX=1 is
the only way to ensure ImportMixedTypes=Text is honored. The resulting
connection string might look like this:
Provider=Microsoft.Jet.OLEDB.4.0;
Data Source=C:\ db.xls;
Extended Properties='Excel 8.0;HDR=Yes;IMEX=1'
Finally, although it is mentioned in MSDN articles that MAXSCANROWS
can be used in the extended properties of the connection string to
override the TypeGuessRows registry keys, this seems to be a fallacy.
Using MAXSCANROWS=0 in this way never does anything under any
circumstances. Put another way, is has just the same effect as putting
ONEDAYWHEN=0 in the extended properties, being none (not even an
error!) The same applied to ImportMixedTypes i.e. can't be used in
the connection string to override the registry setting.
In summary, use TypeGuessRows to get Jet to detect whether a 'mixed
types' situation exists or use it to 'trick' Jet into detecting a
certain data type as being the majority type. In the event of a
'mixed types' situation being detected, use ImportMixedTypes to tell
Jet to either use the majority type or coerce all values as Text
(max 255 characters).
Try changing the date part of your SQL statement to:
"[...] originalAlarmTime BETWEEN '" & Format$(StartDate, "yyyy-mm-dd") & "' AND '" & Format$(EndDate, "yyyy-mm-dd") & "' [...]"
You might also try using a parameterized query.
Related
The sheet name is DATA on excel, using F4 properties in VBA instead of sheet1 I have changed it to sData.
My aql query is not returning any result.I need to extract a column from the Data Sheet of the same workbook in VBA.
My code is :
Sql = "Select productNumber from [shtData$] where produvtDesc = "&pDes&" "
'pDesc is a string variable that stores description.
Is this how we put the sheet name in sql query? I tried [Data$] as well but it doesn't work.
Please help.
Since you didn't add any code concerning the i am assuming that you set up the connection correctly.
use the sheet name you see in excel, not the F4 code (name). Use it with a trailing $ and square brackets like you did
i assume pDes doesn't contain a string in quotes. So, add quotes around your where-criterium
look here on how to set up an ado connection to excel
alternatively, this example of an sql onto an Excel is working even if your actual ado connection refers to another database:
SELECT * FROM [data$A1:E1000] IN 'C:\path\MyFile.xlsx' [Excel 12.0;HDR=YES;IMEX=0]
Edit: For an xls you need different properties, as described in the link above.
i assume your file is called data.xls and that it contains a sheet that is called data
this works if stored in a string and then used as connection string:
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=c:\myFolder\data.xls;
Extended Properties="Excel 8.0;HDR=YES";
and paired with this sql
Sql = "Select productNumber from [data$] where produvtDesc = '" & pDes & "'"
since you want to use a dynamic path try
myCon.ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & filepath & "\data.xls;Extended Properties=""Excel 8.0;HDR=YES"";"
They key is to escape the double quotes " with a 2nd "
I think there is a typo on your pDesc variable. In the note you've called it 'pDesc' but in the SQL command string its called 'pDes'.
Also, Is it possible that you have whatever DBMS it is that you are using to be case sensitive when referring to table names (so should therefore be [DATA$])? I know most DBMS's default to case in-sensitive, but I believe it is possible to configure it to be case sensitive during the set up process...
I'm trying to read a .DBF data base using VB.NET... the code is working until I reach a column on one of the tables that is supposed to be Decimal but it is instead filled with asterisks.
The connection string and reader I'm using go like this:
Using SDBconnection As New OleDbConnection("User ID=;" & _
"DSN=;" & _
"Cache Authentication=False;" & _
"Data Source=""" & TextBox_DBLoc.Text & """;" & _
"Provider=""VFPOLEDB"";" & _
"Collating Sequence=MACHINE;" & _
"Mask Password=False;" & _
"persist security info=False;" & _
"Mode=Share Deny None;" & _
"Extended Properties=;" & _
"Encrypt Password=False")
Dim oleReader As OleDbDataReader = New OleDbCommand("Select * from " & table, SDBconnection).ExecuteReader()
Is there some way to evaluate when the data type does not correspond and then to a cast or something like that?
The table is readable in other DBF reader software and no error is showed when read. How can I do the same?
EDIT:
The error message thrown by my code when it reaches the column with asterisks is this:
The provider could not determine the Decimal value. For example, the row was just created, the default for the Decimal column was not available, and the consumer had not yet set a new Decimal value.
There is a problem with Visual Foxpro that apparently allows the user to enter a overflowed value in numeric fields which doesn't seem to cause an issue in Visual Foxpro. However, it does cause problems when accessing the data using anything other than Visual Foxpro because it violates the settings for this field. VFPOLEDB returns these values as asterisks.
As a workaround, you could try the following query:
SELECT VAL(STR(SaleAmt)) as SaleAmt FROM MyTable
This converts the decimal column (SaleAmt) to a string and then back to a numeric value.
You also can check your data for overflowed values and modify the dbf table structure to solve this problem.
This link explains this problem
I've read through a couple similar posts, but not found a solution for this issue:
I have a form with an unbound rich text, multiline textbox named tbxNote. When the textbox is exited, I use VBA code to create an SQL string which I subsequently execute to UPDATE a table field [Note] with the value in the unbound textbox. [Note] is a "Long Text" field (from my understanding, "Long Text" is equivalent to what used to be called a "Memo" field). The backend is an Access database.
Problem is: Only the first 250 characters of what is in tbxNote get stored in the target table field [Note] even though other "Long Text" fields in other tables are accepting values much longer than 250 characters. So, it does not seem to be an issue with the field type or characteristics in the backend table.
Furthermore, if I manually open the target table and paste 350 characters into the same [Note] field in the target table, all 350 characters get stored. But, if I load up that record into the form or put the same 350 characters into the form's tbxNote textbox, only 250 characters are pulled into tbxNote or saved out to [Note].
Is there a way to store more than 250 characters in an unbound textbox using an UPDATE SQL in code?
In case it matters, here's the SQL code that I used to prove only 250 of 350 characters gets saved to the table field [Note]:
dbs.Execute "UPDATE tblSupeGenNotes " & _
"SET [NoteDate] = #" & Me.tbxNoteDate & "#, " & _
"[SupeType] = " & Chr(34) & Me.cbxSupeType & Chr(34) & ", " & _
"[SupeAlerts] = " & alrt & ", " & _
"[Note] = " & Chr(34) & String(350, "a") & Chr(34) & " " & _
"WHERE [SupeGenNoteID] = " & Me.tbxSupeGenNoteID & ";"
Of course, normally I'd have Me.tbxNote instead of String(350, "a") but the String proves that only 250 of the 350 characters get stored in the [Note] field.
I must be missing something simple, but I cannot figure it out.
Unfortunately, you posted test code works, but you FAILED to post your actual update string that fails. A common (and known) problem is if you include a function (especially aggregates) in your SQL, then you are limited to 255 characters.
In fact this can apply if you have function(s) that surrounds the unbound text box and is used in the query.
So such an update should and can work, but introduction functions into this mix can cause problems with the query processor.
If you included the actual update, then the above issue(s) likely could have been determined.
So the workarounds are:
Don’t use any “functions” directly in the SQL update string, but build up the string.
So in place of say:
Dbs.Execute "update tblTest set Notes = string(350,’a’)"
Note how above the string function is INSIDE the sql.
You can thus place the function(s) OUTSIDE of the query and thus pre-build the string - the query processor is NOT executing nor will it even see such functions.
So we can change above to as PER YOUR EXAMPLE:
Eg:
Dbs.Execute "update tblTest set Notes = ‘" & string(350,’a’) & "’"
(this is how/why your posted example works, but likely why your actual code fails). So functions can (and should) be moved out of the actual query string.
Also make sure there is NO FORMAT in the formatting for the text box, as once again this will truncate the text box to 255.
And as noted here the other suggestion is to consider using a recordset update in place of the SQL update.
Using a recordset can often remove issues of delimiters and functions then become a non issue.
So such SQL updates can work beyond 255 characters, but functions need to be evaluated in your VBA code before the query processor gets its hands on the data as per above examples.
And as noted remove any “format” you have for the text box (property sheet, format tab).
#HansUp's suggested trying a DAO recordset to update the table. That did the trick! Thank you, HansUp. HansUp requested that I post the answer, so, here is the code that worked for anyone else who comes across this thread:
Dim dbs As DAO.Database
Dim rsTable As DAO.Recordset
Dim rsQuery As DAO.Recordset
Set dbs = CurrentDb
'Open a table-type Recordset
Set rsTable = dbs.OpenRecordset("tblSupeGenNotes", dbOpenDynaset)
'Open a dynaset-type Recordset using a saved query
Set rsQuery = dbs.OpenRecordset("qrySupeGenNotes", dbOpenDynaset)
'update the values vased on the contents of the form controls
rsQuery.Edit
rsQuery![NoteDate] = Me.tbxNoteDate
rsQuery![SupeType] = Me.cbxSupeType
rsQuery![SupeAlerts] = alrt
rsQuery![Note] = Me.tbxNote
rsQuery.Update
'clean up
rsQuery.Close
rsTable.Close
Set rsQuery = Nothing
Set rsTable = Nothing
AH! Another bit to the solution is that prior to using the DAO recordset, I was pulling values from the table into a listbox and from the listbox into the form controls (instead of directly into the form controls from the table). Part of the problem (I believe) was that I was then populating the form controls from the selected item in the listbox instead of directly from the table. I believe listboxes will only allow 255 characters (250 characters?) in any single column, so, everytime I pulled the value into the textbox from the listbox, the code was pulling only the first 255 characters into the textbox. Then, when the textbox was exited, the code was updating the table with the full textbox text, but when it was pulled back into the form through the listbox, we'd be back down to 255 characters. Of course, when I switched to the DAO approach, I also switched to reading the textbox value directly from the table instead of pulling it from the listbox.
Moral: Beware of pulling Long Text values through a listbox!
Thanks to everyone who helped me solve this. Sorry for such a newbie error seeming more complicated than it was.
I assume you are using the SqlClient library. In which case, I recommend trying SqlParameters rather than creating a SQL string the way you are. With the SqlParameter you can specify the size of each parameter. http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlcommand.parameters(v=vs.110).aspx?cs-save-lang=1&cs-lang=vb#code-snippet-2 . I am a C# dev so my apologies about doing the example code below in C#:
string param = "Hello World";
byte [] encodedStr = Encoding.UTF8.GetBytes(param);
SqlParameter sqlParam = new SqlParameter();
sqlParam.Size = encodedStr.Count; // uses byte count
you could condense it by calling Encoding.UTF8.GetBytes(param).Count. Anyways, this might fix your issue
I am importing data from an Excel spreadsheet to a VB.NET DataTable. This Excel spreadsheet has a lot of garbage data in the first 18 rows, including a lot of empty cells. I ultimately remove these rows in post-processing, but I need to access the Excel file as is, without modifying it by hand at all.
I realize that setting IMEX=1 instructs the Jet engine to assume all columns are text. However, I have an issue with setting it to another value (explained more below). So, the default Jet engine column type scan wouldn't work particularly well.
I'd like to either:
Manually define column types before the import
Force Excel to scan many more rows (I believe the default is 8) to determine the column type
However, I do have an issue with idea #2. I do not have administrative rights to open regedit.exe, so I can't modify the registry using that method. I did circumvent this before by importing a key somehow, but I can't remember how I did it. So #1 would be an ideal solution, unless someone can help me carry out idea #2.
Is this possible? Currently, I'm using the following method:
If _
SetDBConnect( _
"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & filepath & _
";Extended Properties=""Excel 8.0;HDR=Yes;IMEX=1""", True) Then
dtSchema = _dh.GetOleDbSchemaTable()
If _dh.Errors <> "" Then
Throw New Exception("::LoadFileToBuffer.GetOleDbSchemaTable::" & _dh.Errors())
End If
For Each sheetRow In dtSchema.Rows
If sheetRow("TABLE_NAME").ToString() = "TOTAL_DOLLARS$" Then
totalDollars = sheetRow("TABLE_NAME").ToString()
ElseIf sheetRow("TABLE_NAME").ToString() = "TOTAL_UNITS$" Then
totalUnits = sheetRow("TABLE_NAME").ToString()
End If
Next
'Get total dollars table
sql.Append("SELECT * FROM [" & totalDollars & "]")
dtDollars = _dh.GetTable(sql.ToString())
End If
Thank you!
You should be able to say:
sql.Append("SELECT * FROM [" & totalDollars & "$A18:X95]")
Where totalDollars is a sheet name and x95 is the last valid row. You will not be able to include headers unless they are available at row 18.
I have a SSIS package to upload data from Excel file into an Sql Server 2005 table.
The excel file will have varied lines of data ranging from 20k - 30k lines.
The upload works fine, when all the data are correct. But obviously fails when there is a small problem even in a single row. Examples like mandatory values presented null, inconvertable values (data type mismatch) etc.
I want to validate the excel file before the upload and want to tell the user which row and column has got the error...
Any idea as to how to accomplish this, without consuming much time and resources.
Thanks
It might be easiest to load into a temporary table that does not have any mandatory values etc and check that before appending it to the main table.
EDIT re comment
Dim cn As ADODB.Connection
Dim rs As ADODB.Recordset
''This is not necessarily the best way to get the workbook name
''that you need
strFile = Workbooks(1).FullName
''Note that if HDR=No, F1,F2 etc are used for column names,
''if HDR=Yes, the names in the first row of the range
''can be used.
''This is the Jet 4 connection string, you can get more
''here : http://www.connectionstrings.com/excel
strCon = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & strFile _
& ";Extended Properties=""Excel 8.0;HDR=Yes;IMEX=1"";"
Set cn = CreateObject("ADODB.Connection")
Set rs = CreateObject("ADODB.Recordset")
cn.Open strCon
''Note that HDR=Yes
''Pick one:
strSQL = "SELECT Frst, Secnd FROM TheRange WHERE SomeField Is Null" ''Named range
strSQL = "SELECT Frst, Secnd FROM [Sheet1$C3:C67] WHERE Val(Secnd)=0" ''Range
strSQL = "SELECT Frst, Secnd FROM [Sheet1$] WHERE First<Date()" ''Sheet
rs.Open strSQL, cn
Sheets("Sheet2").Cells(2, 1).CopyFromRecordset rs
I have recently been working on a number of similar packages in SSIS and the only way that I have been able to get around this is to have a holding table similar Remou's suggestion.
This table is extremely generic, where all fields are NULLable and VARCHAR(255). I then have a validation Stored Procedure that checks things such as typing, the existance of data etc before I move the data into a "live" situation. Although it may not be the most elegant of solutions, it gives you alot of control of the way you check the data and also means that you shouldn't have to worry about converting the file(s) to .CSV first.