How to put quotes only on strings (not on numbers) in a VBA comma-separated list? - sql

Context : I want to import in a PostgreSQL database some data that is in a Excel spreadsheet.
To do so, I have established in VBA a connection with the database, and now, I want to carry out one INSERT SQL query for each of the rows, e.g.
INSERT INTO mytable VALUES ('a','b',32,'d',17.2);
if we suppose a pgSQL table with 5 columns, 2 of them being numerical.
Using an array and the VBA join function I already figured out how to take a given line of the Excel table and convert it into a list in which every item is quoted, e.g.
('a','b','32','d','17.2');
from which I can easily write an SQL query.
The drawback of having all the items quoted is that upon inserting, the postgreSQL server will have to convert the numerical values that have been passed as strings back to numerical values again. I fear that this could impact performance, especially in situations where 50000+ lines need to be processed.
The question : In this context, I would like to find a solution to put quotes only on non-numerical values when converting a VBA array into a comma-separated list without using a For loop on the columns, which could affect performance as well.
My current VBA code to convert my horizontal Excel range into such a list is:
myArray = Application.Transpose(Application.Transpose(myRange.Value))
myList = "('" & Join(myArray, "','") & "')"
(in the above example, myArray would be a VBA array containing the values ("a","b",32,"d",17.2)).

Take a few milli-seconds to convert the array before the Join.
Dim myarray As Variant, mylist As String, i As Long
myarray = Application.Transpose(Application.Transpose(Range("a1:f1").Value2))
For i = LBound(myarray) To UBound(myarray)
If Not IsNumeric(myarray(i)) Then _
myarray(i) = Chr(39) & Trim(myarray(i)) & Chr(39)
Next i
mylist = "(" & Join(myarray, Chr(44)) & ")"
Debug.Print mylist

Use a loop with a buffer to build the SQL. You won't notice a difference on the performance. You'll also have more control over the type and you'll be able to escape the quote in case a string contains one:
Sub Test()
Dim data()
data = [{ "a",1 ; "b",2 }]
Debug.Print ToSqlInsert("MyTable (Col1, Col2)", data)
End Sub
Public Function ToSqlInsert(target As String, data()) As String
Dim sb() As String, n As Long, r As Long, c As Long
ReDim sb(0 To UBound(data, 1) * UBound(data, 2) * 2)
sb(n) = "INSERT INTO " & target & " VALUES ("
n = n + 1
For r = 1 To UBound(data, 1)
For c = 1 To UBound(data, 2)
If c > 1 Then sb(n - 1) = ","
Select Case VBA.VarType(data(r, c))
Case vbString: sb(n) = "'" & Replace$(data(r, c), "'", "''") & "'"
Case vbDate: sb(n) = Int((data(r, c) - #1/1/1970#) * 86400) ' to epoche '
Case vbEmpty: sb(n) = "NULL"
Case Else: sb(n) = data(r, c)
End Select
n = n + 2
Next
sb(n - 1) = "),("
Next
sb(n - 1) = ");"
ToSqlInsert = Join$(sb, Empty)
End Function

Related

How do I get all text from all cells to one variable?

I have a large range that I need to find all numbers that is between four and six digits long.
I know I can use regex for this but I don't want to loop each cell and check them all.
What I need is kind of selecting the range copy and paste in notepad and copy back to a variable.
This way I can regex the variable and find all matches at once.
I don't need to know where the number was found, I just need the numbers.
Is there any way to copy the values to a string like this?
Dim text As String
text = ActiveSheet.Range("C9:IQ56").Value
is not compatible datatypes.
If I use variant I get an array of the columns and cells.
My attempt to join the array is not successful either.
text = ActiveSheet.Range("C9:IQ56").Value
textstring = ""
For i = 1 To UBound(text, 1)
textstring = textstring & " " & Join(text(i))
Next i
Any help with this?
use Application Index to do each row at a time:
text = ActiveSheet.Range("C9:IQ56").Value
textstring = ""
For i = 1 To UBound(text, 1)
textstring = textstring & " " & Join(application.Index(text,i,0))
Next i
There are two problems in your code, the declaration and the dimensions of the variable. Here is what you can do:
Dim Text() As Variant
Text = ActiveSheet.Range("C9:IQ56").Value
textstring = ""
For i = 1 To UBound(Text, 1)
For j = 1 To UBound(Text, 2)
textstring = textstring & " " & Text(i, j)
Next j
Next i
Similar approach with delimiters concatenating row strings after loop
Added a Timer and the feature to use separators (delimiters) as well for rows (e.g. "|") as for columns (e.g. ","). Furthermore I demonstrate a way to join all row strings at once after loop via Application.Transpose() just for the sake of the art, though this isn't faster nor slower than #Scott Craner 's valid solution :+).
Code
Sub arr2txt()
Const SEPROWS As String = "|" ' << change to space or any other separator/delimiter
Const SEPCOLS As String = "," ' << change to space or any other separator/delimiter
Dim v
Dim textstring As String, i As Long
Dim t As Double: t = Timer ' stop watch
v = ActiveSheet.Range("C2:E2000").Value ' get data into 1-based 2-dim datafield array
For i = 1 To UBound(v, 1)
v(i, 1) = Join(Application.Index(v, i, 0), SEPCOLS)
Next i
textstring = Join(Application.Transpose(Application.Index(v, 0, 1)), SEPROWS)
Debug.Print Format(Timer - t, "0.00 seconds needed")
End Sub

Application.Match not exact value

Have a piece of code that looks for matches between 2 sheets (sheet1 is customer list and rData is copied pdf with invoices). It usually is exact match but in some cases I'm looking for 6 first characters that matches rData
Dim rData As Variant
Dim r As Variant
Dim r20 As Variant
Dim result As Variant
Dim i As Long
rData = ActiveWorkbook.Sheets(2).Range("A1:A60000")
r20 = ActiveWorkbook.Sheets(1).Range("C2:C33")
For Each r In r20
result = Application.Match(r, rData, 0)
If Not IsError(result) Then
For i = 1 To 5
If (result - i) > 0 Then
If (Left(Trim(rData(result - i, 1)), 3) = "418") Then
MsgBox "customer: " & r & ". invoice: " & rData(result - i, 1)
End If
End If
Next
For i = 1 To 15
If (result + i) > 0 Then
If (Left(Trim(rData(result + i, 1)), 3) = "418") Then
MsgBox "customer: " & r & ". invoice: " & rData(result + i, 1)
End If
End If
Next
End If
Next r
End Sub
Only part of this that is giving me a headache is this part result = Application.Match(r, rData, 0). How do it get match for not exact match?
Sample of Sheet1
This is what more or less looks like. Matching after CustomerNumber# is easy because they are the same every invoice. BUT sometimes invoice does not have it so I'm searching after CustomerName and sometimes they have uppercase letters, sometimes there is extra stuff behind it and therefore it cannot find exact match.
Hope it makes sense.
To match the customer name from your customer list to the customer name in the invoice even if it has extra characters appended, you can use the wildcard * in Match().
You also have a typo in the Match() function. r20 should be rData.
This is your code with the fixes applied:
Sub Test()
'v4
Dim rData As Variant
Dim r As Variant
Dim r20 As Variant
Dim result As Variant
Dim i As Long
rData = ActiveWorkbook.Sheets(2).Range("A1:A60000")
r20 = ActiveWorkbook.Sheets(1).Range("C2:C33")
For Each r In r20
result = Application.Match(r & "*", rData, 0) ' <~ Fixed here
If Not IsError(result) Then
For i = 1 To 5
If (result - i) > 0 Then
If (Left(Trim(rData(result - i, 1)), 3) = "418") Then
MsgBox "customer: " & r & ". invoice: " & rData(result - i, 1)
End If
End If
Next
For i = 1 To 15
If (result + i) > 0 Then
If (Left(Trim(rData(result + i, 1)), 3) = "418") Then
MsgBox "customer: " & r & ". invoice: " & rData(result + i, 1)
End If
End If
Next
End If
Next r
End Sub
Notes:
Match() is case insensitive, so it works with different capitalisations.
The data in Sheets(2) must all be text for Match() to work correctly with wildcards.
EDIT1: New better version
EDIT2: Refactored constants and made data ranges dynamic
EDIT3: Allows for any prefix to an invoice number of a fixed length
The following is a better, rewritten version of your code:
Sub MuchBetter()
'v3
Const s_InvoiceDataWorksheet As String = "Sheet2"
Const s_InvoiceDataColumn As String = "A:A"
Const s_CustomerWorksheet As String = "Sheet1"
Const s_CustomerStartCell As String = "C2"
Const s_InvoiceNumPrefix As String = "418"
Const n_InvoiceNumLength As Long = 8
Const n_InvScanStartOffset As Long = -5
Const n_InvScanEndOffset As Long = 15
Dim ƒ As Excel.WorksheetFunction: Set ƒ = Excel.WorksheetFunction ' Shortcut
With Worksheets(s_InvoiceDataWorksheet).Range(s_InvoiceDataColumn)
With .Parent.Range(.Cells(1), .Cells(Cells.Rows.Count).End(xlUp))
Dim varInvoiceDataArray As Variant
varInvoiceDataArray = ƒ.Transpose(.Cells.Value2)
End With
End With
With Worksheets(s_CustomerWorksheet).Range(s_CustomerStartCell)
With .Parent.Range(.Cells(1), .EntireColumn.Cells(Cells.Rows.Count).End(xlUp))
Dim varCustomerArray As Variant
varCustomerArray = ƒ.Transpose(.Cells.Value2)
End With
End With
Dim varCustomer As Variant
For Each varCustomer In varCustomerArray
Dim dblCustomerIndex As Double
dblCustomerIndex = Application.Match(varCustomer & "*", varInvoiceDataArray, 0)
If Not IsError(dblCustomerIndex) _
And varCustomer <> vbNullString _
Then
Dim i As Long
For i = ƒ.Max(dblCustomerIndex + n_InvScanStartOffset, 1) _
To ƒ.Min(dblCustomerIndex + n_InvScanEndOffset, UBound(varInvoiceDataArray))
Dim strInvoiceNum As String
strInvoiceNum = Right$(Trim$(varInvoiceDataArray(i)), n_InvoiceNumLength)
If (Left$(strInvoiceNum, Len(s_InvoiceNumPrefix)) = s_InvoiceNumPrefix) Then
MsgBox "customer: " & varCustomer & ". invoice: " & strInvoiceNum
End If
Next
End If
Next varCustomer
End Sub
Notes:
It is a good idea to use constants so all literal values are typed once only and kept grouped together.
Using the RVBA naming convention greatly increases the readability of the code, and reduces the likelihood of bugs.
Using long, appropriately named variables makes the code essentially self-documenting.
Using .Value2 whenever reading cell values is highly recommended (it avoids implicit casting, making it slightly faster as well as eliminating certain issues caused by the casting ).
Surprisingly, in VBA there are good reasons to put a variable declaration as close as possible to the first use of the variable. Two such reasons are 1) it improves readability, and 2) it simplifies future refactoring. Just remember that the variable is not reinitialised every time the Dim is encountered. Initialisation only occurs the first time.
The twin loops have been rolled into one according to the DRY principle.
Whilst the check for an empty customer name/number is not strictly necessary if you can guarantee it will never be so, it is good defensive programming as an empty value will cause erroneous results.
The negative index check inside the loop has been removed and replaced with the one-time use of the Max() worksheet function in the For statement.
The Min() worksheet function is also used in the For statement to avoid trying to read past the end of the array.
Always use worksheet functions on the WorksheetFunction object unless you are explicitly checking for errors, in which case use the Application object.

Merge or concatenate column by adjacent cell value

Is it possible to concatenate cells in a column dependent on values (ID) in another column, and then output as a string (possibly in another sheet as it would be cleaner)?
E.g.
ID Text
1234 a
1234 b
1234 c
4321 a
4321 b
4321 c
4321 d
Output:
1234 a b c
4321 a b c d
Issues:
Column IDs aren't in order (but can be sorted).
Different amounts of each ID
This seemed like a possible VBA solution
from How to merge rows in a column into one cell in excel?
Function JoinXL(arr As Variant, Optional delimiter As String = " ")
'arr must be a one-dimensional array.
JoinXL = Join(arr, delimiter)
End Function
Example usage:
=JoinXL(TRANSPOSE(A1:A4)," ")
So I thought maybe if INDEX and MATCH etc could be used in conjuction with TRANSPOSE it could work. Just not sure how to go about it.
I can have a column of the unique IDs in another sheet.
This solution is nice because it works even in cases where:
The text you're concatenating contains spaces.
You want to use a delimiter other than a space (ie a comma).
First, add "Transpose" to your custom function
Function JoinXL(arr As Variant, Optional delimiter As String = " ")
'arr must be a one-dimensional array.
arr = Application.Transpose(arr)
JoinXL = Join(arr, delimiter)
End Function
For vertical arrays, use this formula (replace comma+space with your delimiter of choice, and "{}" with your garbage characters of choice):
{=SUBSTITUTE(SUBSTITUTE(JoinXL(IF($A$2:$A$31=D3,$B$2:$B$31,"{}"),", "),"{}, ",""),", {}", "")}
For horizontal arrays, use this formula:
{=SUBSTITUTE(SUBSTITUTE(JoinXL(IF(TRANSPOSE($E$19:$AH$19)=D12,TRANSPOSE($E$20:$AH$20),"{}"),", "),"{}, ",""),", {}", "")}
Make sure you enter the formulas as array formulas (Ctrl+Shift+Enter after typing formula in cell).
While no convenient function like your cited example, consider using a dictionary of collections with the ID column as the key. Below macro assumes data begins at A2 (column headers in first row) with result outputting in D and E columns:
Sub TransposeValuesByID()
Dim i As Integer, lastrow As Integer
Dim valDict As Object
Dim innerColl As New Collection
Dim k As Variant, v As Variant
Set valDict = CreateObject("Scripting.Dictionary")
lastrow = Cells(Rows.Count, "A").End(xlUp).Row
For i = 2 To lastrow
If Range("A" & i) = Range("A" & i + 1) Then
innerColl.Add Range("B" & i)
Else
innerColl.Add Range("B" & i)
valDict.Add CStr(Range("A" & i).Value), innerColl
Set innerColl = Nothing
End If
Next i
i = 2
For Each k In valDict.keys
Range("D" & i) = k
For Each v In valDict(k)
Range("E" & i) = Trim(Range("E" & i) & " " & v)
Next v
i = i + 1
Next k
End Sub
Since all you want is space between you can use your code with a couple changes.
If your data is Vertical you need to transpose the array to make it a one dimensional array:
Function JoinXL(arr As Variant, Optional delimiter As String = " ")
'arr must be a one-dimensional array.
arr = Application.Transpose(arr)
JoinXL = Join(arr, delimiter)
End Function
If it is horizontal then use what you have.
The main change is how you call it.
Use the following array formula:
=TRIM(JoinXL(IF($A$2:$A$8=C2,$B$2:$B$8,"")," "))
being an array it needs to be confirmed with Ctrl-Shift-Enter instead of Enter when exiting edit mode. If done correctly then Excel will put {} around the formula.
The If passes an array of values or blanks depending on if the cell is equal to the criteria.
Put this in the first cell Hit Ctrl-Shift-Enter. Then drag/copy down
I've come up with a work around, that although a little cumbersome works quite nicely.
Table has to be sorted by ID.
In another sheet.
ID: (Column A)
1234
MIN REF: (Column B)
=ADDRESS(MATCH(A2,OCCRANGE,0)+1,6)
MAX REF: (Column C)
=ADDRESS(MATCH(A2,OCCRANGE)+1,6)
RANGE: (Column D)
=CONCATENATE("'OCCS COMBINED'!",B2,":",C2)
STRING: (Column E)
{=IF([#[MIN REF]]=[#[MAX REF]],INDIRECT(CONCATENATE("'OCCS COMBINED'!",[#[MIN REF]])),JoinXL(TRANSPOSE(INDIRECT(D2)), " "))}

Circular reference is user defined function VBA excel

My aim is to add the values of certain columns using a user defined function in the actual row. The columns are given in another table. I am reading the name of rows, calculating the actual value and sum them. This function called once from excel but executed 4 times. At the end it indicates a circular reference error. There is no such error in the excel file, I checked if the udf return just 42 then there is no error. First I suspected Application.Caller, but ruled out.
Function SumColumnsWithSuffix(suffix As String, rowNumber) As Integer
'can be used only in Összesíto table
Dim myTable As Excel.ListObject
Dim mySheet As Excel.Worksheet
Dim myRow As Excel.ListRow
Set mySheet = ThisWorkbook.Worksheets("összesíto")
Set myTable = mySheet.ListObjects("Számlák")
Dim columnName As String
result = 0
For Each myRow In myTable.ListRows
columnName = Intersect(myRow.Range, myTable.ListColumns("Oszlop név").Range)
columnName = "Összesíto[" & columnName & " " & suffix & "]"
'actualRow = Application.Caller.row
'rowName = actualRow & ":" & actualRow
rowName = rowNumber & ":" & rowNumber
myRowRange = Range(rowName)
actualValue = Intersect(Range(columnName), Range(rowName))
result = result + actualValue
Next myRow
SumColumnsWithSuffix = result
End Function
myRowRange is not explicitly declared (or used, actually) so it is implicitly a Variant. That means your assignment here...
myRowRange = Range(rowName)
...is also making an implicit call to .Value. That call will evaluate the results of every single cell in Range(rowName) to populate the array of Variant that it returns. If any of those cells contains a call to SumColumnsWithSuffix, you'll get a circular reference.

A more complex excel average loop

Alright, Ill try to explain this as best as I can first...
My program creates a dynamic table of information, Im trying to simplify some of the information by using a loop. To make it easy I will just say that the table starts on A1 and goes to (columnindex:rowindex*6) (that part is already done).
what I want it to do is to take the average of each column and put them somewhere else on the same excel sheet.
So Average(A1:A(rowindex*6)), Average(B1:B(rowindex*6)), and so on all the way up till Average((columnindex)1:columnindex(rowindex*6)
Now the tricky part....
Starting in A((rowindex*6)+5).... So 5 rows below the table above starting in Column A....
Have it go...
Average(A), Average(B)
Average(C), Average(D)
Average(E), Average(F)
And so on until all the columns are listed...
Where I am struggling is converting columnindex to the appropriate letters as there is always 34-40 columns
I know that I would want to do something like:
i = 5
for x = 1 to x = columnindex
dim Num2Let1 as string = a=1, b=2, c=3, so on..
dim Num2Let2 as string = a=1, b=2, c=3, so on..
xlWorkSheet2.Cells((rowindex*6)+i), 1) = "Average(" & Num2Let1.ToString & cstr(1) & ":" & Num2Let1.ToString & cstr(rowindex*6))
xlWorkSheet2.Cells((rowindex*6)+i), 2) = "Average(" & Num2Let2.ToString & cstr(1) & ":" & Num2Let2.ToString & cstr(rowindex*6))
i = i + 1
x = x + 2
loop
next
If anyone could throw some advice my way, whether this is the approach that would be most conducive to my problem, or if there is an easier solution, it would be greatly appreciated.
Thanks
Here is a function that will give you the column name based on its index:
Private Function GetExcelColumnName(columnNumber As Integer) As String
Dim col As Integer = columnNumber
Dim columnName As String = ""
Dim num As Integer
While col > 0
num = (col - 1) Mod 26
columnName = Convert.ToChar(65 + num).ToString() & columnName
col = CInt((col - num) \ 26)
End While
Return columnName
End Function
That should help you to create your average function strings.