I'm trying to create a custom script in SSIS 2008 that will loop over the selected input columns and concatenate them so they can be used to create a SHA1 hash. I'm aware of the available custom components but I'm not able to install them on our system at work.
Whilst the example posed here appears to work fine http://www.sqlservercentral.com/articles/Integration+Services+(SSIS)/69766/ when I've tested this selected only a few and not all columns I get odd results. The script only seems to work if columns selected are in sequential order. Even when they are in order, after so many records or perhaps the next buffer different MD5 hashes are generated despite the rows being exactly the same throughout my test data.
I've tried to adapt the code from the previous link along with these articles but have had no joy thus far.
http://msdn.microsoft.com/en-us/library/ms136020.aspx
http://agilebi.com/jwelch/2007/06/03/xml-transformations-part-2/
As a starting point this works fine to display the column names that I have selected to be used as inputs
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
For Each inputColumn As IDTSInputColumn100 In Me.ComponentMetaData.InputCollection(0).InputColumnCollection
MsgBox(inputColumn.Name)
Next
End Sub
Building on this I try to get the values using the code below:
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Dim column As IDTSInputColumn100
Dim rowType As Type = Row.GetType()
Dim columnValue As PropertyInfo
Dim testString As String = ""
For Each column In Me.ComponentMetaData.InputCollection(0).InputColumnCollection
columnValue = rowType.GetProperty(column.Name)
testString += columnValue.GetValue(Row, Nothing).ToString()
Next
MsgBox(testString)
End Sub
Unfortunately this does not work and I receive the following error:
I'm sure what I am trying to do is easily achievable though my limited knowledge of VB.net and in particular VB.net in SSIS, I'm struggling. I could define the column names individually as shown here http://timlaqua.com/2012/02/slowly-changing-dimensions-with-md5-hashes-in-ssis/ though I'd like to try out a dynamic method.
Your problem is trying to run ToString() on a NULL value from your database.
Try Convert.ToString(columnValue) instead, it just returns an empty string.
The input columns are not guaranteed to be in the same order each time. So you'll end up getting a different hash any time the metadata in the dataflow changes. I went through the same pain when writing exactly the same script.
Every answer on the net I've found states to build a custom component to be able to do this. No need. I relied on SSIS to generate the indexes to column names when it builds the base classes each time the script component is opened. The caveat is that any time the metadata of the data flow changes, the indexes may change and need to be updated by re-opening and closing the SSIS script component.
You will need to override ProcessInput() to get store a reference to PipelineBuffer, which isn't exposed in ProcessInputRow, where you actually need to use it to access the columns by their index rather than by name.
The list of names and associated indexes are stored in ComponentMetaData.InputCollection[0].InputColumnCollection, which needs to be iterated over and sorted to guarantee same HASH every time.
PS. I posted the answer last year but it vanished, probably because it was in C# rather than VB (kind of irrelevant in SSIS). You can find the code with all ugly details here https://gist.github.com/danieljarolim/e89ff5b41b12383c60c7#file-ssis_sha1-cs
Related
G'day, everyone.
I've been banging my head against this question the whole day through today, and I haven't managed to find any answers, so I'd appreciate your help.
What I have:
An Access form containing a text field
A query which is the form's data source
A custom function called RegExp defined within a module
RegExp takes two values as input: string (obtained from a table) and pattern (obtained from the form). RegExp returns a boolean value which in turn thins out query results.
The function works perfectly fine and as expected, however, this is only the case when the user fills out the text field. If the field is left blank, no results are returned (and the function's not even getting called if that's the case).
So here's what that particular statement within the query looks like:
... AND (RegExp(tblRole.Description,Trim([Forms]![frmFindRole]![txtRegExp]))<>False) AND ...
(Basically, to sum it up, user types in a value into the text field which gets leading and trailing spaces trimmed off, converted to a regular expression inside a VBA module, and then query results get filtered based on what boolean value the function returns).
There is a number of controls on this form, and they worked prior to me adding that txtRegExp text field. Now the query only returns results if txtRegExp is filled out, and I have no idea why. I've tried adding more checks, but the query's too complicated already, and I haven't succeeded.
If additional code samples are required for an answer to be made, I'll be able to provide them tomorrow.
Thank you in advance.
P.S. Would Nz help? If yes, then how would I go about using it?
Based on the few explanations you gave in comments
Suppose that this is code triggered on the KeyUp event :
Private Sub Form_KeyUp(KeyCode As Integer, Shift As Integer)
Me.Requery
End Sub
Store the default SQL for your form's recordsource somewhere in a local variable. In this example I considered that you stored it in SQLdefault string.
Prior to requery, check if the textbox is empty and if yes change your form's recordsource SQL accordingly:
private SQLdefault as string
Private Sub Form_KeyUp(KeyCode As Integer, Shift As Integer)
Dim SQL As String
If Nz(txtRegExp, "") = "" Then
SQL = SQLdefault
SQL = Replace(SQL, "AND (RegExp(tblRole.Description,Trim([Forms]![frmFindRole]![txtRegExp]))<>False)", "")
Me.RecordSource = SQL ' Normally this is enought to requery, if not uncomment below
'Me.Requery
Else
Me.RecordSource = SQLdefault ' Normally this is enought to requery, if not uncomment below
' Me.Requery
End If
End Sub
In this example I just remove the SQL part containning :
AND (RegExp(tblRole.Description,Trim([Forms]![frmFindRole]![txtRegExp]))<>False)
Replace it by something else if that's not correct.
That's obviously not the most elegant solution but it's difficult to provide with the best solution with what you've shown.
I've managed to make it work by modifying my initial query to include a check for the value of txtRegExp.
I am still not entirely sure why it failed with a blank txtRegExp value. I have a feeling the RegExp function somehow didn't fire when provided with NULL as the second parameter.
I am very grateful to Thomas G for all the help he's provided.
I'm making a form that shows all of the parts in our database (from our master components table named TAGS). All of the values are displaying fine because it's pulling from the TAG table. The thing is, when the table was created, they never entered the descriptions of each component type.
For example, in the table, TagLabel "CON" should have description "Condensers" on the report for clarity sake (some of the tag labels aren't very intuitive).
I'm trying to use VBA code in order to format a textbox (TagDescription) next to the TagLabel box that displays the descriptions based off of what the TagLabel is. The "simple" but horribly tedious solution to this is just going through the TAG table and manually adding in a description column.
I'm trying to do something like this to avoid that:
Private Sub Report_Page()
If Me.TagLabel = "CON" Then
Set Me.TagDescription = "Condensers"
End If
End Sub
I'm sure it's just a simple syntax fix, but I've tried a couple things that don't work, such as adding .value and .text to the end of Me.TagLabel etc. Thanks for all your help in advance!
This is probably the syntax you're looking for, but I'm sure if if you'll get the effect you want. It sounds like a very ugly solution.
Private Sub Report_Page()
If TagLabel = "CON" Then
TagDescription = "Condensers"
End If
End Sub
If you're going to go to all the trouble of building a big IF block for each possible TagLabel, you should probably bite the bullet and do it the right way with a lookup table.
Just create a new column in a Tags lookup table and update the tags - much less typing then doing all the code you're talking about. Then it's easy to join the table to the report query and your textbox will fill automatically
I'm developing a macro using extra attachmate. we have a 3rd party tool called CSS from where macro fetches required data. Now I'm trying to arrange dates from old to new.
Firstly I have changed date(10/01/14) to number(140110) and then extracted all values into array. From array is there any Function to sort the numbers or should i compare each value manually in loop?
After comparing the records from array, I have to write this data in different columns of single row.
IF my approach is incorrect then please suggest me correct way.
Thanks,
Yaswanth
Your question talks about a lot of different things. I assume that you want to sort an array of integer. To do that you can use the Sort function.
Dim arr(2) As Integer
arr(0) = 140110
arr(1) = 130110
arr(2) = 150110
Array.Sort(arr)
I have a report that needs to process the data that it get from SQL before show it.
For that, I have a custom code, and a Dictionary where I push all the processed data.
My problem is that if I save the dictionary in a report variable when I export the report to Word that variable seems to be cleaned.
What is the lifecycle of the reports variables? What is the most convenient way of saving an object during the report life.
Thanks!
I have been playing around with custom code for a about 6 weeks so I can answer some parts of the question of variable lifecycle in SSRS 2008 R2.
I have report that uses a Dictionary to store totals, allows me provide some specialist subtotals for financial stuff. I have something you can check (as I can't yet comment on things).
Have you declared the variable as 'shared', this is a custom code specific keyword that doesn't translate into VB.net. It ensures the variable lives to the next page, I tested this to Excel and word seemed to work fine transferring over the variable's data.
There is a trade off however under SSRS "report on demand" engine (on web, but not on BIDS) it holds the variable and doesn't garbage collect until the cache itself is cleared. I wrote some more custom code to indicate when my parameters changed and clears the variable.
Code;
Public Shared Dim Totals As New System.Collections.Generic.Dictionary(Of String, Decimal)
Public Function WipeKeys() as Decimal 'Clear Data from Dictionary (this will clear the cached object as well)
Totals.Clear()
Return 0D
End Function
I will also to recommend overwrite the key where ever possible to ensure reduction of addition loops.
Regards,
I'm trying to create an import program from CSV.
My code is
csv = New CsvReader(New StreamReader("CSVFileLocation"), True)
Dim fieldCount As Integer = csv.FieldCount
The error message "An item with the same key has already been added." on the second line.
If I changed "HasReaders" to "False", there's no such error. But, I'm not able to get the Headers.
Could somebody help me on this, please?
FYI: I'm using Visual Studio 2010 version.
Regards,
Richard
Check that your CSV file may have duplicate column names, or multiple empty cells, in the header row?
If that's the case, try to loop through your csv object, and try rename the headers in code before calling the property FieldCount.
My guess is that the CsvReader class is going through the first row adding strings to a dictionary, and the header row has two cells with the same value (so two identically named fields). Take a look at your data and see if this is the case. Alternately, if you have access to the source code for CsvReader, you could have it handle this case by naming the second field something slightly different (e.g., by appending a "1" onto the end of its name).