I've got a background thread which adds numbers to an array, this call happens often so i'm just curious if there's a faster method of adding the numbers to the array?
I need to add all the numbers between two values (values will be different depending on the situation), example 1 to 4 which would add 1,2,3 & 4.
However, when I make the call, it would be adding much larger arrays similar to 500 to 1000 etc which sometimes takes a little longer depending on how many numbers need to be added.
At the moment i'm using something similiar to this:
Dim ListOfNumbers As New List(Of Integer)
For i = 1 To 100000
If ListOfNumbers.Contains(i) = False Then
ListOfNumbers.Add(i)
End If
Next
Is there any other method that I could use which might be faster?
(Avoiding duplicate values in the array if possible)
I doubt this is driving your program's performance. Moreover, the best option may depend on how you plan to use these numbers, where you may find you actually can noticeably improve performance by using something like IEnumerable throughout your code, rather than an array.
With that in mind, I suggest this option:
int start = 1; int stop = 100000;
var list = Enumerable.Range(start, stop - start).ToArray()
Such that you can easily remove the ToArray() later if desired.
As for the existing code... You control both the List and the loop. Checking .Contains()is not necessary, and probably taking up a significant part of the execution time here. Just remove that check. You can also optimize some by pre-setting the size of the list:
Dim size As Integer = 100000
Dim ListOfNumbers As New List(Of Integer)(size)
For i = 1 To size
ListOfNumbers.Add(i)
Next
Related
I'm developing a macro using extra attachmate. we have a 3rd party tool called CSS from where macro fetches required data. Now I'm trying to arrange dates from old to new.
Firstly I have changed date(10/01/14) to number(140110) and then extracted all values into array. From array is there any Function to sort the numbers or should i compare each value manually in loop?
After comparing the records from array, I have to write this data in different columns of single row.
IF my approach is incorrect then please suggest me correct way.
Thanks,
Yaswanth
Your question talks about a lot of different things. I assume that you want to sort an array of integer. To do that you can use the Sort function.
Dim arr(2) As Integer
arr(0) = 140110
arr(1) = 130110
arr(2) = 150110
Array.Sort(arr)
I'm trying to find the Min/Max value of a textbox that has 1 variable and display it to the user, so the variable changes everytime the button is clicked. How would I find the maximum value of something that is constantly changing? The trick is that I can NOT use if statements or case statements. I'm totally at a loss here.
Ok, the things that limit you.
One variable
no if/case statements
The lesson seems to revolve around using Math.Max().
Math.Max(), as we can see on MSDN returns
the larger of two 32-bit signed integers.
The one variable we are going to use needs to exist outside of the button's click event. So, just make it a class variable.
This variable will essentially store the largest value. Math.Max() returns the largest of two values... see what I am getting at here? You can pass the current largest variable as a parameter to Math.Max() without any issues.
Example:
Dim max As Integer
max = Math.Max(1, 100)
'max would be 100
max = Math.Max(max, 10)
'max would be 100
max = Math.Max(max, 1000)
'max would be 1000
I'm new to stack overflow and I'm learning a new language which is vb.net. I'm working on a first app that need the use of a database, and what i'm trying to do is taking every entries in a column, putting them in an array, suffle the array and then putting back the new array in the column of the database. Sort of a names in a hat application.. I don't have any code right now, I'm doing my lessons first and I do search many forums and I have already many pieces of the puzzle like the randomize array, database handling.. My problem is that I don't find any code to prevent data to get the same order after the shuffle for an example:
Let's pretend I have 4 entries in my database:
Sarah, james, alex, daniel.
When suffling the array, how can I prevent sarah to comes first, and or james second, etc..
If you could just give me a point to start.. As I told you I'm starting to learn this language and I don't want you guys to write the app for me but just having a little clue will be much apreciate. I check lessons online but I'm getting a little bored with begginers course and the "hello world" first app demonstration so.. I think I'm ready for the next step!
#Steven got a valid point. But I think you can even go without adding the SortIndex, because you can shuffle results directly in your query, like this (assuming your backend is MSSQL):
SELECT * FROM table1 ORDER BY NEWID()
If still necessary, here is how an array can be shuffled programmatically:
Dim a() As String = {"Sarah", "James", "Alex", "Daniel"}
Dim shuffledList As New List(Of String)
Randomize()
For i = 0 To UBound(a)
Dim allowedValues() As String = a.Except(shuffledList).ToArray
shuffledList.Add(allowedValues(Math.Floor(Rnd() * allowedValues.Count)))
Next
If you want to make sure original order is not repeated for any of the elements, use this code instead:
Dim a() As String = {"Sarah", "James", "Alex", "Daniel"}
Dim ub As Integer = UBound(a)
Dim shuffledList As New List(Of String)
Randomize()
For i = 0 To ub
Dim allowedValues() As String = a.Except(shuffledList).Except({a(i)}).ToArray
Dim randomValue As String
If i = ub - 1 And allowedValues.Contains(a(ub)) Then
randomValue = a(ub)
Else
randomValue = allowedValues(Math.Floor(Rnd() * allowedValues.Count))
End If
shuffledList.Add(randomValue)
Next
Here the most important part is where the item before the last one is forced to be the last item, if the last item was not already picked. This is to make sure last item goes somewhere except the last position, otherwise there is nothing to pick from in the last step. Overall, the sequence appears to be randomly sorted and it's guaranteed that item #1 will not be #1, #2 will not be #2 etc. The algorithm always ends in N steps, where N is the number of items.
In the first instance, why would the initial order be an invalid shuffling?
Putting that aside, the simplest method would be to clone your original list and element compare it with the result of your function and keep randomising until it isn't. Better check for 0 or 1 element lists or else you will get an infinite loop.
In SQL compliant databases, the physical order of the rows in a table is supposed to be largely irrelevant, since you can sort on anything when you select the data out of it (the physical order in SQL Server is determined/set by the clustered index). So, in your situation, where you want to persist the sort order to the database, I would recommend adding an additional column that stores the sort index of each row.
So, for instance, when sorted in alphabetical order, the data would look like this:
Name SortIndex
------ ---------
Sarah 3
James 2
Alex 0
Daniel 1
And when shuffled, the physical order wouldn't change, but the sort indexes would, for instance:
Name SortIndex
------ ---------
Sarah 2
James 0
Alex 1
Daniel 3
Thanks for your answers. I already did that script in another language, the way I achieve it is pretty simple:
Taking the same array: Sarah, James, Alex, Daniel;
I would duplicate the array;
Then every entries in the first array would pick a random entry in the second one; Sarah would go first - if she doe's pick her own name, cancel it then pick again until she pick another name;
Then retire the picked name from the second array;
Then repeat every step with the second name, third...
All this done by a loop of course.
Maybe all this could be achieve very simply in vb.net but I'm pretty sure ( without knowing much of it) that there is no "Ramdomize.PreventSameSortIndexOrder" kind of function in vb.net ;). I understand what Steven was demonstrating but could you explain me how this way it can prevent the Index IDs to come back to the same place after the suffle? I was learning to handle MS Access database with VB but if SQL is better for some reason then it won't be a problem to switch. #Neolisk Could you tell me more about this command you wrote:
SELECT * FROM table1 ORDER BY NEWID()
Thanks all!
I'm trying to create a custom script in SSIS 2008 that will loop over the selected input columns and concatenate them so they can be used to create a SHA1 hash. I'm aware of the available custom components but I'm not able to install them on our system at work.
Whilst the example posed here appears to work fine http://www.sqlservercentral.com/articles/Integration+Services+(SSIS)/69766/ when I've tested this selected only a few and not all columns I get odd results. The script only seems to work if columns selected are in sequential order. Even when they are in order, after so many records or perhaps the next buffer different MD5 hashes are generated despite the rows being exactly the same throughout my test data.
I've tried to adapt the code from the previous link along with these articles but have had no joy thus far.
http://msdn.microsoft.com/en-us/library/ms136020.aspx
http://agilebi.com/jwelch/2007/06/03/xml-transformations-part-2/
As a starting point this works fine to display the column names that I have selected to be used as inputs
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
For Each inputColumn As IDTSInputColumn100 In Me.ComponentMetaData.InputCollection(0).InputColumnCollection
MsgBox(inputColumn.Name)
Next
End Sub
Building on this I try to get the values using the code below:
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Dim column As IDTSInputColumn100
Dim rowType As Type = Row.GetType()
Dim columnValue As PropertyInfo
Dim testString As String = ""
For Each column In Me.ComponentMetaData.InputCollection(0).InputColumnCollection
columnValue = rowType.GetProperty(column.Name)
testString += columnValue.GetValue(Row, Nothing).ToString()
Next
MsgBox(testString)
End Sub
Unfortunately this does not work and I receive the following error:
I'm sure what I am trying to do is easily achievable though my limited knowledge of VB.net and in particular VB.net in SSIS, I'm struggling. I could define the column names individually as shown here http://timlaqua.com/2012/02/slowly-changing-dimensions-with-md5-hashes-in-ssis/ though I'd like to try out a dynamic method.
Your problem is trying to run ToString() on a NULL value from your database.
Try Convert.ToString(columnValue) instead, it just returns an empty string.
The input columns are not guaranteed to be in the same order each time. So you'll end up getting a different hash any time the metadata in the dataflow changes. I went through the same pain when writing exactly the same script.
Every answer on the net I've found states to build a custom component to be able to do this. No need. I relied on SSIS to generate the indexes to column names when it builds the base classes each time the script component is opened. The caveat is that any time the metadata of the data flow changes, the indexes may change and need to be updated by re-opening and closing the SSIS script component.
You will need to override ProcessInput() to get store a reference to PipelineBuffer, which isn't exposed in ProcessInputRow, where you actually need to use it to access the columns by their index rather than by name.
The list of names and associated indexes are stored in ComponentMetaData.InputCollection[0].InputColumnCollection, which needs to be iterated over and sorted to guarantee same HASH every time.
PS. I posted the answer last year but it vanished, probably because it was in C# rather than VB (kind of irrelevant in SSIS). You can find the code with all ugly details here https://gist.github.com/danieljarolim/e89ff5b41b12383c60c7#file-ssis_sha1-cs
I want to extract large amounts of data from Excel, manipulate it and put it back. I have found the best way to do this is to extract the data from an Excel Range in to a large array, change the contents on the array and write it back to the Excel Range.
I am now rewriting the application using VB.NET 2008/2010 and wish to take advantage of any new features.
Currently I have to loop through the contents of the array to find elements with certain values; also sorting large arrays is cumbersome. I am looking to use the new features, including LINQ to manipulate the data in my array.
Does anybody have any advice on the easiest ways to filter / query, sort etc. data in a large array. Also what are the reasonable limits to the size of the array?
~Many Thanks
Although you have a multi-pronged question, I've chosen to answer the part about manipulating arrays using LINQ...
Edit - LINQ against 2D array, VB.NET
System.Array (from where all arrays are derived) supports an extension method for .AsQueryable() which means you can process a 2D array with LINQ directly like so:
Option Infer On
Option Strict Off
Public Class DataProcessor
'something like this came from Excel, maybe a Range
Dim arrValues2D(,) As String = { _
{"1", "2", "3"}, _
{"100", "99", "98"}, _
{"2", "3", "4"} _
}
Private Sub FindLargeNumbers()
'LINQ against 2D array elements
Dim query = From v In arrValues2D Where (Int32.Parse(v) > 50 And Int32.Parse(v) < 101) Select CStr(v)
For Each thing As String In query
'do stuff
Console.Write(thing + ",")
Next
End Sub
End Class
Original thoughts
(pretty much deprecated since the above solution)
You could Implement IEnumerable(Of T) and IEnumerator(Of T) in Visual Basic to "flatten" your 2D array (effectively return its values one at a time as if it were a 1D array) and perform operations with the values using LINQ.
Depending on the data type of array elements, in order to write back to them, you might need to wrap each value in something so you can reference the original element instead of a copy of its value.
This is just a general plan to allow you to use LINQ and "new features" against a large efficient 2D array that you already have from Excel.
If you want any performance, you have to stick with extracting the data into arrays, manipulating the arrays, then writing back the data into the range. The performance benefit is significant. I have tested and it is hundreds of times faster.
I do all my array manipulation with custom functions. I do not believe LINQ can compare to my custom functions in speed. What LINQ does bring to the table is flexibility and readability and maintainability. If you design custom array manipulation functions with LINQ principles in mind, then you should be able to take advantage of LINQ without giving up performance. For instance, I had a Median array function. I converted it to a Median extension function that accepted a sequence of doubles then used LINQ over the array to call the Median function.
<Extension()> Public Function Median(ByVal source As IEnumerable(Of Double)) As Double
As in;
Dim m = Aggregate v In arrValues2D Into Median(CDbl(v))
I checked this out of curiosity, and I don't think LINQ is the way to go. I assume that you are reading named ranges into "rectangular" arrays of objects. If that's the case, the bad news is, a rectangular array is non-enumerable and non-queryable, which means you can't execute a LINQ query against it. See the following discussion for details:
querying 2-dimension array with LINQ I stand corrected. Thx to jdk and AMissico for taking the time to explain!