Combining two list, only records with 1 specific unique property - vb.net

I'm combining two lists in visual basic. These lists are of a custom object. The only record I want to combine, are the once with a property doesn't match with any other object in the list so far. I've got it running. However, the first list is just 1.247 records. The second list however, is just short of 27.000.000 records. The last time I successfully merged the two list with this restriction, it took over 5 hours.
Usually I code in C#. I've had a similar problem there once, and solved it with the any function. It worked perfectly and really fast. So as you can see in the code, I tried that here too. However it takes way too long.
Private Function combineLists(list As List(Of Record), childrenlist As List(Of Record)) As List(Of Record) 'list is about 1.250 entries, childrenlist about 27.000.000
For Each r As Record In childrenlist
Dim dublicate As Boolean = list.Any(Function(record) record.materiaalnummerInfo = r.materiaalnummerInfo)
If Not dublicate Then
list.Add(r)
End If
Next
Return list
End Function
The object Record looks like this ( I wasn't sure how to make a custom object in VB, and this looks bad, but it worked):
Public Class Record
Dim materiaalnummer As String
Dim type As String 'Config or prefered
Dim materiaalstatus As String
Dim children As New List(Of String)
Public Property materiaalnummerInfo()
Get
Return materiaalnummer
End Get
Set(value)
materiaalnummer = value
End Set
End Property
Public Property typeInfo()
Get
Return type
End Get
Set(value)
type = value
End Set
End Property
Public Property materiaalstatusInfo()
Get
Return materiaalstatus
End Get
Set(value)
materiaalstatus = value
End Set
End Property
Public Property childrenInfo()
Get
Return children
End Get
Set(value)
children = value
End Set
End Property
End Class
I was hoping that someone could point me in the right direction to shorten the time needed. Thank you in advance.

I'm not 100% sure what you want the output to be such as all differences or just ones from the larger list etc but I would definitely try do it with LINQ! Basically sql for vb.net data so would something similar to this:
Dim differenceQuery = list.Except(childrenlist)
Console.WriteLine("The following lines are in list but not childrenlist")
' Execute the query.
For Each name As String In differenceQuery
Console.WriteLine(name)
Next
Also side-note i would suggest not calling one of the lists "list" as it is bad practice and is a in use name on the vb.net system
EDIT
Please try this then let me know what results come back.
Private Function combineLists(list As List(Of Record), childrenlist As List(Of Record)) As List(Of Record) 'list is about 1.250 entries, childrenlist about 27.000.000
list.AddRange(childrenlist) 'combines both lists
Dim result = From v In list Select v.materiaalnummerInfo Distinct.ToList
'result hopefully may be a list with all distinct values.
End Function
Or Don't combine them if you dont want to.

Related

Windows forms CheckedListBox issue

I am working on a desktop application developed in vb.net. I am trying to select the items in a checkedlistbox depending on the values I get from database. Below is the code to populate the checkedlistboxes
Private Sub LoadDisapprovalList()
cblFedralReasons.Items.Clear()
cblStateReasons.Items.Clear()
cblFedralReasons.DataSource = Main.DataClient.DisapprovalReasonList_Get(FedralReason)
cblFedralReasons.DisplayMember = "DisapprovalReasonTypeDesc"
cblFedralReasons.ValueMember = "DisapprovalReasonTypeGenId"
cblStateReasons.DataSource = Main.DataClient.DisapprovalReasonList_Get(StateReason)
cblStateReasons.DisplayMember = "DisapprovalReasonTypeDesc"
cblStateReasons.ValueMember = "DisapprovalReasonTypeGenId"
End Sub
After that I am trying to select the items based on the values from database. Here is the code
Private Sub LoadApplicationDisapprovalReasons()
Dim lstApplicationDisapprovalReasons As New List(Of DataService.usp_ApplicationDisapprovalReason_Get_Result)
lstApplicationDisapprovalReasons = Main.DataClient.ApplicationDisapprovalReason_Get(_SeqID)
If lstApplicationDisapprovalReasons.Count > 0 Then
For Each item In lstApplicationDisapprovalReasons
Dim selectedDisapprovalId As Integer = item.DisapprovalReasonTypeGenId
Select Case item.DisapprovalReasonType
Case FedralReason
Dim selectedIndex = cblFedralReasons.Items.IndexOf(selectedDisapprovalId)
cblFedralReasons.SetItemCheckState(selectedIndex, CheckState.Checked)
Case StateReason
Dim selectedIndex = cblStateReasons.Items.IndexOf(selectedDisapprovalId)
cblStateReasons.SetItemCheckState(selectedIndex, CheckState.Checked)
End Select
Next
End If
End Sub
But the problem is cblFedralReasons.Items.IndexOf always returns -1. All the data from database is coming correctly but something weird happening with checkedlistbox which I couldn't understand.
EDIT:
Also when I try to get the text of an item by using the following code it returns me name of my collections instead of the text.
cblFedralReasons.items(1).tostring
It returns
DisapprovalReasonList
and not the text of that item!
I'll try to explain what I think about this:
If cblFedralReasons has as Datasource a List(Of DataService.usp_DisapprovalReasonList), if you search a selectedDisapprovalId vía IndexOf passing an Integer on the list.... that -1 value returned, its coherent.
IndexOf, on a collection, are internally doing a Equals comparison. So you are comparing different types: an Integer vs a DataService.usp_DisapprovalReasonList.
There are many ways to get the correct object from the collection.
One idea could be do an override of object.equals in your class:
Public Overrides Function Equals(ByVal p_oAnotherObject As Object) As Boolean
If TypeOf p_oAnotherObject Is DataService.usp_DisapprovalReasonList AndAlso Me.GetType.Equals(p_oAnotherObject.GetType) Then
Return Me.DisapprovalReasonTypeGenId.Equals(DirectCast(p_oAnotherObject, DataService.usp_DisapprovalReasonList).DisapprovalReasonTypeGenId)
Else
Return False
End If
End Function
Assuming you have a constructor accepting an ID, you now can do this:
cblFedralReasons.Items.IndexOf(New DataService.usp_DisapprovalReasonList(selectedDisapprovalId))
and then, you will get it.
Finally, cblFedralReasons.items(1).tostring, you are getting the default GetType.Name. Do this in your class, then:
Public Overrides Function ToString() As String
Return DisapprovalReasonTypeDesc
End Function
Hope I have explained.

How Do I loop through this class once I have added items

How do i loop through this class once I add items via this method. Just I am quite new to generic lists so was wonding if someone could point me in right direction in datatables im used to doing the following:
For Each thisentry In dt.rows
Next
What do I use in collections
Calling Code
Calling this in my delciarations of main class
Dim infoNoProductAvail As List(Of infoProductsNotFound) = New List(Of infoProductsNotFound)()
this is how i am adding the files but I have checked in the routine and the count for the list is at 2 products
If medProductInfo.SKU.SKUID = 0 Then
infoNoProductAvail.Add(New infoProductsNotFound(thisenty2.Item("EAN13").ToString(), True))
End If
this is the class itselfs
Public Class infoProductsNotFound
Public Sub New(tbcode As String, notfound As Boolean)
Me.tagbarcode = tbcode
Me.notfound = notfound
End Sub
Private tagbarcode As String = String.Empty
Private notfound As Boolean
Public Property tbcode() As String
Get
Return tagbarcode
End Get
Set(ByVal value As String)
tagbarcode = value
End Set
End Property
Public Property isNotFound() As Boolean
Get
Return notfound
End Get
Set(ByVal value As Boolean)
notfound = value
End Set
End Property
End Class
Tried
I tried using the following
Function BuildExceptionsForEmail()
Dim retval As String = ""
Dim cnt As Int32 = 0
retval = "The following products are not avialable" & vbCrLf
For Each info As infoProductsNotFound In infoNoProductAvail
retval &= info.tbcode
cnt &= 1
Next
Return retval
but for some reason at this point my info noproductAvail is blank even though in the routine above its sitting at count of 2 what gives?
First I'd shrink that declaration a bit:
Dim infoNoProductAvail As New List(Of infoProductsNotFound)
Next, to iterate there are several options. First (and what you're likely most used to):
For Each info as infoProductsNotFound in infoNoProductAvail
If info.tbCode = "xyz" Then
DoSomething(info)
End If
Next
Or you might want to use lambda expressions (if you're using .Net 3.5 and above I think - might be .Net 4):
infoNoProductAvail.ForEach (Function(item) DoSomething(item))
Remember that generics are strongly typed (unlike the old VB collections) so no need to cast whatever comes out: you can access properties and methods directly.
If infoNoProductAvail(3).isNotFound Then
'Do something
End If
(Not that that is a great example, but you get the idea).
The For Each syntax is the same. It works the same way for all IEnumerable objects. The only "trick" to it is to make sure that your iterator variable is of the correct type, and also to make sure that you are iterating through the correct object.
In the case of the DataTable, you are iterating over it's Rows property. That property is an IEnumerable object containing a list of DataRow objects. Therefore, to iterate through it with For Each, you must use an iterator variable of type DataRow (or one of its base classes, such as Object).
To iterate through a generic List(Of T), the IEnumerable object is the List object itself. You don't need to go to one of it's properties. The type of the iterator needs to match the type of the items in the list:
For Each i As infoProductsNotFound In infoNoProductAvail
' ...
Next
Or:
Dim i As infoProductsNotFound
For Each i In infoNoProductAvail
' ...
Next
Or:
For Each i As Object In infoNoProductAvail
' ...
Next
Etc.

The fastest way to query a vb array for datetimes within two boundaries

I am writing a function to increase the time scale of raw calculation data with a time density of about two minutes to five minutes(and other larger scales after). There are over 100k data points held in an array that isn't in chronological order. I am looking for the fastest way to query the array and to find data within two datetimes. As the code runs every data point will need to be used only once, but will have to be read several times as the data is not in order. I have several ideas of how to do this:
Just look at all of the time values in the array to check whether they are within the two datetimes given. This will force the code to run through the entire array for each new time point ~50k times.
Create a boolean in the array with my timedata that will become true if the value has been used. This will use a boolean check of the point has been used before the datetime comparison which should be faster.
Reorganize the array into order, I am not sure how long this would take based on datetimes. It would greatly increase the time required to import data in the first place, however it could make the scaling query much faster. Any idea on vaguely the ratio of time it would take to reorder the array compared to running it out of order?
Any other suggestions are welcome.
I will add some code if people feel it is necessary. Thanks in advance.
EDIT: A few examples as requested.
Here are the definitions of the arrays.:
Dim ScaleDate(0) As Date
Dim ScaleData(0) As Double
I use redim preserve as the data is added to them with an SQL.
Here is an example of a datetime point copied from the array.
(0) = #2/12/2012 12:01:36 AM#
First, as Tim Schmelter recommended, I would use a List(Of T) instead of an array. It will likely be more efficient and will definitely be easier to work with. Second, I would recommend defining your own type which stores all the data for a single item rather than storing each property for the item in a separate list. Doing so will make it easier to modify in the future, but it will also be more efficient because you'll only need to resize one list rather than two:
Public Class MyItem
Public Property ScaleDate() As Date
Get
Return _scaleDate
End Get
Set(ByVal value As Date)
_scaleDate = value
End Set
End Property
Private _scaleDate As Date
Public Property ScaleData() As Double
Get
Return _scaleData
End Get
Set(ByVal value As Double)
_scaleData = value
End Set
End Property
Private _scaleData As Double
End Class
Private _myItems As New List(Of MyItem)()
It's hard to say which will be faster, sorting the list or searching through it. It all depends how big it is, how often it's changed, and how often you search it. So, I would recommend trying both options and seeing for yourself which works better in your scenario.
For sorting, if you have your own type, you could simply make it implement IComparable(Of T) and then call the Sort method on the list:
Public Class MyItem
Implements IComparable(Of MyItem)
Public Property ScaleDate() As Date
Get
Return _scaleDate
End Get
Set(ByVal value As Date)
_scaleDate = value
End Set
End Property
Private _scaleDate As Date
Public Property ScaleData() As Double
Get
Return _scaleData
End Get
Set(ByVal value As Double)
_scaleData = value
End Set
End Property
Private _scaleData As Double
Public Function CompareTo(ByVal other As MyItem) As Integer Implements IComparable(Of MyItem).CompareTo
Return ScaleDate.CompareTo(other.ScaleDate)
End Function
End Class
Private _myItems As New List(Of MyItem)()
'To sort the list after it's been modified:
_myItems.Sort()
You'd want to only sort the list once each time it is modified. You wouldn't want to sort it every time you search through the list. Also, sorting it, in and by itself, doesn't make searching it front-to-back any faster, so you would want to implement a find method which quickly searches through a sorted list. For instance, something along these lines should work:
Private Function FindIndex(ByVal startDate As Date) As Integer
FindIndex(startDate, 0, _myItems.Count - 1)
End Function
Private Function FindIndex(ByVal startDate As Date, ByVal startIndex As Integer, ByVal endIndex As Integer) As Integer
If endIndex >= startIndex Then
Dim midIndex As Integer = ((endIndex - startIndex) \ 2) + startIndex
If _myItems(midIndex).ScaleDate < startDate Then
Return FindIndex(startDate, midIndex, endIndex)
Else
Return FindIndex(startDate, startIndex, midIndex)
End If
Else
Return startIndex
End If
End Function
For searching through an unsorted list, I simply loop through front-to-back on the whole list and I would create a new list of all the matching items:
Dim matches As New List(Of MyItem)()
For Each item As MyItem In _myItems
If (item.ScaleDate >= startDate) And (item.ScaleDate <= endDate) Then
matches.Add(item)
End If
Next
Alternatively, if the dates on these items are mostly sequential without giant gaps between them, it may be worth using a Dictionary(Of Date, List(Of MyItem)) object to store your list of items. This would contain separate lists of items for each date, all stored in a hash table. So, to get or set a list of items for a particular day would be very fast, but to get a list of all the items in a date range, you'd have to loop through every day in the date range and get the list for that day from the dictionary and combine them into one list of matches:
Dim _days As New Dictionary(Of Date, List(Of MyItem))()
'You'd need to loop through and add each item with code like this:
Private Sub AddItem(ByVal item As MyItem)
Dim dayItems As List(Of MyItem) = Nothing
_days.TryGetValue(item.ScaleDate, dayItems)
If dayItems Is Nothing Then
dayItems = New List(Of MyItem)()
_days(item.ScaleDate) = dayItems
End If
dayItems.Add(item)
End Sub
'And then to find all the items in a date range, you could do something like this:
Private Function FindItemsInRange(ByVal startDate As Date, ByVal endDate As Date) As List(Of MyItem)
Dim matches As New List(Of MyItem)()
Dim i As Date = startDate
While i <= endDate
Dim dayItems As List(Of MyItem) = Nothing
_days.TryGetValue(i, dayItems)
If dayItems Is Nothing Then
matches.AddRange(dayItems)
End If
i = i.AddDays(1)
End While
Return matches
End Function

How can I copy an object of an unknown type in VB.net?

Rather than giving the very specific case (which I did earlier), let me give a general example. Let's say that I have a function, called callingFunction. It has one parameter, called parameter. Parameter is of an unknown type. Let us then say that I wish to copy this parameter, and return it as a new object. For example, in pseudo code, something along the lines of...
Function callingFunction(ByVal parameter As Object) As Object
Dim newObj As New Object
'newObj has the same value as parameter, but is a distinctly different object
'with a different reference
newObj = parameter
return newObj
End Function
EDIT: Additional Information
The first time I posted this question, I received only one response - I felt that perhaps I made the question too specific. I guess I will explain more, perhaps that will help. I have an ASP page with 10 tables on it. I am trying, using the VB code behind, to come up with a single solution to add new rows to any table. When the user clicks a button, a generic "add row" function should be called.
The difficulty lies in the fact that I have no guarantee of the contents of any table. A new row will have the same contents as the row above it, but given that there are 10 tables, 1 row could contain any number of objects - text boxes, check boxes, etc. So I want to create a generic object, make it of the same type as the row above it, then add it to a new cell, then to a new row, then to the table.
I've tested it thoroughly, and the only part my code is failing on lies in this dynamic generation of an object type. Hence why I asked about copying objects. Neither of the solutions posted so far work correctly, by the way. Thank you for your help so far, perhaps this additional information will make it easier to provide advice?
You can't do this in general. And it won't be a good idea, for example, if parameter is of a type which implements the singleton pattern. If parameter is of a type which supports copying, it should implement the ICloneable interface. So, your function could look like this:
Function MyFunc(ByVal parameter As Object) As Object
Dim cloneableObject As ICloneable = TryCast(parameter, ICloneable)
If Not cloneableObject Is Nothing Then
Return cloneableObject.Clone()
Else
Return Nothing
End If
End Function
You could implement something like this:
Dim p1 As Person = New Person("Tim")
Dim p2 As Object = CloneObject(p1)
Dim sameRef As Boolean = p2 Is p1 'false'
Private Function CloneObject(ByVal o As Object) As Object
Dim retObject As Object
Try
Dim objType As Type = o.GetType
Dim properties() As Reflection.PropertyInfo = objType.GetProperties
retObject = objType.InvokeMember("", System.Reflection.BindingFlags.CreateInstance, Nothing, o, Nothing)
For Each propertyInfo As PropertyInfo In properties
If (propertyInfo.CanWrite) Then
propertyInfo.SetValue(retObject, propertyInfo.GetValue(o, Nothing), Nothing)
End If
Next
Catch ex As Exception
retObject = o
End Try
Return retObject
End Function
Class Person
Private _name As String
Public Property Name() As String
Get
Return _name
End Get
Set(ByVal value As String)
_name = value
End Set
End Property
Public Sub New()
End Sub
Public Sub New(ByVal name As String)
Me.Name = name
End Sub
End Class
Here's a simple class that will work for most objects (assumes at least .Net 2.0):
Public Class ObjectCloner
Public Shared Function Clone(Of T)(ByVal obj As T) As T
Using buffer As MemoryStream = New MemoryStream
Dim formatter As New BinaryFormatter
formatter.Serialize(buffer, obj)
buffer.Position = 0
Return DirectCast(formatter.Deserialize(buffer), T)
End Using
End Function
End Class

Populate an Array of Object Based on DataReader Data

I am not sure how to phrase my question properly but I want to achieve something like this.
I have a class named Products
public class Products
private ID as Integer
private Name as String
Public Property ProductID()
Get
Return ID
End Get
Set(ByVal value)
ID = value
End Set
End Property
In one of my code behind pages, I am retrieving data from an SQL Command and placing the same into a datareader object.
How would I be able to declare the class so that each row in my datareader would actually be an instance of the said class?
Like for example:
Dim myProduct() as New Product
Dim intCnt as Integer
While datareaderData.read()
intCnt += 1
myProduct(intCnt) = new Product
myProduct(intCnt).ID = datareaderData("ID")
myProduct(intCnt).Name = datareaderData("Name")
End While
When I do the same, I am getting an error "Object Reference Not Set to an Instance of an Object.
I am quite stumped on this one. Any tips greatly appreciated. Thanks.
You should use an Arraylist or -better- a generic List(of Product).
Besides i would strongly recommend to set Option Strict On in your project's Compiler Settings.
Dim products As New List(Of Product)
While datareaderData.read()
Dim nextProduct As New Product
nextProduct.ProductID = CType(datareaderData("ID"), System.Int32)
nextProduct.Name = datareaderData("Name").ToString
products.add(nextProduct)
End While