What does my program do?
As at 05 May 2021
This program was developed with the language VB.Net, the .NET framework 4.8 and with Visual Studio 2019 CE. The point of this program is to run a rudimentary database. The view is similar to a classic Internet forum—there are threads, in the threads there are different numbers of postings and in each post there are different numbers of pictures and long texts. If the thread is selected using the ComboBox, all posts with their images and texts are displayed one below the other. When you click on a specific post, only its images are displayed. Since the database is only intended for the company's products, it was decided not to use categories (e.g. images vs. videos vs. offtopic because it doesn't make any sense) and sub-categories (e.g. electrical vs. wood products).
When the program is closed, you will be asked whether the data should be saved. (still in the beta version). These data are read in when the program is loaded. If images are not found, their paths will be displayed in a window.
The user also has the option of searching through all threads and viewing the results with various sorting options. In this case, only the posts found are listed in the ListBox, and here, too, the user can select individual posts and have them displayed enlarged.
The program reads in the user data when it starts. A user can log in and, depending on his role, has certain power to make decisions. A “normal” user can create threads and posts, but only an administrator or moderator can edit and delete posts; and block a user. If you are not logged in or if you are locked, you can only read threads and posts.
The number of contributions is counted for each user. In the future, it should be possible to give a user stars.
About the classes
There is the Form1.vb class, and three other important classes: Class_Forum, Class_Post and Class_Thread. There is also the Class_User class. If a new post is created, this instance of Class_Post is added to a List(of Class_Post), which is located in Class_Thread (“The thread knows which posts it has”). Class_Post has a member ‘Made_by’, which is an instance of the Class_Users (“Every post knows which user made it”). Class_Post contains the member ‚Bilder‘ (=Images), which is a List(Of Bitmap). That is, every instance of class_post has got a List(of Bitmap).
There are also several forms for 1) editing, deleting posts, 2) for blocking or unblocking users, 3) for displaying enlarged images, 4) for logging in, 5) for displaying when images are not loading found, 6) to open the thread, 7) to post.
When the program is started, i.e. when data is read in, the thread instances and post instances are created.
To do list:
1.)
I would like, however, that only the images are in the RAM, which belong to the thread selected with the combobox1. My question is: Do I have to dispose all unnecessary images and read them in again when required? Do we get that built in?
This is my Code to load the data from the formatted txt file. I have a feeling, somewhere in here, or immediately after here, I have to do something.
Private Sub Daten_laden()
Dim Pfad As String 'file path
Using OFD1 As New CommonOpenFileDialog
OFD1.Title = "Textdatei auswählen"
OFD1.Filters.Add(New CommonFileDialogFilter("Textdatei", ".txt"))
OFD1.InitialDirectory = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments)
Dim Result As CommonFileDialogResult
Me.Invoke(Sub() Result = OFD1.ShowDialog())
If Result = CommonFileDialogResult.Ok Then
Pfad = OFD1.FileName
Else
Return
End If
End Using
Dim Pruef_Anzahl_Posts_in_dem_Thread As Integer = 0 'Check number of posts in the thread
Dim Liste_mit_den_Pfaden As List(Of String) 'List with file paths
Dim Liste_mit_den_Bildern As List(Of System.Drawing.Bitmap)
'read all Text
Dim RAT() As String = System.IO.File.ReadAllLines(Pfad, System.Text.Encoding.UTF8)
Pfade_von_nicht_gefundenen_Bildern = New List(Of String) ' Paths of not found images
For i As Integer = 3 To RAT.Length - 2 Step 1
Liste_mit_allen_Threads.Add(New Class_Thread(RAT(i)))
Me.Invoke(Sub() RaiseEvent Es_wurde_ein_neuer_Thread_eroeffnet()) 'A new thread has been opened
Pruef_Anzahl_Posts_in_dem_Thread = CInt(RAT(i + 1))
For j As Integer = (i + 2) To RAT.Length - 2 Step 1
Liste_mit_den_Pfaden = New List(Of String)
Liste_mit_den_Bildern = New List(Of Bitmap)
Dim Index As Integer
Dim Die_ID_des_Nutzers_der_den_Post_erstellt_hat As ULong = CULng(RAT(j + 4)) 'The ID of the user who created the post
For u As Integer = 0 To alle_Nutzer_Liste.Count - 1 Step 1
If alle_Nutzer_Liste(u).ID = Die_ID_des_Nutzers_der_den_Post_erstellt_hat Then
Index = u
Exit For
End If
Next
Dim neuerPost As New Class_Post(RAT(j), RAT(j + 1), CUShort(RAT(j + 2)), Liste_mit_den_Bildern, CDate(RAT(j + 3)), Liste_mit_den_Pfaden, alle_Nutzer_Liste(Index))
'how many threads are there already
Dim wie_viele_Threads_gibt_es_bereits As Integer = Liste_mit_allen_Threads.Count
Liste_mit_allen_Threads(wie_viele_Threads_gibt_es_bereits - 1).Posts_in_diesem_Thread.Add(neuerPost)
' Set the index to the last possible one in the ComboBox. This causes the program to run into the Selected Index event and SI becomes the selected index.
Me.Invoke(Sub() ComboBox1.SelectedIndex = Liste_mit_allen_Threads.Count - 1)
j += 5
Do
Liste_mit_den_Pfaden.Add(RAT(j))
If System.IO.File.Exists(RAT(j)) Then
Liste_mit_den_Bildern.Add(New Bitmap(RAT(j)))
Else
Pfade_von_nicht_gefundenen_Bildern.Add(RAT(j))
End If
j += 1
Loop Until RAT(j) = "#" ' Marker: a post is over
If RAT(j + 1) = "" AndAlso RAT(j + 2) = "" Then ' A new thread is marked with 2 blank lines one below the other.
i = (j + 2)
Exit For
End If
Next
Next
Me.Invoke(Sub() alle_Posts_in_diesem_Thread_anzeigen()) 'show all posts in this thread
If Pfade_von_nicht_gefundenen_Bildern.Count > 0 Then ' In Case something went wrong
Using FBNG As New Form_Bild_nicht_gefunden
FBNG.Datei_anzeigen(Pfade_von_nicht_gefundenen_Bildern)
FBNG.ShowDialog()
End Using
End If
End Sub
On this image, the thread is being switched using the combobox which changes the variable SI which I use often. In this example, the thread Caucasian contains 1 post which contains 2 images. Which means, (still in this moment) I don't need images form shepherds thread
It is not wise to keep all data in RAM. When your database grows, there will be a time that there is too much data.
To overcome this problem, people invented databases: the data is saved on a disk, and only the data that you request is put into memory. Smart databases will keep important data and often used values in memory, to minimize the request time.
If I look at your program, it seems that you only have to change the display after operator input. Operator input is relatively slow: you're a good typist if you can type more than 3 character per second. Usually the response time after operator input is not a problem: if you get the data within half a second, no one will complain.
For a modern computer half a second is enough time to examine a million records. In your application a database won't be a problem.
So my advice would be: start using a database and load only the data that is needed right now, instead of reading all data at startup. Only if you experience long request times, consider to load data that you expect you will need very soon.
Alas, to use a database you will need to learn something new: at least how to structure a databases. If I look at your tables, I have the impression that you already mastered this. Furthermore you'll have to learn how to add / query / update / remove data. This is usually done using SQL or software that supports LINQ, like entity framework.
It seems to me that your queries are quite limited in number: you won't have hundreds of different queries. If you already know SQL, and you don't think you need to know entity framework in the near future, I would go for accessing the database using SQL.
If you don't know SQL very much, or if you need to do an awful lot of different queries, consider to access the database using LINQ. This requires entity framework.
If you haven't got a database already, my first shot would be to use SQLight: a database in one file, fast enough for your application.
If you hide properly that you use SQLight, migrating to a smarter database if the need arises won't require a big change in your application.
class Repository
{
public long AddPost(Post post) {...} // add Post to the database, returns the Id
public long AddUser(User user) {...}
...
// fetch all Posts of a User:
public User FetchUserWithHisPosts(int userId);
// fetch Posts of a User after a certain data:
public User FetchUserWithHisPosts(int userId, DateTime startDate);
...
}
A Repository is some kind of warehouse: all you know is that you can store items in it, and later retrieve them, even after your computer is restarted.
The Repository hides how it does this: the constructor might load everything in memory (like your current application), it could also be that the repository uses SQLight, or a smarter database, or even Entity Framework.
A good way to migrate would be to first translate your current application such that everyone only accesses the data using the Repository. The Repository accesses your "in-memory data" which is in a separate class that is loaded at startup.
Later you can change the repository such that it doesn't use the "in-memory data" anymore, but accesses the database: users of your repository won't have to change.
About loading pictures
No, you don't have to load all pictures at startup. It will be fast enough to load the pictures only when shown: after all, you won't be showing 1000 pictures on the screen at once.
As a picture uses a lot of memory, it is wise to Dispose() the picture as soon as you don't need it anymore:
Image GetImage(long imageId)
{
Repository repository = new Repository();
return repository.GetImageById(imageId);
}
void DisposeImage(Image image)
{
image.Dispose();
}
You hide how you Load the image, and how you free up memory after an Image is not needed anymore. This makes it easier to change this, might the need arise later. It also makes your code easier to read and to unit test.
Hello #Harald Coppoolse,
I thank you for your detailed and understandable answer. Admittedly, I have to look at and learn those SQL (Light) and Entity procedures.
For now, I've temporarily made sure that I only load the pictures that are shown and dispose all the others. I coded this as follows:
Private Sub ComboBox1_SelectedIndexChanged(sender As Object, e As EventArgs) Handles ComboBox1.SelectedIndexChanged
If ComboBox1.SelectedIndex <> (-1) Then
SI = ComboBox1.SelectedIndex
If Programm_fertig_geladen Then
Load_the_pictures_only_when_shown_and_free_up_memory_after_an_Image_is_not_needed()
End If
alle_Posts_in_diesem_Thread_anzeigen()
End If
End Sub
Private Sub Load_the_pictures_only_when_shown_and_free_up_memory_after_an_Image_is_not_needed()
For g As Integer = 0 To Liste_mit_allen_Threads.Count - 1 Step 1
If g <> SI Then 'Diese Bilder verwerfen
For i As Integer = 0 To Liste_mit_allen_Threads(g).Posts_in_diesem_Thread.Count - 1 Step 1
For j As Integer = 0 To Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).Bilder.Count - 1 Step 1
Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).Bilder(j).Dispose()
Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).BoolArray(j) = False
Next
Dim CNT As Integer = Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).Bilder.Count
For z As Integer = (CNT - 1) To 0 Step -1
Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).Bilder.RemoveAt(z)
Next
Next
Else 'Diese Bilder laden
'für jeden Post
For i As Integer = 0 To Liste_mit_allen_Threads(g).Posts_in_diesem_Thread.Count - 1 Step 1
'die Bilder in dem Post
For j As Integer = 0 To Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).Pfade_der_Bilder.Count - 1 Step 1
Dim neu_geladenes_Bild As New Bitmap(Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).Pfade_der_Bilder(j))
If Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).BoolArray(j) = False Then
Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).Bilder.Add(neu_geladenes_Bild)
Liste_mit_allen_Threads(g).Posts_in_diesem_Thread(i).BoolArray(j) = True
End If
'End Using
Next
Next
End If
Next
End Sub
I built a Boolean array into the Class_Post, which has as many members as there are images and paths in the post. The purpose of this is to make sure that I remember which pictures are there and which are not.
When disposing of the images of a post, the respective array member is set to false.
When reloading the images of a post, the respective array member is set to True.
Dispose alone is not enough, because funnily enough, the place in the List(of Bitmap) remains occupied. You also have to completely delete the picture with RemoveAt, but in a backward running for loop, because the list gets smaller during the deletion. ;)
Using cannot not be used.
For the next part of this project that I've been working on to speed up follow up emails to clients for the office, I'm looking to grab a specific attachment from a specified filepath based on the items that the user selected on the userform. These emails will always be sending the exact same files so the less time the user has to spend manually attaching files, the better it will be. My first assumption right off the bat was that I'd need a loop to do this, so I began to do my groundwork, but now I'm generally stuck.
The first loop grabs what the user selected from the userform:
For i = 0 To List1.ListCount - 1
If List1.Selected(i) Then
Counter = Counter + 1
msg = msg & "<font style = 'background: yellow'>" & List1.List(i) & "<br />"
Else: If Counter = 0 Then End
End If
Next
And the second loop attaches the files based on the selections above:
For i = 0 To List1.ListCount - 1
If List1.Selected(i) Then
Counter = Counter + 1
.Attachments.Add List1.List(i)
Else: If Counter = 0 Then End
End If
Next
The attachments process just fine.However, the program ends up displaying the highlighted body of the message as the filepath I tried to associate with the list item:
' test files
List1.List(0) = "C:\Users\jmarkman\Dropbox\Python Practice\ex1.py"
List1.List(1) = "C:\Users\jmarkman\Dropbox\Python Practice\ex2.py"
List1.List(2) = "C:\Users\jmarkman\Dropbox\Python Practice\ex3.py"
List1.List(3) = "C:\Users\jmarkman\Dropbox\Python Practice\ex4.py"
List1.List(4) = "C:\Users\jmarkman\Dropbox\Python Practice\ex5.py"
List1.List(5) = "C:\Users\jmarkman\Dropbox\Python Practice\ex6.py"
So my question is, how do I associate these file paths with corresponding items in the listbox within the userform? I'm pretty sure that although it worked, the process changed the list items since I assigned them a different value.
I'm not sure how complex or simple this might end up being, so your time and patience are well appreciated.
People have said that one can learn best just by doing and making mistakes, but let me tell you that not having someone you can at least talk to about where to start making mistakes is incredibly frustrating. I hope anyone else who, like me, was on the cusp of solving this problem is able to look here and have their own "a-ha!" moment.
When I first asked this question, I was on the right path with choosing to put all of my filepath locations within an array and I was further along the right path by using a loop to go through it. I declared the myAttach array with a range as I did at first, but unlike whatever example I saw during my google research, I kept my second attachment loop the same but simply changed out List1.List(i) for myAttach(i). That made the script's wheels turn and performed all the functions I needed it to.
The long and short of it was that I made an array that matched the quantity and order of the items in the userform and re-used the loop I made for picking the subjectivities, I was able to associate the choices in the userform to those in the array by having the second loop go through the array. Each listing in the array had a filepath associated with it.
' example
myAttach(0) = "[filepath]"
myAttach(1) = "[filepath]"
myAttach(2) = "[filepath]"
myAttach(3) = "[filepath]"
' ... and so on ...
It helped to visualize the array and the list items as two side-by-side columns; the first item in the list is parallel to the first item in the array, and can therefore be recognized by the second loop. If a mental image isn't forthcoming, try inputting some data into two or more Excel columns to witness the relationship or review Matricies.
A caveat of this line-up approach, I found out shortly after while continuing research to figure out exactly what I did right this time would be if I had to have the items in the list in a specific order (i.e., alphabetical), but fortunately the list items are phrases and complete sentences and don't require sorting.
Happy learning and programming!
When adding a new DataRow, on of the items gets a random number from 100000000 to 999999999.
I need to loop through the data when this number is created to make sure it isn't there already. ( yes for a number this big chances are it will not ever be there) On the off chance that it is there, I need to generate a new number and loop through the data again.
I need to generate a random number until the random number does not match one in the database. I just can't seem to get my head around the logic today.
Here is the only thing I could come up with and I am not sure if it will work.
Dim TaskID As Integer
Do
intMergeID = Merge.RandomNumber()
For Each GetTask As DataRow In MasterDBDataSet.DBTasks
TaskID = GetTask.Item("MergeID")
If TaskID = intMergeID Then
Exit For
End If
Next
Loop Until intMergeID <> TaskID
It seems to be a pretty straightforward bit of logic.
You need to:
Generate the number
Check for the value in the database
If it exists go back to 1
If it doesn't exist then add it
A recursive function could do this pretty well. Some pseudo code for the function is:
Function CheckDatabase(ByVal number As Long) as Bool
Run database query
if(queryResult returns true)
generate new number
CheckDatabase(newNumber)
Else
Add to database
End Function
For the life of me, I just cannot seem to get this to work. Here is the situation: I am trying to add the already existing customer custom field (already had a definition, but no value) to an estimate that I am currently creating via QBSDK 12. So far, I can add the estimate, the custom fields to the line items, but not the custom fields belonging to the customer in the estimate header area (reserved for customer information).
Here is my attempted code for the header (doesn't work):
If Not (DE.sconProof(x) Is Nothing) Or Not (DE.sconProof(x) = "") Then
Dim DataExtModRq As IDataExtMod
DataExtModRq = requestMsgSet.AppendDataExtModRq
' DataExtModRq.ORListTxn.TxnDataExt.TxnID.SetValue(sEstID)
DataExtModRq.OwnerID.SetValue("0")
DataExtModRq.DataExtName.SetValue("Proof Required")
DataExtModRq.ORListTxn.TxnDataExt.TxnDataExtType.SetValue(ENTxnDataExtType.tdetEstimate)
DataExtModRq.ORListTxn.ListDataExt.ListObjRef.FullName.SetValue(DE.sconCompany(x))
DataExtModRq.DataExtValue.SetValue(DE.sconProof(x))
End If
Here is my working code for line items within the estimate (Does work):
If Not DE.sitemDateNeeded(i) = "" Then
Dim DataExt53 As IDataExt
DataExt53 = EstimateLineAdder.EstimateLineAdd.DataExtList.Append()
'Set field value for OwnerID
DataExt53.OwnerID.SetValue("0")
DataExt53.DataExtName.SetValue("In Hands By")
'Set field value for DataExtValue
DataExt53.DataExtValue.SetValue(DE.sitemDateNeeded(i))
End If
If Not DE.sitemSPC(i) = "" Then
Dim DataExt54 As IDataExt
DataExt54 = EstimateLineAdder.EstimateLineAdd.DataExtList.Append
DataExt54.DataExtName.SetValue("SPC")
DataExt54.DataExtValue.SetValue(DE.sitemSPC(i))
End If
The error message say's I am missing the TxnID but I am not modifying the estimate, I am creating a new one. I have tried the "IDataExt" as well but that doesn't work any better. If I need to save the newly created estimate, and then go back and add the TxnID, that would be really weird, and I'm not sure of a simple way to do that. I should be able to add data to the custom field in the header portion of the estimate without going through so much "hoopla". Please help me if you know the answer.
I figured it out...
If Not (DE.sconProof(x) Is Nothing) Or Not (DE.sconProof(x) = "") Then
Dim DataExtModRq As IDataExtMod
DataExtModRq = requestMsgSet.AppendDataExtModRq
DataExtModRq.DataExtName.SetValue("Proof Required")
DataExtModRq.DataExtValue.SetValue(DE.sconProof(x))
DataExtModRq.OwnerID.SetValue("0")
'DataExtModRq.ORListTxn.TxnDataExt.TxnDataExtType.SetValue(ENTxnDataExtType.tdetEstimate)
DataExtModRq.ORListTxn.ListDataExt.ListDataExtType.SetValue(ENListDataExtType.ldetCustomer)
DataExtModRq.ORListTxn.ListDataExt.ListObjRef.FullName.SetValue(DE.sconCompany(x))
'DataExtModRq.ORListTxn.TxnDataExt.TxnID.SetValue(sTxnID)
End If
This loop takes forever to run as the amount of items in the loop approach anything close to and over 1,000, close to like 10 minutes. This needs to run fast for amounts all the way up to like 30-40 thousand.
'Add all Loan Record Lines
Dim loans As List(Of String) = lar.CreateLoanLines()
Dim last As Integer = loans.Count - 1
For i = 0 To last
If i = last Then
s.Append(loans(i))
Else
s.AppendLine(loans(i))
End If
Next
s is a StringBuilder. The first line there
Dim loans As List(Of String) = lar.CreateLoanLines()
Runs in only a few seconds even with thousands of records. It's the actual loop that's taking a while.
How can this be optimized???
Set the initial capacity of your StringBuilder to a large value. (Ideally, large enough to contain the entire final string.) Like so:
s = new StringBuilder(loans.Count * averageExpectedStringSize)
If you don't specify a capacity, the builder will likely end up doing a large amount of internal reallocations, and this will kill performance.
You could take the special case out of the loop, so you wouldn't need to be checking it inside the loop. I would expect this to have almost no impact on performance, however.
For i = 0 To last - 1
s.AppendLine(loans(i))
Next
s.Append(loans(last))
Though, internally, the code is very similar, if you're using .NET 4, I'd consider replacing your method with a single call to String.Join:
Dim result as String = String.Join(Envionment.NewLine, lar.CreateLoanLines())
I can't see how the code you have pointed out could be slow unless:
The strings you are dealing with are huggggge (e.g. if the resulting string is 1 gigabyte).
You have another process running on your machine consuming all your clock cycles.
You haven't got enough memory in your machine.
Try stepping through the code line by line and check that the strings contain the data that you expect, and check Task Manager to see how much memory your application is using and how much free memory you have.
My guess would be that every time you're using append it's creating a new string. You seem to know how much memory you'll need, if you allocate all of the memory first and then just copy it into memory it should run much faster. Although I may be confused as to how vb.net works.
You could look at doing this another way.
Dim str As String = String.Join(Environment.NewLine, loans.ToArray)