I am learning linq to sql and I am trying to setup a lookup combo box in my web page. I was using a linq data source but I then moved my linqtosql code to it's own class library for my DAL dll (took out of app_code folder). So, I am converting the page code to be able to still have lookups driven now by a biz object.
Here's what I have done in my biz layer...
Public Class KosherTypes
Public Shared Function GetKosherTypes() As List(Of KosherTypeLookup)
Dim db As New DBDataContext
Dim types = From kt In db.KosherTypes _
Where kt.IsDeleted = False _
Order By kt.Name _
Select New KosherTypeLookup With {.Name = kt.Name, .ID = kt.KosherTypeID}
Return types.ToList
End Function
End Class
I then setup an object data source and mapped it to this class.
I have a few questions as when I was doing internet searches I didn't find anyone that seemed to be doing this and yet I know lookup tables / combo boxes are common...
Did I totally just miss something and there are better way(s) to do this?
I went with returning a list but I could have returned an IEnumerable or IQueryable. In my reading it seemed IQueryable had more functionality for linq to sql but since I am returning a simple two column list then I only need to return an IEnumerable or List. I went with a List since it's strongly typed. Did I make a good decision? AKA - Should I just have returned and IEnumerable or perhaps gone with IQueryable?
Thanks!
I'll answer in reverse order:
I wouldn't use IQueryable outside of your repos / DAL for the simple reason that since execution is deferred, you lose control of what exactly is executed (i.e., an aribtrary function could be assigned as a delegate for WHERE), making maintenance and testing much harder. I don't see an issue with you returning an IEnumberable(Of KosherTypeLookup) though.
If the lookup is a static lookup that never or rarely changes, you might want to look at a way to cache the lookup after the first use, to avoid having to hit the db every time that box is called. It really depends on your expected load, though. As always, performance/load testing will highlight where you need to optimize.
Related
I'm working with TableAdapter on VS, and have something like this:
TableAdapter.Insert(parameter1,parameter2,parameter3,...,parameterN)
And would like to know if is it possible to pass a single variable containing those parameters instead, and get something like this:
TableAdapter.Insert(all_parameters)
NOTE: I would like to know because I have sometimes a lot of parameters to pass and the line code becomes very large
In a word; no. The TableAdapter Insert method is generated in response to the GenerateDBDirect checkbox being ticked during the tableadapter wizard setup, and the direct methods for Insert/Update/Delete only exist in the format that you see
That being said, you can pass a strongly typed DataRow to the Update method, as it has overloads that are intended for persisting changes in a dataset/datatable/collection of datarow/single datarow, to a database
This means you could, instead of doing the following to insert a row:
personTA.Insert("John", "Smith", 30, "1 The Road");
Build a datarow that is state Added, and Update it, which will cause the SQL INSERT to be run:
var ro = personDataTable.NewPersonRow();
ro.FirstName = "John";
ro.LastName = "Smith";
ro.Age = 30;
ro.Address = "1, The Road";
personDataTable.AddPersonRow(ro);
//even though it's called Update, this will insert the record to the table, because it's RowState is Added
personTA.Update(ro);
This is more the way TableAdapters were intended to be used: you'd bind the datarow to text boxes, the user would type in them, and then you'd use Update() to save the changes to the database. You haven't really had to do any work in doing so, and it's hence less laborious to code, than doing something like:
personTA.Insert(_firstNameTextBox.Text, _lastNameTextBox.Text, …)
When you work with strongly typed datasets, databound textboxes etc can be created with a simple drag operation, by pulling a relevant node out of the DataSOurces window and dropping it onto the form. If you aren't using this / don't have a UI / aren't databinding then consider that there are some other ways you can make your life easier.. If your data is already in an array, then you could just set the ItemArray of the row:
var ro = personDataTable.NewPersonRow();
ro.ItemArray = array_with_person_data;
personDataTable.AddPersonRow(ro);
personTA.Update(ro);
Apologies that this is C# syntax; I've just seen that you've tagged VB. There are, however, no significant differences in the two syntaxes for this particular aspect of coding - I think mentally removing the semicolons and changing var to Dim would be all that's required to turn this particular C# into VB
Edit:
Don't forget that you're typically allowed to add your own code to anything that Microsoft provide for you in a designer. If you double click a tableadapter in the design surface you'll be taken to an empty partial class where you can add another/your own overload of Insert, perhaps one that takes an array and then simply calls the generated Insert, using all the parameters in order:
Namespace YourProgram.YourDataSetTableAdapters{
Partial Class YourTableAdapter
//ADD YOUR OWN INSERT METHOD HERE
Public Function Insert(things as Object()){
Return this.Insert( _
DirectCast(things[0] as String), _
DirectCast(things[1] as String), _
DirectCast(things[2] as Int32), _
DirectCast(things[0] as String) _
)
End Function
End Class
End Namespace
I've guessed at this VB; it might have syntax errors, hopefully you get the idea
I know, this sounds strange... But this is what I'm trying to do, how far along I am, and why I'm even doing it in the first place:
The class is configured, as an instance, with the name of the class. The context class is also available for preparing the batch class.
Pass a generic list of objects (entities, really) to a class.
That class (which can be pre-configured for this particular class) does just one thing: Adds a new entity to the backend database via DBContext.
The same class can be used for any entity described in the Context class, but each instance of the class is for just one entity class.
I want to write a blog article on my blog showing the performance of dynamically adjusting the batch size when working with EF persistence, and how constantly looking for the optimum batch size can be done.
When I say "DBContext" class, I mean a class similar to this:
Public Class CarContext
Inherits DbContext
Public Sub New()
MyBase.New("name=vASASysContext")
End Sub
Public Property Cars As DbSet(Of Car)
Protected Overrides Sub OnModelCreating(modelBuilder As DbModelBuilder)
modelBuilder.Configurations.Add(Of Car)(New CarConfiguration())
End Sub
End Class
That may sound confusing. Here is a use-case (sorta):
Need: I need to add a bunch of entities to a database. Let's call them cars, for lack of an easier example. Each entity is an instantiation of a car class, which is configured via Code First EF6 to be manipulated like any other class that is well defined in DBContext. Just like in real world classes, not all attributes are mapped to a database column.
High Level:
-I throw all the entities into a generic list, the kind of list supported by our 'batch class'.
-I create an instance of our batch class, and configure it to know that it is going to be dealing with car entities. Perhaps I even pass the context class that has the line:
-Once the batch class is ready (may only need the dbcontext instance and the name of the entity), if is given the list of entities via a function something like:
Public Function AddToDatabase(TheList as List(Of T)) As Double
The idea is pretty simple. My batch class is set to add these cars to the database, and will add the cars in the list it was given. The same rules apply to adding the entities in batch as they do when adding via DBContext normally.
All I want to happen is that the batch class itself does not need to be customized for each entity type it would deal with. Configuration is fine, but not customization.
OK... But WHY?
Adding entities via a list is easy. The secret sauce is the batch class itself. I've already written all the logic that determines the rate at which the entities are added (in something akin to "Entities per Second). It also keeps track of the best performance, and varies the batch size occasionally to verify.
The reason the function above (called AddToDatabase() for illustrative purposes) returns a double is that the double represents the amount of entities that were added per second.
Ultimately, the batch class returns to the calling method the number of entities to put in the next batch to maintain peak performance.
So my question for you all is how I can determine the entity fields, and use that to save the entities in a given list.
Can it be as simple as copying the entities? Do I even need to use reflection at all, and have the class use 'GetType' to figure out the entity class in the list (cars)?
How would you go about this?
Thank yu very much in advance for your reading this far, and your thoughtful response..
[Don't read further unless you are into this kind of thing!]
The performance of a database operation isn't linear, and is dependent on several factors (memory, CPU load, DB connectivity, etc.), and the DB is not always on the same machine as the application. It may even involve web services.
Your first instinct is to say that more entities in a single batch is best, but that is probably not true in most cases. When you add entities to a batch add, at first you see an increase in performance (increase in entities/second). But as the batch size increases, the performance may reach a maximum, then start to decrease (for a lot of reasons, not excluding environmental, such as memory). For non-memory issues, the batch performance may start to level off, and we haven't even discussed the impact of the batch on the system itself.
So in the case of a leveling off, I don't want my batch size any larger than it needs to be to be in the neighborhood of peak performance. Also, with smaller batch sizes, the class is able to evaluate the system's performance more frequently.
Being new to Code First and EF6, I can see that there must be some way to use reflection to determine how to take the given list of entities, break them apart into the entity attributes, and persist them via the EF itself.
So far, I do it this way because I need to manually configure each parameter in the INSERT INTO...
For Each d In TheList
s = "INSERT INTO BDTest (StartAddress, NumAddresses, LastAddress, Duration) VALUES (#StartAddress, #NumAddresses, #LastAddress, #Duration)"
Using cmd As SqlCommand = New SqlCommand(s, conn)
conn.Open()
cmd.Parameters.Add("#StartAddress", Data.SqlDbType.NVarChar).Value = d.StartIP
cmd.Parameters.Add("#NumAddresses", Data.SqlDbType.Int).Value = d.NumAddies
cmd.Parameters.Add("#LastAddress", Data.SqlDbType.NVarChar).Value = d.LastAddie
singleRate = CDbl(Me.TicksPerSecond / swSingle.Elapsed.Ticks)
cmd.Parameters.Add("#Duration", Data.SqlDbType.Int).Value = singleRate
cmd.ExecuteNonQuery()
conn.Close()
End Using
Next
I need to steer away in this test code from using SQL, and closer toward EF6...
What are your thoughts?
TIA!
So, there are two issues I see that you should tackle. First is your question about creating a "generic" method to add a list of entities to the database.
Public Function AddToDatabase(TheList as List(Of T)) As Double
If you are using POCO entities, then I would suggest you create an abstract base class or interface for all entity classes to inherit from/implement. I'll go with IEntity.
Public Interface IEntity
End Interface
So your Car class would be:
Public Class Car
Implements IEntity
' all your class properties here
End Class
That would handle the generic issue.
The second issue is one of batch inserts. A possible implementation of your method could be as follows. This will insert a batches of 100, modify the paramater inputs as needed. Also replace MyDbContext with the actual Type of your DbContext.
Public Function AddToDatabase(ByVal entities as List(Of IEntity)) As Double
Using scope As New TransactionScope
Dim context As MyDbContext
Try
context = new MyDbContext()
context.Configuration.AutoDetectChangesEnabled = false
Dim count = 0
For Each entityToInsert In entities
count += 1
context = AddToContext(context, entityToInsert, count, 100, true)
Next
context.SaveChanges()
Finally
If context IsNot Nothing
context.Dispose()
End If
End Try
scope.Complete()
End Using
End Function
Private Function AddToContext(ByVal context As MyDbContext, ByVal entity As IEntity, ByVal count As Integer, ByVal commitCount As Integer, ByVal recreateContext as Boolean) As MyDbContext
context.Set(entity.GetType).Add(entity)
If (count % commitCount = 0)
context.SaveChanges()
If (recreateContext)
context.Dispose()
context = new MyDbContext()
context.Configuration.AutoDetectChangesEnabled = false
End If
End If
return context
End Function
Also, please apologize if this is not 100% perfect as I mentally converted it from C# to VB.Net while typing. It should be very close.
Quick overview:
Visual Basic 2010 WinForm app pulls data from DB2. The app allows users to filter the data.
The problem:
I'm doing something wrong with my LINQ query (or object definition), as I'm not able to access fields within the dataset. Pulling data from DB2 is fine, I get that data and store it as an IEnumerable.
I plan to run this app as disconnected since its readonly for 95% of the users and it accesses 100,000+ records. Because of this, I have two datasets: 1) 'data' which is the full dataset pulled from DB2 (I don't plan on doing any modifications to it), (2) 'filteredData' which is a subset of data based on user entered filters.
Dim data As IEnumerable
Dim dataFiltered = From record in data
Select record
'Filter data based on version
Select case uxCodeVersion.Text
Case "10"
dataFiltered = From rec in dataFiltered
Where rec.
... (other parts of case statement removed)
End Select
and this is my problem. I'm expecting to see the list of fields within 'rec.' (such as rec.CodeVersion) ;however, I just receive object methods (Equals, GetHashCode, GetType, ReferenceEquals, ToString).
What simple thing am I missing?
Performance is an issue too, but I figured one problem at a time...
Thank you,
Brian.
Here is the answer as provided below.
When defining data, I need to define it to the generic list DTO. So in my case, that becomes:
Dim data As IEnumerable(Of DataAccessLayer.DiagnosisAndDiagnosisIndustryCombinedDTO)
Then when accessing the code, its the same as before, though I temporarily took out the dataFiltered field and just used data.
dataFiltered = From rec in data
Where rec.CodeVersion = uxCodeVersion.Text
From your code sample, you define data as IEnumerable. From your comment you say that your data layer returns a List(of T),
While the assignement of List(of T) to IEnumerable is valid, the type contained in the IEnumerable is Object. That's why you don't get intellisense.
You should either declare IEnumerable(of T), or do something like:
Dim data = datalayer.GetFoo()
This will cause type inference and you'll get intellisense.
Until you assign a type to IEnumerable, you won't be able to access the fields in this way. For example (c#): IEnumerable<YourType> should work.
I am taking over a project that was written by third party consultants who have already left.
I come from EF backgournd. One of the DAO class has the following which I find very hard to get my head around on details of what is exactly happening step by step. If anyone could kindly help me to understand this code section will be much appreciated.
return HibernateTemplate.Execute(
delegate(ISession hbSession) // <<--What is this code actually trying to do?
{
string queryText = "from {0} x where x.Code = :Code";
queryText = string.Format(queryText, typeof(Product));
IQuery query = hbSession.CreateQuery(queryText);
query.SetParameter("Code", productCode);
query.SetCacheable(true);
query.SetCacheRegion(CoreCacheConstants.ProductQueryCacheRegion); // <-- What is this code trying to do.
var fund = query.UniqueResult(); // <-- Is this similar to DISTINCT keyword in LINQ?
if (fund == null)
throw new ArgumentException(String.Format("No product found with productcode: {0}", productCode: ));
NHibernateUtil.Initialize(((Product)Product).Details); // <--What is this code trying to do. And where is the execute method for above queries.
return fund;
}
) as Product
Basically I am confused with delegate part and why delegate is being used instead of simple query to database. And what is the benefit of above approach.
Also I cant see any nHibernate ORM mapping xml. Does Spring.NET requires mapping files in order to pass data from/to data source?In order words how does ISession knows which database to connect to and which table to use etc
This is what in the spring documents is referred to as Classic Hibernate Usage. It is not the currently recommended approach to work with NHibernate, which is described in the chapter on object relational mappers.
The "convenient" usage of delegates here is basically done to provide the HibernateTemplate the means to manage a session and hand this managed session over to other custom methods (in this particular case an anonymous method). (I think it's an implementation of the visitor pattern, btw).
Using this approach, the classic HibernateTemplate can provide functionality to methods it doesn't "know of", such as opening and closing sessions correctly and participating in transactions.
The query is actually being executed by HibernateTemplate.Execute(myMethod); I imagine it creates and initializes a session for you, does transaction management, executes your method with the managed session and cleans everything up.
I've never used HibernateTemplate, but I'm sure it would require mapping files and a SessionFactory, so if this code is hit during execution and doesn't throw any exceptions, the configuration for those has to be there somewhere!
With respect to the questions (besides the delegate part) within your posted code: the use of NHibernateTemplate hasn't really got anything to do with it: you can just as well run this code in any piece of code where you've got hold of a valid ISession instance:
the session executes a HQL query; this query is added to the query cache. I've never used SetCacheRegion myself, but apparently it gives you "fine-grained control over query cache expiration policies".
query.UniqueResult
NHibernateUtil.Initialize
I am looking for some guidance as to what is going on when using proto-buf net with obfuscation (Dotfuscator). One half of the project is a DLL and the other is an EXE elsewhere and using proto-buf NET they exchange data flawlessly. Until I obfuscate the DLL.
At that point P-BN fails without raising an exception, returning variously a 0 length byte array or a foreshortened one depending on what I have fiddled with. The class is fairly simple (VB):
<ProtoContract(Name:="DMailer")> _
Friend Class DMailer
Private _Lic As Cert
Private _Sys As Sys
Private _LList As List(Of LItem)
..
..
End Class
There are 3 props all decorated with ProtoMember to get/set the constituent class objects. Snipped for brevity.
Again, it works GREAT until I obfuscate the DLL. Then, Dotfuscator renames each of these to null, apparently since they are all Friend, and that seems to choke proto-buff. If I exempt the class from renaming (just the class name, not props/members), it seems to work again. It makes sense that P-BN would only be able to act on objects with a proper name, though when asked to serialize a null named object, it seems like an exception might be in order.
On the other hand, much of the charm of PB-N is supposed to be serialization independent of .NET names working from attributes - at least as I understand it. Yet in this case it only seems to work with classes with names. I tried using the Name qualifier or argument as shown above, to no avail - it apparently doesnt do what I thought it might.
So, I am curious if:
a) ...I have basically surmised the problem correctly
b) ...There is some other attribute or flag that might facilitate serializing
a null named object
c) ...if there are any other insights that would help.
If I exempt all 3 or 4 classes from Dotfuscator renaming (LList is not actually implemented yet, leaving DMailer, Cert and Sys), the DLL seems to work again - at least the output is the correct size. I can live with that, though obscured names would be better: Dotfuscator (CE) either exempts them or sets the names to Null - I cant seem to find a way to force them to be renamed.
Rather than exempt 3 or 4 classes from renaming, one alternative I am considering is to simply store the Serializer output for Cert and Sys as byte arrays or Base64 strings in DMailer instead of classes. Then have the receiver Deserialize each object individually. It is kind of nice to be able to unpack just one thing and have your toys right there as if by magic though.
(many)TIA
Interesting. I confess I have never tried this scenario, but if you can walk me through your process (or better: maybe provide a basic repro example with "run this, then this, then this: boom") I'll happily investigate.
Note: the Name on ProtoContract is mainly intended for GetProto() usage; it is not needed by the core serializer, and can be omitted to reduce your exposure. Also, protobuf-net isn't interested in fields unless those fields are decorated with the attributes, so that shouldn't be an issue.
However! there's probably a workaround here that should work now; you can pre-generate a static serialization dll; for example in a separate console exe (just as a tool; I really need to wrap this in a standalone utility!)
So if you create a console exe that references your unobfuscated library and protobuf-net.dll:
var model = RuntimeTypeModel.Create();
model.Add(typeof(DMailer), true); // true means "use the attributes etc"
// and other types needed, etc
model.Compile("MailSerializer", "MailSerializer.dll");
this should write MailSerializer.dll, which you can then reference from your main code (in addition to protobuf-net), and use:
var ser = new MailSerializer(); // our pre-genereated serializer
ser.Serialize(...); // etc
Then include MailSerializer.dll in your obfuscation payload.
(this is all v2 specific, btw)
If this doesn't work, I'll need to investigate the main issue, but I'm not an obfuscation expert so could do with your repro steps.
Since there were a few upticks of interest, here is what looks like will work:
a) No form of reflection will be able to get the list of properties for an obfuscated type.
I tried walking thru all the types to find the ones with ProtoContract on it, I could find them
but the property names are all changed to a,m, b, j, g.
I also tried Me.GetType.GetProperties with the same result.
You could implement a map from the output to indicate that Employee.FirstName is now a0.j, but distributing this defeats the purpose of obfuscation.
b) What does work to a degree is to exempt the class NAME from obfuscation. Since PB-N looks for the ProtoMember attributes to get the data, you CAN obfuscate the Property/Member names, just not the CLASS/type name. If the name is something like FederalReserveLogIn, your class/type has a bullseye on it.
I have had initial success doing the following:
1) Build a simple class to store a Property Token and value. Store everything as string using ConvertFromInvariantString. Taking a tip from PBN, I used an integer for the token:
<ProtoMember(propIndex.Foo)>
Property Foo As String
An enum helps tie everything together later. Store these in a Dictionary(Of T, NameValuePair)
2) add some accessors. these can perform the type conversions for you:
Public Sub Add(ByVal Key As T, ByVal value As Object)
If _col.ContainsKey(Key) Then
_col.Remove(Key)
End If
_col.Add(Key, New TValue(value))
End Sub
Public Function GetTItem(Of TT)(key As T) As TT
If _col.ContainsKey(key) Then
Return CType(_col(key).TValue, TT)
Else
Return Nothing
End If
End Function
T is whatever key type you wish to use. Integer results in the smallest output and still allows the subscribing code to use an Enum. But it could be String.
TT is the original type:
myFoo = props.GetTItem(Of Long)(propsEnum.Foo)
3) Expose the innerlist (dictionary) to PBN and bingo, all done.
Its also very easy to add converters for Point, Rectangle, Font, Size, Color and even bitmap.
HTH