I am looking for a way to test a db field via an EF query simply to see if it is null or not, but without reading the entirety of the data. If it's not null, it's going to be huge and there are many rows, so doing this the standard way (as in the snippet below) is a very heavy operation on the db and takes ages. What I would like is to just let the user know which rows have some data in that field, and then they can choose to view it, which will then perform a single operation on that row to return just that data.
I feel like this should be possible, testing to see if something has a value or not doesn't require reading all of the data, so is there a way to achieve the desired result?
Class MyClassModel
Property HasData As Boolean?
Function Query (DB as DBContext) As IEnumerable(Of MyClassLineModel)
Dim data As IQueryable(Of MyClass) = DB.MyClasses
If HasData.HasValue Then data = From t In data Where t.Data IsNot Nothing = HasData
Dim ret = (From t In data Select New MyClassLineModel With {
.HasData = t.Data IsNot Nothing}).ToArray
Return ret
End Function
End Class
Class MyClassLineModel
Property HasData As Boolean?
End Class
p.s. This is an MVC project, hence the "model" etc. But this question is still applicable in general.
EDIT
NVM, turns out that EF does this for me. The above code converts to a db query
CASE WHEN Data IS NULL THEN 0 ELSE 1 END
which doesn't read the whole field. Response time is fine. Mods can delete the Q, or keep it for anyone else who wants to know. Thanks for all the answers guys.
You can use .Any() - if you use this on a IQueryable backed by SQL it will do something like a WHERE EXISTS in SQL which won't read all the data...
i.e.
var dataExists = _dataContext.Entity.Any(e => e.Field != NULL);
Related
Quick overview:
Visual Basic 2010 WinForm app pulls data from DB2. The app allows users to filter the data.
The problem:
I'm doing something wrong with my LINQ query (or object definition), as I'm not able to access fields within the dataset. Pulling data from DB2 is fine, I get that data and store it as an IEnumerable.
I plan to run this app as disconnected since its readonly for 95% of the users and it accesses 100,000+ records. Because of this, I have two datasets: 1) 'data' which is the full dataset pulled from DB2 (I don't plan on doing any modifications to it), (2) 'filteredData' which is a subset of data based on user entered filters.
Dim data As IEnumerable
Dim dataFiltered = From record in data
Select record
'Filter data based on version
Select case uxCodeVersion.Text
Case "10"
dataFiltered = From rec in dataFiltered
Where rec.
... (other parts of case statement removed)
End Select
and this is my problem. I'm expecting to see the list of fields within 'rec.' (such as rec.CodeVersion) ;however, I just receive object methods (Equals, GetHashCode, GetType, ReferenceEquals, ToString).
What simple thing am I missing?
Performance is an issue too, but I figured one problem at a time...
Thank you,
Brian.
Here is the answer as provided below.
When defining data, I need to define it to the generic list DTO. So in my case, that becomes:
Dim data As IEnumerable(Of DataAccessLayer.DiagnosisAndDiagnosisIndustryCombinedDTO)
Then when accessing the code, its the same as before, though I temporarily took out the dataFiltered field and just used data.
dataFiltered = From rec in data
Where rec.CodeVersion = uxCodeVersion.Text
From your code sample, you define data as IEnumerable. From your comment you say that your data layer returns a List(of T),
While the assignement of List(of T) to IEnumerable is valid, the type contained in the IEnumerable is Object. That's why you don't get intellisense.
You should either declare IEnumerable(of T), or do something like:
Dim data = datalayer.GetFoo()
This will cause type inference and you'll get intellisense.
Until you assign a type to IEnumerable, you won't be able to access the fields in this way. For example (c#): IEnumerable<YourType> should work.
I'm trying to write a function in a basic MVC (experimental project) that should be called with the following (or as close to):
datastore = myTable.FetchData({{column1, constraint1},
{column2, constraint2}, ...
{colN, conN}})
Function purpose - It queries a table with the constraints passed to it. For example,
FetchData({{Me.Fields("Price"), "<100"}, {Me.Fields("Type"), "=Component"}})
will ultimately produce the query
SELECT * FROM table a WHERE Price < 100 AND Type = "Component"
(the query production is more involved than that, involving relationships defined and so forth, but that's outside the scope of this question)
How should I write the function definition to take these parameters?
`Public Function FetchData( ??? ) As foobar
Essentially, it will be something similar to a Dictionary, since it's a list of pairs of values. However, it needs to be non-unique (it could be called to produce col1 > 1 AND col1 < 5 for eg). A two-dimensional arraylist has been considered also, but each half of the pair needs to be a specific type - the 'key' needs to be of my ColumnDefinition object type or string, the 'value' should always be a string.
What would be the best way to handle this?
Bonus question: merging the operator in with the constraint ("=component") seems ugly. Any idea about how to write the function def with the operator separate? I tried an enum but that just made it too verbose - I want this to be a fairly easy to use library.
If you are on .NET 4.0 or higher, suggest trying out the Tuple class to represent your queries. Your code may look like below.
Public Function FetchData(ByVal params As List(Of Tuple(Of ColumnDefinition, Char, String))) As foobar
Tuples are only recommended for API's under your control, where the context is obvious. If this is a public or shared API, suggest creating a named class with appropriate properties (as in Nico Schertler's comment). Then, the code may look like.
Public Function FetchData(ByVal params As List(Of MyContainerClass)) As foobar
Hope this helps.
Or according to the shape of the function call you are describing, if you are definitely using strings, and it is always {"Col", "Constraint"} then you could do this
Public Function FetchData(ByVal MultiDimensionArray(,) As String)
'this is then how you could pull the pairs of cols and constraints back out
'where n is the col constraint pair count up to (n) number
For n As Integer = 0 To MultiDimensionArray.Length - 1
Dim Col As String = MultiDimensionArray(0, n)
Dim Constraint As String = = MultiDimensionArray(1, n)
Next
End Function
I'm writing a solution where the data entities are passed to clients using datasets through a WCF service, with nHibernate as ORM.
My predecessor has written some translator classes, converting datasets to entities and vice versa. In most cases he's declared the return object as a parameter for the object.
For example:
Public Shared Function CreateEntity(ByVal ds As DataSetObject, ByVal entity As EntityObject) As EntityObject
Dim row As ds.EntityObjectRow = ds.EntityObject(0)
entity.Id = row.Id
// Etc.
Return entity
End Function
While I want to do it like:
Public Shared Function CreateEntity(ByVal ds As DataSetObject) As EntityObject
Dim row As ds.EntityObjectRow = ds.EntityObject(0)
Dim entity As New EntityObject
entity.Id = row.Id
// Etc.
Return entity
End Function
He's not with the company anymore, so I can't ask him why he's done it this way. Hence my question here. Is there some performance gain, or traceablity thing with nHibernate by using the first implementation rather than the latter?
When at the university I was always told not to pass the return object to the method, unless there was a very specific reason for it.
Please advice. :)
From the information you gave there is no problem to create the object to return from inside the method and not receive it from outside.
The only reason I can see for this is that maybe he passes the entity as a parameter with its ID already defined and doesn't change it inside, because the dataset could not know the entity's ID. But I guess that's not the case. So in my opinion go ahead and do the way you propose.
I have a SQL Server database for which I have created a strongly-typed DataSet (using the DataSet Designer in Visual Studio 2008), so all the adapters and select commands and whatnot were created for me by the wizard.
It's a small database with largely static data, so I would like to pull the contents of this DB in its entirety into my application at startup, and then grab individual pieces of data as needed using LINQ. Rather than hard-code each adapter Fill call, I would like to see if there is a way to automate this (possibly via Reflection).
So, instead of:
Dim _ds As New dsTest
dsTestTableAdapters.Table1TableAdapter.Fill(_ds.Table1)
dsTestTableAdapters.Table2TableAdapter.Fill(_ds.Table2)
<etc etc etc>
I would prefer to do something like:
Dim _ds As New dsTest
For Each tableName As String In _ds.Tables
Dim adapter as Object = <routine to grab adapter associated with the table>
adapter.Fill(tableName)
Next
Is that even remotely doable? I have done a fair amount of searching, and I wouldn't think this would be an uncommon request, but I must be either asking the wrong question, or I'm just weird to want to do this.
I will admit that I usually prefer to use unbound controls and not go with strongly-typed datasets (I prefer to write SQL directly), but my company wants to go this route, so I'm researching it. I think the idea is that as tables are added, we can just refresh the DataSet using the Designer in Visual Studio and not have to make too many underlying DB code changes.
Any help at all would be most appreciated. Thanks in advance!
I saw all above solutions and they all are good, they inspired me to find my solution,
I made a more squeezed one, I know this is an old post, but I hope it helps people in the time to come,
Private Sub FillDataSet(ByRef ds As SvuDS)
For Each t As DataTable In ds.Tables
Dim adType As Type = Assembly.GetExecutingAssembly.GetType("ProjectNameSpace.MyDSTableAdapters." & t.TableName & "TableAdapter")
'Create Adapter Instance
Dim adapter As Object = Activator.CreateInstance(adType)
'Fill the Table
adapter.GetType().GetMethod("Fill").Invoke(adapter, New Object() {t})
Next
End Sub
I could've even inferred the namespace somehow too, but I wanted it to be simple, and it worked for me
There does not exists any api that lets you do this auto-fill of the entire typed-dataset or no such code is generated within typed-dataset that supports this. It is also difficult to do this because TableAdapters do not have a common base-class that can let you do this.
If you really need to do this, you'll have to maintain a collection of DataTable type-names and TableAdapter type-names and iterate over the collection to perform the dataset fill.
So I recommend to fill dataset for each table in 'hard-code' manner as your first code examples states.
EDIT
Here's one possible solution.
Define an Interface ITableAdapter as following
public interface ITableAdapter<TDataTable> : where TDataTable : DataTable
{
TDataTable SelectAll();
}
All TableAdapters are partial classes, so you can extend them and add your custom code in partial custom class for TableAdapter. Implement ITableAdapter on each TableAdapter in your typed-data-set. so it might look like this.
public partial class YourTableAdapter : ITableAdapter<YourDataSet.YourDataTableDataTable>
{
public YourDataSet.YourDataTableDataTable SelectAll()
{
return this.GetData();
}
}
Now, you can iterate over each type in your assembly and filter those of type ITableAdapter and call SelectAll() method on each of them fill it into your Dataset. :)
EDIT2
I just came up with another elegant solution for this problem. All you need to do is define the Interface ITableAdapter to map the already implemented methods in TableAdapters that are generated by the dataset-designer.
public interface ITableAdapter<TDataTable> : where TDataTable : DataTable
{
void Fill(TDataTable);
}
And extend your TableAdapter partial classes like this.
public partial class YourTableAdapter : ITableAdapter<YourDataSet.YourDataTableDataTable>
{
//No code required here, since Fill method is already defined in TableAdapter :)
}
OK, I think I have this worked out, and just want to share the results on the off chance that there are people out there who are as insane as I am.
Basically, all the magic happens using a couple of LINQ queries and reflection. For the purposes of this example, we will assume:
There is a strongly-typed DataSet created using the DataSet Designer in Visual Studio 2008, called dsTest. A module-level variable holds an instance of this DataSet and is called (appropriately enough), m_DataSet.
The tables themselves all follow a standard SQL Server naming convention, starting with "tbl".
As a part of this wizard, a series of table adapters were created for each table inside a namespace called dsTestTableAdapters.
Each adapter is named according to the table (so if we have "tblThingy", then an adapter named "tblThingyTableAdapter" would be created).
The application is in a namespace called, for lack of anything better, MyNamespace.
Here's the routine, called on Form Load:
Private Sub PopulateDataSet()
' Get our table adapters
Dim adapters As List(Of Type) = (From t As Type In System.Reflection.Assembly.GetExecutingAssembly.GetTypes Where t.Namespace = "MyNameSpace.dsTestTableAdapters" And t.Name.StartsWith("tbl") Select t).ToList
' Initialize our dataset
m_DataSet = New dsUtility
' Get our table names
Dim tableNames as List(Of String) = (From dtbl As DataTable In m_DataSet.Tables Select dtbl.TableName).ToList
' Loop through each table name and fill the table with the corresponding adapter
For Each iter As String In tableNames
' Grab the corresponding adapter name
Dim tableName As String = iter ' Grab a copy of the table name to avoid LINQ issues with iteration variables
Dim adapterType As Type = (From t As Type In adapters Where t.Name.StartsWith(tableName) Select t).First
' Given the adapter type name, use Reflection to create an instance
Dim adapter As Object = Activator.CreateInstance(adapterType)
' Use the instance to fill the appropriate table
adapter.Fill(m_DataSet.Tables(tableName))
Next
End Sub
I tried that, and it worked like a charm. Thanks, everyone, for your help and I hope you find this useful!
I think You have only one problem !
if this Typed dataset has relations between tables, this code won't load the datatables in the correct order !
Thanks, Mike, for the very thoughtful solution. Like you, I have been searching for a way to do what you've done, and to use the same mechanism to avoid an ugly switch statement (in C#) that has to case the generated TableAdapters to perform data binding updates.
I converted your VB code to C# as noted below. I made two changes (I'm using VS Express 2010 and .NET 4.0):
I changed the StartWith("tbl") method to EndsWith("TableAdapter") since a number of generated members in the TableAdapters namespace other than just the TableAdapters begin with "tbl" (assuming you want or need to follow that convention anyway), but only the TableAdapters end with "TableAdapter."
I changed the call to the Fill method since VS tells me at build time that the object referenced by "adapter" (which does look like a TableAdapter in the debugger) doesn't have a Fill method and there is no Fill extension method. I therefore cannot perform the Fill. I'm not at all sure why this didn't work. But in any case, I changed it to explicitly find the Fill method and then invoke that method. That seems to work.
Steve
public PopulateDataSet ()
{
// Get the TableAdapters
List<Type> tableAdapters = (from t in
System.Reflection.Assembly.GetExecutingAssembly().GetTypes()
where t.Namespace == "MyNameSpace.m_DataSetTableAdapters"
&& t.Name.EndsWith("TableAdapter")
select t).ToList();
// Get the DataTable names
List<string> tableNames = (from DataTable dtbl in m_DataSet.Tables
select dtbl.TableName).ToList();
// Loop thru each table and fill it using the corresponding TableAdapter
foreach (string iter in tableNames)
{
string tableName = iter; // Stopt Linq issues with iteration vbls
Type adapterType = (from t in tableAdapters
where t.Name.StartsWith(tableName)
select t).First();
// Given the adapter type name, use Reflection to create an instance
Object adapter = Activator.CreateInstance(adapterType);
// Get a reference to the Fill method of the relevant adapter
MethodInfo method = adapter.GetType().GetMethod("Fill");
// Invoke the Fill method, passing in the relevant DataTable parameter
method.Invoke(adapter, new Object[] {m_DataSet.Tables[tableName]});
}
}
Some time ago I've found this thread and since then use this approach with success in my small project. But, recently I've found it out a bit limited. I have few queries for each table adapter in a data set with the names like "FillByContext", "FillById", "FillByName", etc., each one with a different set of parameters of different data types. All methods return tables with the same structure, but with different contents. So that I added some small "generalization" to the approach.
Private Sub MethodsAndParams(ds As DataSet,
dt As DataTable,
taParams() As Object,
taMethod As String)
Dim taType As Type = Assembly.GetExecutingAssembly.GetType(
"MyProjectName." +
ds.DataSetName +
"TableAdapters." +
dt.TableName +
"TableAdapter")
Dim ta As Object = Activator.CreateInstance(taType)
dt = ds.Tables(dt.TableName)
ta.GetType().GetMethod(taMethod).Invoke(
ta, New Object() {dt}.Union(taParams).ToArray)
End Sub
Now I can pass table adapter method names as strings and appropriate parameter sets as arrays of objects to this routine.
I want to run a query against two tables (which happen to be mapped in ActiveRecord). The query returns a result list that cannot be mapped to an ActiveRecord object (as it is custom aggregate information).
For instance
Dim query_str as string = "Select distinct d.ID, (select count(1) as exp from Sales_Leads where date_created <= :todays_date) as NbrLeads from Dealer d"
Dim q As Queries.HqlBasedQuery = New Queries.HqlBasedQuery(GetType(ICollection), query_str)
q.SetParameter("todays_date", DateTime.Today)
Dim i As ICollection = ActiveRecordMediator.ExecuteQuery(q)
What I'm looking for is simple execution of SQL, without an ActiveRecord object returned.
So, ideally, I'd be able to look at i("NbrResults") for each item in the collection.
The error I am getting is:
You have accessed an ActiveRecord
class that wasn't properly
initialized. The only explanation is
that the call to
ActiveRecordStarter.Initialize()
didn't include
System.Collections.ICollection class
Well, this was asked a long time ago but I have a working answer.
public static IList<T> ExecuteQuery<T>(HqlBasedQuery query)
where T : new()
{
query.SetResultTransformer(new NHibernate.Transform.AliasToBeanResultTransformer(typeof(T)));
var results = (ArrayList)ActiveRecordMediator.ExecuteQuery(query);
List<T> list = new List<T>(results.Count);
for (int i = 0; i < results.Count; i++)
{
list.Add((T)results[i]);
}
return list;
}
This will give you back results of type T. Type T can be anything you want. Type T needs a no argument constructor and it needs public fields or properties that match the column names or aliases in the query you build.
We do this all the time. Particularly when you want to use aggregate function in HQL to produce aggregate data.
A companion function will allow you to just pass in your query as a string as well as any positional parameters you might have:
public static IList<T> ExecuteQuery<T, U>(string hqlQuery, params object[] parameters)
where T : new()
{
return ExecuteQuery<T>(new HqlBasedQuery(typeof(U), hqlQuery, parameters));
}
Type U is any type that is a valid ActiveRecord type. It doesn't even have to be one of the types you are referencing. If you want you could replace it will some type you know is gonna be valid int he session and ditch the extra parameter.
Here was my final solution:
Dim query_str As String = "SELECT DISTINCT d.ID, count( l ) from LEAD as l join l.Dealer as d where l.DateCreated >= DATEADD(day, -30, :todays_date) GROUP BY d.ID"
Then obtain the active record session (or NHibernate, still don't even know what is returned here):
Dim sess As ISession = activerecordmediator.GetSessionFactoryHolder().CreateSession(GetType(ActiveRecordBase))
Dim q As NHibernate.IQuery = sess.CreateQuery(query_str)
q.SetParameter("todays_date", DateTime.Today)
Dim i As IList = q.List() ' get results
In the .aspx page, the result can be accessed in a GridView like so:
You're stepping outside the NHibernate paradigm to call down to SQL, which is somewhat against the spirit of ORM. It's not 'bad' per-se, but I'd certainly avoid breaking the abstraction if I could to try to maintain a looser coupling.
You can do what you want with a straight HQL query, which will return a collection of tuples containing the results of your query. I explained how to do this here
Custom query with Castle ActiveRecord
which you might want to have a look at. Even though you must specify a type when you new an HQLBasedQuery, NH is smart enough to know that if you don't select a type instance it should assemble a result-set based on object tuples.
(IMNSHO it's still a bit non-pure - I'd be tempted to try to model this relationship as an object and map it accordingly, but then I'd have to bash the DB into shape to suit the object model and that's not going to fly in all cases.)