Getting error 'Unable to translate a collection subquery in a projection' when using union and select together in EF Core 7 - asp.net-core

I wrote a code that EF Core creates an expression for that looks like this:
DbSet<Reception>()
.Include(x => x.Employee)
.Include(x => x.ReceptionSignatures)
.Where(x => x.Employee.FirstName.Contains("mo"))
.Union(DbSet<Reception>()
.Include(x => x.Employee)
.Include(x => x.ReceptionSignatures)
.Where(x => x.Employee.PersonelId.Contains("mo")))
.Union(DbSet<Reception>()
.Include(x => x.Employee)
.Include(x => x.ReceptionSignatures)
.Where(x => x.Employee.LastName.Contains("mo")))
.Union(DbSet<Reception>()
.Include(x => x.Employee)
.Include(x => x.ReceptionSignatures)
.Where(x => x.Employee.NationId.Contains("mo")))
.OrderBy(x => x.Employee.FirstName.CompareTo("mo") == 0 ? 0 : 1)
.Select(r => new ReceptionAllDTO{
ReceptionId = r.Id,
NationId = r.Employee.NationId,
PersonelId = r.Employee.PersonelId,
FirstName = r.Employee.FirstName,
LastName = r.Employee.LastName,
Birthday = r.Employee.Birthday,
RecepDate = r.RecepDate,
Height = r.Height,
Weight = r.Weight,
ReceptionSignatures = r.ReceptionSignatures,
}
)
In Reception entity, I have a relation to Signature like this:
public virtual ICollection<Signature> ReceptionSignatures { get; set; }
but when EF Core wants to create a query for SQL, it throws this exception:
Unable to translate a collection subquery in a projection since either parent or the subquery doesn't project necessary information required to uniquely identify it and correctly generate results on the client side. This can happen when trying to correlate on keyless entity type. This can also happen for some cases of projection before 'Distinct' or some shapes of grouping key in case of 'GroupBy'. These should either contain all key properties of the entity that the operation is applied on, or only contain simple property access expressions.

It seems like you are querying for more data which is really not efficient. Its better to project your required columns using the Select() and then write a Union.
When writing the Union the number of columns Selected must be same as shown below from a code base i wrote 2 weeks ago and which works.
var billPaymentVoucherQuery = _context.Set<BillPaymentVoucher>().AsQueryable();
var billsQuery = _context.Set<Bill>().AsQueryable();
var anon_billsQuery = billsQuery.Where(w => w.InvoiceDate.Date <= filter.AsAtDate.Date)
.Where(w => w.OperationalStatus == OperationalBillStatus.Approved &&
(
w.FinancialStatus == FinancialBillStatus.Pending ||
w.FinancialStatus == FinancialBillStatus.OnHold ||
w.FinancialStatus == FinancialBillStatus.PartiallyApproved ||
w.FinancialStatus == FinancialBillStatus.Approved
))
.Select(s => new
{
VendorName = s.VendorInvoice.Vendor!.Name,
Type = "Bill",
Date = s.InvoiceDate,
Number = Convert.ToString(s.InvoiceNumber),
Amount = s.LineItemTotal + s.VATAmount
}).AsQueryable();
var anon_billPaymentVoucherQuery = billPaymentVoucherQuery
.Where(w => (
w.UpdatedOn.HasValue &&
w.UpdatedOn.Value.Date <= filter.AsAtDate.Date
)
||
(
w.UpdatedOn.HasValue == false &&
w.CreatedOn.Date <= filter.AsAtDate.Date
))
.Where(w => w.BillPaymentVoucherStatus == BillPaymentVoucherStatus.Paid)
.Select(s => new
{
VendorName = s.PaymentApprovedBill.Bill.VendorInvoice.Vendor!.Name,
Type = "Payment",
Date = s.UpdatedOn ?? s.CreatedOn,
Number = Convert.ToString(s.PaymentApprovedBill.Bill.InvoiceNumber + " | " +
s.PaymentVoucherNumber),
Amount = -s.PayAmount
}).AsQueryable();
var unionedQuery = anon_billsQuery.Union(anon_billPaymentVoucherQuery)
.Where(w => string.IsNullOrWhiteSpace(filter.Type) || w.Type == filter.Type);
int pageSize = 2;
bool hasMoreRecords = true;
var transactionData = await unionedQuery.OrderBy(w => w.VendorName)
.ThenBy(w => w.Date)
.Skip((paginator.PageNumber - 1) * pageSize)
.Take(pageSize)
.ToListAsync(token);

Related

Entity Framework Core 6, Contains Optimization and Pagination

I have made a LINQ query to a database using entity framework.
Currently I have a database of about 10000 entries. When performing a search, it takes about 15-20 seconds before i receive a response.
Here is my query code:
private async Task<StationSearchResult> SimpleSearch(SearchQuery filter, Guid currentUserId, bool showInformationAsPublic) {
var query = _context.Station
.Include(x => x.Workers)
.Include(x => x.WorkerLeads)
.Include(x => x.StationCustomFields)
.ThenInclude(x => x.CustomFieldResponsible)
.Include(x => x.StationType)
.ThenInclude(x => x.StationTypeResponsible)
.Where(x => x.Title.Contains(filter.SearchText)
|| x.Description.Contains(filter.SearchText)
|| x.Alias.Contains(filter.SearchText)
|| x.StationCustomFields.FirstOrDefault(t => t.StringValue.Contains(filter.SearchText)).StringValue.Contains(filter.SearchText));
if (!showInformationAsPublic)
{
query = query.Where(x =>
x.IsPublic == true
|| x.Workers.Any(t => t.ad_id == currentUserId)
|| x.WorkerLeads.Any(t => t.ad_id == currentUserId));
}
var result = await query.OrderBy(GetOrderByProperty(filter.OrderBy)).ToListAsync();
return new StationSearchResult
{
Total = result.Count,
Workplaces = result.Skip(filter.Skip).Take(filter.Count == 0 ? 10 : filter.Count).ToList()
};
}
If i remove my includes, the query time is drastically reduced ~1s, but I need to include data from the station relations.
Is there any way this can be optimized using linq? Or will I have to do a RAW sql query to send to the database?
It is also a requirement, that my searchText matches a value from Title, Alias or The string value from the station custom fields, which can be any given value depending on the station.

Convert SQL to LINQ with IN and MAX

How would I convert the SQL statement below into LINQ?
select
kcustnum,
(custsnum + 1),
custartype,
kcustsrch
from
custmast
where
kcustnum = 'cn'
and
custsnum in (
select
max(custsnum)
from
custmast
where
kcustnum = 'cn'
)
As #Dai mention you don't need IN since you use MAX.
What you're looking for:
var res = _context.custmast
.Where(x => x.kcustnum == "cn"
&& x.custsnum == _context.custmast
.Where(y => y.kcustnum == 'cn')
.Max(y => y.custsnum))
.Select(x => new
{
x.kcustnum,
(x.custsnum + 1),
x.custartype,
x.kcustsrch
});
this is my code that does not work:
public ActionResult NameSelect(string customerNumber)
{
var shipto = from n in db.cs_Custmast
.Where(x => x.kcustnum == customerNumber
&& x.custsnum == db.cs_Custmast
.Where(y => y.kcustnum == customerNumber)
.Max(y => y.custsnum))
.Select (n => new CustomersViewModel
{
Account = n.kcustnum,
Suffix = (n.custsnum + 1),
AccountType = n.custartype,
Search = n.kcustsrch
});
return View("_NameSelection", shipto);
}

Search column names and tables

Is there any way to search your database for column names or tables in linqpad. Im looking for a similar feature that you can get in SSMS through red gates sql search.
You can get the table and column names from the Linq mapping. The following should dump out the table and column names.
var columns =
(from t in this.Mapping.GetTables()
from dm in t.RowType.DataMembers
where dm.DbType != null
select new
{
TableName = t.RowType.Name ,
TableSqlName = t.TableName,
dm.DbType,
ColumnName = dm.Name,
dm.IsPrimaryKey,
ColumnSqlName = dm.MappedName
}
);
columns.Dump();
So it should be straightforward to filter this query.
If you enable system tables under properties for your connection you can use a query like this (this is for MS SQL but you can probably adapt it to others)
void Main()
{
var text = "ThingToFind";
SearchColumns(text).Dump("Columns: " + text);
SearchModules(text).Dump("Modules: " + text);
}
#region
IEnumerable<dynamic> SearchColumns(string text)
{
return sys
.columns
.Join(sys.objects, o => o.object_id, i => i.object_id, (o, i) => new { Object = i, Column = o })
.Join(sys.types, o => o.Column.user_type_id, i => i.user_type_id, (o, i) => new { o.Column, o.Object, Type = i })
.Where(c => c.Object.type_desc != "INTERNAL_TABLE")
.Where(c => c.Object.type_desc != "SYSTEM_TABLE")
.OrderBy(c => c.Object.type)
.ThenBy(c => c.Object.name)
.Select(c => new { c.Object, c.Column, c.Type, Default = c.Column.default_object_id != 0 ? sys.default_constraints.Single(d => d.object_id == c.Column.default_object_id).definition : null })
.Select(c => new { Table_Type = c.Object.type_desc, Table = c.Object.name, Name = c.Column.name, Type = c.Type.name, Length = c.Column.max_length, Precision = c.Column.precision, Scale = c.Column.scale, Nullable = c.Column.is_nullable, c.Default })
.AsEnumerable()
.Where(c => c.Name.ContainsIgnoreCase(text));
}
IEnumerable<dynamic> SearchModules(string text, bool findRelatedModules = false)
{
var modules = sys
.sql_modules
.AsEnumerable()
.Join(sys.objects, o => o.object_id, i => i.object_id, (o, i) => new { i.name, definition = o.definition.Trim() })
.ToList();
var result = modules
.Where(m => m.name.ContainsIgnoreCase(text) || m.definition.ContainsIgnoreCase(text))
.ToList();
while (findRelatedModules)
{
var add = result
.SelectMany(r => r.definition.Split(" \t\n\r!##$%^&*()-=+[]{};':\",.<>/?\\|`~".ToCharArray(), StringSplitOptions.RemoveEmptyEntries))
.Distinct()
.Where(token => modules.Any(m => m.name.ToLower() == token.ToLower()))
.Where(token => !result.Any(m => m.name.ToLower() == token.ToLower()))
.ToList();
result.AddRange(add.Select(a => modules.Single(m => m.name.ToLower() == a.ToLower())));
findRelatedModules = add.Any();
}
result
.Where(m => !m.definition.ContainsIgnoreCase(m.name))
.Dump("Renamed Modules");
return result.OrderBy(r => r.name);
}
#endregion
public static class StringExtensions
{
public static bool ContainsIgnoreCase(this string source, string toCheck, bool bCaseInsensitive )
{
return source.IndexOf(toCheck, bCaseInsensitive ? StringComparison.OrdinalIgnoreCase : StringComparison.Ordinal) >= 0;
}
}

fluent hibernate Transformers.AliasToBean is not working as expected

I have three tables. One is the master table: TableA. One table is referenced by TableA called ReferencedTable and lastly a lookup table referenced by ReferencedTable.
I have this query that returns the ten most recent objects as:
TableADTO TableAlias = null;
LookupTableDTO LookupTableAlias = null;
ReferencedDTO ReferencedAlias = null;
dtos = session.QueryOver(() => TableAlias)
.JoinAlias(() => TableAlias.Object, () =>ReferencedAlias)
.JoinAlias(() => ReferencedAlias.ObjectType, () => LookupTableAlias)
.Where(() => ReferencedAlias.PersonId == user.Id &&
(LookupTableAlias.Id != INVOICE_ID ||
LookupTableAlias.Id != FINANCIAL_ID) &&
TableAlias.Status == NEW_STATUS_FLAG &&
ReferencedAlias.ReceivedDate < DateTime.Now)
.Take(10)
.List()
.Select(dto=>
new AbreviatedDTO
{
Id = dto.Referenced.Id,
Field1 = dto.Field1,
Priority = dto.Referenced.Priority,
ReceivedDate = dto.Referenced.ReceivedDate,
Field1 = dto.Referenced.Field1,
Type = dto.Referenced.Lookup.TypeCode,
Status = dto.Status
}).ToList();
This works as expected. However, I thought the the transformation below would work too. It does bring 10 objects but the objects have all default values and are not populated (e.g. AbreviatedDTO.ReceivedDate = DateTime.Minimum). Am I doing something wrong with the QueryOver?
Any help would be appreciated.
Bill N
TableDTO TableAlias = null;
LookupTableDTO LookupTableAlias = null;
ReferencedDTO ReferencedAlias = null;
dtos = session.QueryOver(() => TableAlias)
.JoinAlias(() => TableAlias.Object, () =>ReferencedAlias)
.JoinAlias(() => ReferencedAlias.ObjectType, () => LookupTableAlias)
.Where(() => ReferencedAlias.PersonId == user.Id &&
(LookupTableAlias.Id != INVOICE_ID ||
LookupTableAlias.Id != FINANCIAL_ID) &&
TableAlias.Status == NEW_STATUS_FLAG &&
ReferencedAlias.ReceivedDate < DateTime.Now)
.SelectList(list => list
.Select(x => TableAlias.Field1)
.Select(x => ReferencedAlias.Id)
.Select(x => ReferencedAlias.Field1)
.Select(x => ReferencedAlias.ReceivedDate)
.Select(x => ReferencedAlias.Priority)
.Select(x => LookupTableAlias.TypeCode))
.TransformUsing(Transformers.AliasToBean<AbreviatedDTO>())
.Take(10)
.List<AbreviatedDTO>()
you'll need to define an alias for each selected field same as the propertyname in the resulting dto
AbreviatedDTO alias = null;
// in query
.SelectList(list => list
.Select(() => TableAlias.Field1).WithAlias(() => alias.Field1)

Nhibernate queryover order by random

I'm trying to write a query which returns randomly ordered results. I've found this post Linq Orderby random ThreadSafe for use in ASP.NET which gave me some basic clue how to do that. But i'm getting following exception:
variable 'x' of type 'Accomodations.DAL.Model.Generated.Accomodation' referenced from scope '', but it is not defined
Here is my query:
var query = session.QueryOver<Accomodation>()
.OrderBy(x => (~(x.Id & seed)) & (x.Id | seed)).Asc; // this is the problematic line of code
if (searchParams.District != 0)
query = query.Where(x => x.District.Id == searchParams.District);
if (searchParams.Region != 0)
query = query.Where(x => x.Region.Id == searchParams.Region);
if (searchParams.Location != 0)
query = query.Where(x => x.Location.Id == searchParams.Location);
var futureCount = query.Clone().Select(Projections.RowCount()).FutureValue<int>();
SearchAccomodationResultItem resultItemAlias = null;
var futurePage = query
.SelectList(list => list
.Select(x => x.Id).WithAlias(() => resultItemAlias.Id)
.Select(x => x.AccomodationType.Id).WithAlias(() => resultItemAlias.AccomodationTypeId)
.Select(x => x.Region.Id).WithAlias(() => resultItemAlias.RegionId)
.Select(x => x.Name).WithAlias(() => resultItemAlias.Title)
.Select(x => x.MaxCapacity).WithAlias(() => resultItemAlias.MaxCapacity)
.Select(x => x.MinPrice).WithAlias(() => resultItemAlias.MinPrice)
.Select(x => x.MinStayLength).WithAlias(() => resultItemAlias.MinStayLength)
.Select(x => x.MainImageName).WithAlias(() => resultItemAlias.ImgSrc)
)
.TransformUsing(Transformers.AliasToBean<SearchAccomodationResultItem>())
.Skip(skip)
.Take(searchParams.PageSize)
.Future<SearchAccomodationResultItem>();
searchResults = futurePage.ToList();
numberOfResults = futureCount.Value;
});
Any suggestion will be appreciated. Thanks
Here is a good example of how to do this. This is a technique that I'm currently using.
http://puredotnetcoder.blogspot.com/2011/09/nhibernate-queryover-and-newid-or-rand.html
Edit
Below is taken from the above article and I've modified it slightly to include Skip as well.
public IList<CmsTestimonial> GetRandomTestimonials(int count, int skip) {
return Session
.QueryOver<CmsTestimonial>()
.OrderByRandom()
.Take(count)
.Skip(skip)
.List();
}
To use the above approach with paging, you could store the seed for the user (on a per session basis probably) and then use the RAND(seed) function in SQL - because you use the same seed, it will generate the same sequence of pseudo-random numbers and thus allow paging