I wrote a code that EF Core creates an expression for that looks like this:
DbSet<Reception>()
.Include(x => x.Employee)
.Include(x => x.ReceptionSignatures)
.Where(x => x.Employee.FirstName.Contains("mo"))
.Union(DbSet<Reception>()
.Include(x => x.Employee)
.Include(x => x.ReceptionSignatures)
.Where(x => x.Employee.PersonelId.Contains("mo")))
.Union(DbSet<Reception>()
.Include(x => x.Employee)
.Include(x => x.ReceptionSignatures)
.Where(x => x.Employee.LastName.Contains("mo")))
.Union(DbSet<Reception>()
.Include(x => x.Employee)
.Include(x => x.ReceptionSignatures)
.Where(x => x.Employee.NationId.Contains("mo")))
.OrderBy(x => x.Employee.FirstName.CompareTo("mo") == 0 ? 0 : 1)
.Select(r => new ReceptionAllDTO{
ReceptionId = r.Id,
NationId = r.Employee.NationId,
PersonelId = r.Employee.PersonelId,
FirstName = r.Employee.FirstName,
LastName = r.Employee.LastName,
Birthday = r.Employee.Birthday,
RecepDate = r.RecepDate,
Height = r.Height,
Weight = r.Weight,
ReceptionSignatures = r.ReceptionSignatures,
}
)
In Reception entity, I have a relation to Signature like this:
public virtual ICollection<Signature> ReceptionSignatures { get; set; }
but when EF Core wants to create a query for SQL, it throws this exception:
Unable to translate a collection subquery in a projection since either parent or the subquery doesn't project necessary information required to uniquely identify it and correctly generate results on the client side. This can happen when trying to correlate on keyless entity type. This can also happen for some cases of projection before 'Distinct' or some shapes of grouping key in case of 'GroupBy'. These should either contain all key properties of the entity that the operation is applied on, or only contain simple property access expressions.
It seems like you are querying for more data which is really not efficient. Its better to project your required columns using the Select() and then write a Union.
When writing the Union the number of columns Selected must be same as shown below from a code base i wrote 2 weeks ago and which works.
var billPaymentVoucherQuery = _context.Set<BillPaymentVoucher>().AsQueryable();
var billsQuery = _context.Set<Bill>().AsQueryable();
var anon_billsQuery = billsQuery.Where(w => w.InvoiceDate.Date <= filter.AsAtDate.Date)
.Where(w => w.OperationalStatus == OperationalBillStatus.Approved &&
(
w.FinancialStatus == FinancialBillStatus.Pending ||
w.FinancialStatus == FinancialBillStatus.OnHold ||
w.FinancialStatus == FinancialBillStatus.PartiallyApproved ||
w.FinancialStatus == FinancialBillStatus.Approved
))
.Select(s => new
{
VendorName = s.VendorInvoice.Vendor!.Name,
Type = "Bill",
Date = s.InvoiceDate,
Number = Convert.ToString(s.InvoiceNumber),
Amount = s.LineItemTotal + s.VATAmount
}).AsQueryable();
var anon_billPaymentVoucherQuery = billPaymentVoucherQuery
.Where(w => (
w.UpdatedOn.HasValue &&
w.UpdatedOn.Value.Date <= filter.AsAtDate.Date
)
||
(
w.UpdatedOn.HasValue == false &&
w.CreatedOn.Date <= filter.AsAtDate.Date
))
.Where(w => w.BillPaymentVoucherStatus == BillPaymentVoucherStatus.Paid)
.Select(s => new
{
VendorName = s.PaymentApprovedBill.Bill.VendorInvoice.Vendor!.Name,
Type = "Payment",
Date = s.UpdatedOn ?? s.CreatedOn,
Number = Convert.ToString(s.PaymentApprovedBill.Bill.InvoiceNumber + " | " +
s.PaymentVoucherNumber),
Amount = -s.PayAmount
}).AsQueryable();
var unionedQuery = anon_billsQuery.Union(anon_billPaymentVoucherQuery)
.Where(w => string.IsNullOrWhiteSpace(filter.Type) || w.Type == filter.Type);
int pageSize = 2;
bool hasMoreRecords = true;
var transactionData = await unionedQuery.OrderBy(w => w.VendorName)
.ThenBy(w => w.Date)
.Skip((paginator.PageNumber - 1) * pageSize)
.Take(pageSize)
.ToListAsync(token);
convert sql to linq that has STRING_AGG
SELECT
V.pkid,
V.[Name] AS VendorName,
SFTP.SFTP_Paths AS SFTP_Paths
FROM
dbo.Vendor V
LEFT OUTER JOIN (
SELECT
connected_to,
STRING_AGG(rootfolder, ', ') AS SFTP_Paths
FROM
dbo.FTP
WHERE
connected_to_type = 4 -- Vendor
GROUP BY
connected_to
) SFTP ON v.pkid = SFTP.connected_to
WHERE
V.active = 1
order by
V.[name]
This query returns the same sql query result.
Instead of STRING_AGG (rootfolder, ',') AS SFTP_Paths I used SFTP_Paths = string.Join(",", b.Select (c => c.rootfolder).ToArray()) for equivalence.
The rest of the query is understandable.
var query = vendorList.Join(lstFtp.Where(x => x.connected_to_type == 4).GroupBy(a => a.connected_to)
.Select(b => new
{
connected_to = b.Key,
SFTP_Paths = b.Select(c => c.rootfolder).ToList()
}).AsEnumerable()
.Select(b => new
{
connected_to = b.connected_to,
SFTP_Paths = string.Join(",", b.SFTP_Paths).ToArray()
}),
right => right.pkid,
left => left.connected_to,
(right, left) => new
{
V = right,
SFTP = left
}).Where(d => d.V.active == 1)
.Select(e => new
{
pkid = e.V.pkid,
VendorName = e.V.Name,
SFTP_Paths = e.SFTP.SFTP_Paths
})
.OrderBy(e => e.VendorName).ToList();
first time I was trying to join this many tables using Entity Framework.
Everything works well but loading time is about 1 minute, very bad. I dont know if is some different solution to join this many tables, or something what could help.
It is about 5000 records in database. I think in future number of this records can be more higher. I need something what will take it. Thanks.
Code below:
var JoinedTopCars = db.CustomerAdvertisement.Where(o => o.Top == true)
.Join(
db.CarManufacturerName,
n => n.CarManufacturerId,
q => q.CarManufacturerId,
(n, q) => new
{
n,
q,
})
.Join(
db.CarManufacturerModel,
o => o.n.CarModelId,
j => j.CarModelId,
(o, j) => new
{
j,
o,
}).Where(y => y.j.CarManufacturerId == y.o.q.CarManufacturerId)
.Join(
db.TypeOFFuel,
e => e.o.n.FuelId,
r => r.Id,
(e, r) => new
{
e,
r,
})
.Join(
db.TypeOFGearBox,
t => t.e.o.n.GearBoxId,
i => i.Id,
(t, i) => new
{
t,
i,
})
.Join(
db.Country,
y => y.t.e.o.n.CountryOfOrigin,
u => u.Id,
(y, u) => new
{
y,
u,
})
.Join(
db.TypeOFChassis,
c => c.y.t.e.o.n.ChassisId,
d => d.Id,
(c, d) => new
{
c,
d,
})
.Join(
db.CarDoors,
e => e.c.y.t.e.o.n.CarDoorsId,
f => f.Id,
(e, f) => new
{
e,
f,
})
.Join(
db.TypOfMovement,
g => g.e.c.y.t.e.o.n.MovementId,
f => f.Id,
(g, f) => new
{
g,
f,
})
.Join(
db.Area,
i => i.g.e.c.y.t.e.o.n.AreaOfOrigin ?? 0,
f => f.Id,
(i, f) => new
{
i,
f,
})
.Join(
db.District,
j => j.i.g.e.c.y.t.e.o.n.OkresOfOrigin ?? 0,
f => f.Id,
(j, f) => new
{
j,
f,
})
.Join(
db.CarColor,
k => k.j.i.g.e.c.y.t.e.o.n.CarColorId,
x => x.Id,
(k, x) => new JoinedTopCars
{
Id = k.j.i.g.e.c.y.t.e.o.n.Id,
Objem = k.j.i.g.e.c.y.t.e.o.n.cm3,
Carname = k.j.i.g.e.c.y.t.e.o.q.CarName,
CarModel = k.j.i.g.e.c.y.t.e.j.CarModel,
Typ = k.j.i.g.e.c.y.t.e.o.n.ModelType,
Color = x.ColorName,
Karoseria = k.j.i.g.e.d.ChassisName,
Dvere = k.j.i.g.f.NumberOfDoors,
Pohon = k.j.i.f.Movement,
VIN = k.j.i.g.e.c.y.t.e.o.n.VIN,
Metalic = k.j.i.g.e.c.y.t.e.o.n.Metalic,
Poskodene = k.j.i.g.e.c.y.t.e.o.n.Crashed,
Pojazdne = k.j.i.g.e.c.y.t.e.o.n.Drivable,
DPH = k.j.i.g.e.c.y.t.e.o.n.DPH,
Leasing = k.j.i.g.e.c.y.t.e.o.n.Leasing,
Emisie = k.j.i.g.e.c.y.t.e.o.n.Emmisions,
Spotreba = k.j.i.g.e.c.y.t.e.o.n.Consumption,
Km = k.j.i.g.e.c.y.t.e.o.n.KM,
Rok = k.j.i.g.e.c.y.t.e.o.n.DateOfOrigin.ToString(),
Vykon = k.j.i.g.e.c.y.t.e.o.n.HP,
Palivo = k.j.i.g.e.c.y.t.r.Fuel,
Prevodovka = k.j.i.g.e.c.y.i.GearBox,
Krajina = k.j.i.g.e.c.u.CountryName,
Okres = k.f.DistrictName,
Kraj = k.j.f.AreaName,
Vybava = k.j.i.g.e.c.y.t.e.o.n.Equipment,
Popis = k.j.i.g.e.c.y.t.e.o.n.Description,
Kontakt = k.j.i.g.e.c.y.t.e.o.n.ContInfo,
ZobrazMeno = k.j.i.g.e.c.y.t.e.o.n.ShowName,
ZobrazCislo = k.j.i.g.e.c.y.t.e.o.n.ShowPhone,
Cena = k.j.i.g.e.c.y.t.e.o.n.Price,
TitleImage = k.j.i.g.e.c.y.t.e.o.n.TitlePhoto,
})
.OrderByDescending(z => z.Id)
.Take(15);
EDIT: I make what #romfir write bellow and decrase loading time from 1 minute to 3 sec.
Create SQL view
insert SQL query with JOINed tables (if you expect empty value use LEFT JOIN)
Update ADO.NET EF model, .edmx file
use this SQL View simmilar like table
You can try to create View in the database, like this:
CREATE VIEW my_view_name
AS
SELECT
CustomerAdvertisement.ID as CustomerAdvertisementID -- 'as Name' is optional
CustomerAdvertisement.cm3
-- other columns You want to include
FROM
CustomerAdvertisement
JOIN CarManufacturerName
on CarManufacturerName.CarManufacturerId = CustomerAdvertisement.CarManufacturerId
JOIN SomeTable
on some_condition
-- other joins
WHERE
CustomerAdvertisement.Top = true
and other_conditions
After creating view that matches Your criteria, You can scaffold it, and then use in Your code.
I am using EF core to perform a complex query. I need to use group by and pick the top result from the group and then perform filter on the top rows of each group.
When I use EF profiler, it shows that query fetched 21647 rows and the final result after the filter is 28. I assume that EF uses the where query after retrieving the columns from DB.
Here is the structure of query.
private IQueryable<long> GetAssignedIds(int? lpId = null, int? customerTenantId = null)
{
var islpAdmin = _permissionChecker.IsGranted(AppPermissions.Pages_PP_lpAdmin);
var query = (
from -------QUERY--------------------
join -------QUERY--------------------
join -------QUERY--------------------
join -------QUERY--------------------
select new
{
-----SELECT VARIABLES----------
})
.OrderBy(c => c.Priority)
.GroupBy(c => new { c.ScpId, c.OrderId })
.Select(c => c.FirstOrDefault())
.WhereIf(lpId.HasValue, c => c.lpId == lpId)
.WhereIf(islpAdmin, c => c.lpId == currentTenantId)
.WhereIf(!islpAdmin, c => c.eId == currentTenantId)
.Select(c => c.ScoId)
.Distinct();
}
How can I optimize this query? I know .Select(c => c.FirstOrDefault()) is the problem here but I'm not able to find proper alternative here.
Table: one
Column (PK): employee_ssn
Column: employee_active bit
Table: two
Column (pk): date
Column (FK): employee_ssn
Column: total_income
Column: total_expenses
Working Linq code to sum total_income and total_expenses for the year, per employee:
var SumOfSections = db.two
.Select(x => x)
.Where(x => x.employee_ssn.Equals(xxxxxxxxx))
.Where(x => x.Date.Year.Equals(year))
.GroupBy(x => x.Date)
.Select(g => new
{
Total_Income = g.Sum(x => x.total_expenses),
Total_Expenses= g.Sum(x => x.total_income)
})
.ToArray();
I need to incorporate INNER JOIN to the above to only include active employees. Working SQL:
select two.total_income,total_expenses
from two
INNER JOIN one
ON one.SSN = two.SSN
WHERE one.Active = 1 AND two.Date='1/1/2014' AND two.SSN='xxxxxxxxx';
How can I modify my linq code to what my sql code is doing?
var SumOfSections = (from t in db.two
join o in db.one on t.employee_ssn equald o.employee_ssn
where t.employee_ssn = "xxxxxxxxx" && o.employee_active == true
group t by t.date into g
select new {
Total_Income = g.Sum(x => x.total_expenses),
Total_Expenses= g.Sum(x => x.total_income)
}).ToArray();
I used query syntax because it seems to be more readable.
You can also continue with the notation you are using:
var SumOfSections = db.two.Join(db.one, o=>o.employee_ssn, t=>t.employee_ssn, (o,t)=>new {One = o, Two = t)
.Select(x => x)
.Where(x => x.Two.employee_ssn.Equals(""))
.Where(x => x.Two.date.Year.Equals(1234))
.Where(x=> x.One.employee_active == true)
.GroupBy(x => x.Two.date)
.Select(g => new
{
Total_Income = g.Sum(x => x.Two.total_expenses),
Total_Expenses = g.Sum(x => x.Two.total_income)
})
.ToArray();