Unknown column in WHERE clause using addExpressionFieldToSelect in Magento Collection - sql

I've a magento grid. I'm preparing the collection this way:
protected function _prepareCollection()
{
$listingData = Mage::helper('M2ePro/Data_Global')->getValue('temp_data');
// Get collection products in listing
//--------------------------------
$collection = Mage::helper('M2ePro/Component_Amazon')->getCollection('Listing_Product');
$collection->addExpressionFieldToSelect(
'hello',
'sum({{prod_id}})',
array('prod_id' => '`main_table`.`product_id`')
);
$collection->getSelect()->distinct();
$collection->getSelect()->where("`main_table`.`listing_id` = ?",(int)$listingData['id']);
//--------------------------------
The resulting select is:
SELECT DISTINCT `main_table`.*, `second_table`.*, sum(`main_table`.`product_id`) AS `hello`, `cpe`.`sku` AS `magento_sku`, `cisi`.`is_in_stock`, `cpev`.`value` FROM `sm_m2epro_listing_product` AS `main_table` INNER JOIN `sm_m2epro_amazon_listing_product` AS `second_table` ON `second_table`.`listing_product_id` = `main_table`.`id` INNER JOIN `sm_catalog_product_entity` AS `cpe` ON (cpe.entity_id = `main_table`.product_id) INNER JOIN `sm_cataloginventory_stock_item` AS `cisi` ON (cisi.product_id = `main_table`.product_id AND cisi.stock_id = 1) INNER JOIN `sm_catalog_product_entity_varchar` AS `cpev` ON ( `cpev`.`entity_id` = `main_table`.product_id ) INNER JOIN `sm_eav_attribute` AS `ea` ON (`cpev`.`attribute_id` = `ea`.`attribute_id` AND `ea`.`attribute_code` = 'name') WHERE (`main_table`.`component_mode` = 'amazon') AND (`main_table`.`listing_id` = 3) AND (cpev.store_id = (SELECT MAX(`store_id`) FROM `sm_catalog_product_entity_varchar` WHERE (`entity_id` = `main_table`.`product_id`) AND (`attribute_id` = `ea`.`attribute_id`) AND (`store_id` = 0 OR `store_id` = 7)))
But the program fails on:
$this->addColumn('ASIN_duplicate', array(
'header' => Mage::helper('M2ePro')->__('Duplicated'),
'width' => '20px',
'index' => 'hello',
));
With "Unknown column ‘hello' in WHERE clause".
Cannot figure out where's the problem.

Related

.Net Core 3.1 Entity Framework Slow Query Problem

There was slowness in my .net core webapi application, I reviewed and redesigned my queries to resolve this issue. The code snippet below was written to bring in sales, payments for sales, and sales territories.
I first include the condition query and then the sub-entities in the query. Then I choose which columns to work with ".Select()".
Then I want to combine this table with the name information in other tables. For example, information such as Customer Name, Product Name is in the other table.
The first block of code I share below is a slow running linq query.
result.Results =
(from s in context.Sales
join sa in context.SaleProductServiceAreas on s.SaleId equals sa.SaleId
join sp in context.SalePayments on s.SaleId equals sp.SaleId into spleft
from sp in spleft.DefaultIfEmpty()
join c in context.Customers on s.CustomerId equals c.CustomerId
join p in context.ProductServices on s.ProductServiceId equals p.ProductServiceId
where s.Date >= firstDayOfMonth && s.Date <= lastDayOfMonth
group s by
new
{
s.SaleId,
s.CustomerId,
CustomerNameSurname = c.Name + ' ' + c.Surname,
ProductServiceName = p.Name,
s.Date,
s.Total,
s.Session,
p.ProductServiceId
} into grp
select new SaleJoinDto()
{
SaleId = grp.Key.SaleId,
CustomerNameSurname = grp.Key.CustomerNameSurname,
ProductServiceName = grp.Key.ProductServiceName,
Total = grp.Key.Total,
Date = grp.Key.Date,
Session = grp.Key.Session,
CustomerId = grp.Key.CustomerId,
ProductServiceId = grp.Key.ProductServiceId,
SaleProductServiceAreas = (from sps in context.SaleProductServiceAreas
join spa in context.ProductServiceAreas on sps.ProductServiceAreaId equals spa.ProductServiceAreaId
where sps.SaleId == grp.Key.SaleId
select new SaleProductServiceAreaJoinDto()
{
SaleProductServiceAreaId = sps.SaleProductServiceAreaId,
ProductServiceAreaName = spa.Name,
ProductServiceAreaId = sps.ProductServiceAreaId,
SaleId = sps.SaleId
}).ToList(),
SalePayments = (from spp in context.SalePayments
where spp.SaleId == grp.Key.SaleId
select new SalePaymentDto()
{
SaleId = spp.SaleId,
SalePaymentId = spp.SalePaymentId,
Total = spp.Total,
PaymentMethod = spp.PaymentMethod,
Date = spp.Date
}).ToList()
}).ToList();
["NEW"] This query I wrote is the query I rewrite as a result of the articles I have shared below.
Document of Microsoft Link for query
using (var context = new AppDbContext())
{
var result = new PagedResult<SaleJoinDto>();
result.CurrentPage = pageNumber;
result.PageSize = pageSize;
var pageCount = (double)result.RowCount / pageSize;
result.PageCount = (int)Math.Ceiling(pageCount);
var skip = (pageNumber - 1) * pageSize;
var firstDayOfMonth = new DateTime(date.Year, date.Month, 1);
var lastDayOfMonth = firstDayOfMonth.AddMonths(1).AddDays(-1);
result.Results = await context.Sales
.Where(x => x.Date >= firstDayOfMonth && x.Date <= lastDayOfMonth)
.Include(x => x.SalePayments)
.Include(x => x.SaleProductServiceAreas)
.Select(s => new Sale()
{
SaleId = s.SaleId,
CustomerId = s.CustomerId,
Date = s.Date,
Total = s.Total,
Session = s.Session,
SalePayments = s.SalePayments,
SaleProductServiceAreas = s.SaleProductServiceAreas
})
.Join(context.Customers, sales => sales.CustomerId, customers => customers.CustomerId, (sales, customers) => new
{
sales,
customers
})
.Join(context.ProductServices, combinedSaleAndCus => combinedSaleAndCus.sales.ProductServiceId, product => product.ProductServiceId, (combinedSaleAndCus, product) => new SaleJoinDto()
{
SaleId = combinedSaleAndCus.sales.SaleId,
CustomerId = combinedSaleAndCus.sales.CustomerId,
Date = combinedSaleAndCus.sales.Date,
Total = combinedSaleAndCus.sales.Total,
Session = combinedSaleAndCus.sales.Session,
CustomerNameSurname = combinedSaleAndCus.customers.Name,
SalePayments = combinedSaleAndCus.sales.SalePayments.Select(x => new SalePaymentDto()
{
Date = x.Date,
PaymentMethod = x.PaymentMethod,
SaleId = x.SaleId,
SalePaymentId = x.SalePaymentId,
Total = x.Total
}).ToList(),
SaleProductServiceAreas = combinedSaleAndCus.sales.SaleProductServiceAreas.Join(context.ProductServiceAreas, sp => sp.ProductServiceAreaId, psa => psa.ProductServiceAreaId, (sp, psa) => new SaleProductServiceAreaJoinDto()
{
SaleProductServiceAreaId = sp.SaleProductServiceAreaId,
ProductServiceAreaName = psa.Name
}).ToList()
})
.ToListAsync();
result.RowCount = result.Results.Count();
result.Results = result.Results.OrderByDescending(x => x.Date).Skip(skip).Take(pageSize).ToList();
return result;
}
The problem is I do get the following error when I rewrite it according to this new query.
Where am I making a mistake in the query I just wrote?
{"The LINQ expression 'DbSet\r\n .Where(s => s.Date >=
__firstDayOfMonth_0 && s.Date <= __lastDayOfMonth_1)\r\n .Join(\r\n outer: DbSet, \r\n inner: s => s.CustomerId, \r\n
outerKeySelector: c => c.CustomerId, \r\n innerKeySelector: (s,
c) => new TransparentIdentifier<Sale, Customer>(\r\n Outer
= s, \r\n Inner = c\r\n ))\r\n .Join(\r\n outer: DbSet, \r\n inner: ti => new Sale{ \r\n
SaleId = ti.Outer.SaleId, \r\n CustomerId =
ti.Outer.CustomerId, \r\n Date = ti.Outer.Date, \r\n
Total = ti.Outer.Total, \r\n Session = ti.Outer.Session,
\r\n SalePayments = (MaterializeCollectionNavigation(\r\n
navigation: Navigation: Sale.SalePayments,\r\n
subquery: DbSet\r\n .Where(s0 =>
EF.Property(ti.Outer, "SaleId") != null &&
EF.Property(ti.Outer, "SaleId") == EF.Property(s0,
"SaleId"))), \r\n SaleProductServiceAreas =
(MaterializeCollectionNavigation(\r\n navigation:
Navigation: Sale.SaleProductServiceAreas,\r\n subquery:
DbSet\r\n .Where(s1 =>
EF.Property(ti.Outer, "SaleId") != null &&
EF.Property(ti.Outer, "SaleId") == EF.Property(s1,
"SaleId"))) \r\n }\r\n .ProductServiceId, \r\n
outerKeySelector: p => p.ProductServiceId, \r\n
innerKeySelector: (ti, p) => new
TransparentIdentifier<TransparentIdentifier<Sale, Customer>,
ProductService>(\r\n Outer = ti, \r\n Inner =
p\r\n ))'
could not be translated. Either rewrite the query in
a form that can be translated, or switch to client evaluation
explicitly by inserting a call to either AsEnumerable(),
AsAsyncEnumerable(), ToList(), or ToListAsync(). See
https://go.microsoft.com/fwlink/?linkid=2101038 for more
information."}
Removed not needed joins and grouping. It should be faster.
var query =
from s in context.Sales
join c in context.Customers on s.CustomerId equals c.CustomerId
join p in context.ProductServices on s.ProductServiceId equals p.ProductServiceId
where s.Date >= firstDayOfMonth && s.Date <= lastDayOfMonth
select new SaleJoinDto()
{
SaleId = s.SaleId,
CustomerNameSurname = c.Name + ' ' + c.Surname,
ProductServiceName = p.ProductServiceName,
Total = s.Total,
Date = s.Date,
Session = s.Session,
CustomerId = s.CustomerId,
ProductServiceId = p.ProductServiceId,
SaleProductServiceAreas = (from sps in context.SaleProductServiceAreas
join spa in context.ProductServiceAreas on sps.ProductServiceAreaId equals spa.ProductServiceAreaId
where sps.SaleId == s.SaleId
select new SaleProductServiceAreaJoinDto()
{
SaleProductServiceAreaId = sps.SaleProductServiceAreaId,
ProductServiceAreaName = spa.Name,
ProductServiceAreaId = sps.ProductServiceAreaId,
SaleId = sps.SaleId
}).ToList(),
SalePayments = (from spp in context.SalePayments
where spp.SaleId == s.SaleId
select new SalePaymentDto()
{
SaleId = spp.SaleId,
SalePaymentId = spp.SalePaymentId,
Total = spp.Total,
PaymentMethod = spp.PaymentMethod,
Date = spp.Date
}).ToList()
};
result.Results = query.ToList();
Anyway, even joins can be simplified if you have navigation properties which are not shown in original question.
I fixed the slow responsiveness of the app.
I was running the application on windows host. I changed my hosting to Centos 7 which is a linux distribution. Then when I run the application on Centos 7, the application accelerated perfectly and took flight.
My advice to all .net Core app developers, migrate your apps to linux. It is built to run on .net core linux distributions.
I share the problems I encountered while migrating the application to linux and how I solved it in the link below.
.Net Core 3.1 deploy on Centos 7

How to do a SQL JOIN where multiple joined rows need to contain things

When someone on my site search for an image that has multiple tags I need to query and find all images that have the searched tags, but can't seem to figure out this query.
I have an Images table.
The Images table has a relation to Posts_Images.
Posts_Images would have a relation to Posts table.
Posts has a relation to Posts_Tags table.
Posts_Tags table will have the relations to Tags table.
The query I have so far:
SELECT "images".* FROM "images"
INNER JOIN "posts_images" ON posts_images.image_id = images.id
INNER JOIN "posts" ON posts.id = posts_images.post_id AND posts.state IS NULL
INNER JOIN "posts_tags" ON posts_tags.post_id = posts.id
INNER JOIN "tags" ON posts_tags.tag_id = tags.id
WHERE (("images"."order"=0) AND ("images"."has_processed"=TRUE)) AND (LOWER(tags.tag)='comic') AND ("tags"."tag" ILIKE '%Fallout%') ORDER BY "date_uploaded" DESC LIMIT 30
It gets no results, it is checking if the tags equals both values, but I want to see if any of the tags that were joined have all the values I need.
The desired result would be any Post that has a tag matching Comic and ILIKE '%Fallout%'
You seem to want something like this:
SELECT i.*
FROM images JOIN
posts_images pi
ON pi.image_id = i.id JOIN
posts p
ON p.id = pi.post_id AND p.state IS NULL JOIN
posts_tags pt
ON pt.post_id = p.id JOIN
tags t
ON pt.tag_id = t.id
WHERE i."order" = 0 AND
i.has_processed AND
(LOWER(t.tag) = 'comic') OR
(t.tag ILIKE '%Fallout%')
GROUP BY i.id
HAVING COUNT(DISTINCT tag) >= 2
ORDER BY date_uploaded DESC
LIMIT 30;
The logic is in the HAVING clause. I'm not 100% sure that this is exactly what you want for multiple matches.
In addition to gordon-linoff’s response - query can be described using ActiveQuery:
Images::find()
->alias('i')
->joinWith([
'postsImages pi',
'postsImages.posts p',
'postsImages.posts.tags t'
])
->where([
'p.state' => null,
'i.order' => 0,
'i.has_processed' => true,
])
->andWhere(
'or'
'LOWER(t.tag) = "comic"',
['like', 't.tag', 'Fallout']
])
->groupBy('id')
->having('COUNT(DISTINCT tag) >= 2')
->orderBy('date_uploaded DESC')
->limit(30)
->all()
$images = Images::find()
->innerJoin('posts_images', 'posts_images.image_id = images.id')
->innerJoin('posts', 'posts.id = posts_images.post_id AND posts.state IS NULL')
->where(['images.order' => 0, 'images.has_processed' => true]);
if (!is_null($query)) {
$tags = explode(',', $query);
$images = $images
->innerJoin('posts_tags', 'posts_tags.post_id = posts.id')
->innerJoin('tags', 'posts_tags.tag_id = tags.id');
$tagsQuery = ['OR'];
foreach ($tags as $tag) {
$tag = trim(htmlentities($tag));
if (strtolower($tag) == 'comic') {
$tagsQuery[] = ['tags.tag' => $tag];
} else {
$tagsQuery[] = [
'ILIKE',
'tags.tag', $tag
];
}
}
if (!empty($tagsQuery)) {
$images = $images->andWhere($tagsQuery)
->having('COUNT(DISTINCT tags.tag) >= ' . sizeof($tags));
}
}
$images = $images
->groupBy('images.id')
->orderBy(['date_uploaded' => SORT_DESC])
->offset($offset)
->limit($count);
return $images->all();

Linq To entity no full join issue

I have the following SQL script that gives me the results i'm after , i'm having trouble replicating the full join and the count distinct on the activities count:
select ActivityCount,ActivityComplete, ImagePath, SubjectTitle from
(select
Count(Distinct(Act.[ActivityID])) as ActivityCount,
Sum(Case when (([OutcomeID] = 1 or [OutcomeID] = 3) and AOU.ActivityUserID=1 ) Then 1 Else 0 End) As ActivityComplete,
Sub.[SubjectImageName] as ImagePath,
Sub.SubjectTitle as SubjectTitle
from [dbo].[UserActivityOutcome] AOU
full join Activity Act on Act.[ActivityID] = AOU.[ActivityID]
left join Category Cat on Act.[CategoryID] = Cat.[CategoryID]
left join Subject Sub on Cat.[SubjectID] = Sub.[SubjectID]
group by Sub.SubjectTitle, Sub.[SubjectImageName]
) as x
results:
</head>
<body><div class="spacer"><table id="t1"><tr><td class="typeheader" colspan="4">Result Set (2 items)</td></tr><tr><th title="System.Int32">ActivityCount</th><th title="System.Int32">ActivityComplete</th><th title="System.String">ImagePath</th><th title="System.String">SubjectTitle</th></tr><tr><td class="n">25</td><td class="n">3</td><td>Subject 1.png</td><td>Subject 1</td></tr><tr><td class="n">1</td><td class="n">1</td><td>Subject 2.png</td><td>Subject 2</td></tr><tr><td title="Total=26
Average=13" class="columntotal">26</td><td title="Total=4
Average=2" class="columntotal">4</td><td title="Totals" class="columntotal"></td><td title="Totals" class="columntotal"></td></tr></table></div></body>
</html>
my linq looks like this:
from x in (
(from Act in Activities
join AOU in UserActivityOutcomes on Act.ActivityID equals AOU.ActivityID into AOU_join
from AOU in AOU_join.DefaultIfEmpty()
join ActNA in Activities on AOU.ActivityID equals ActNA.ActivityID into ActNA_join
from ActNA in ActNA_join.DefaultIfEmpty()
join Cat in Categories on AOU.Activity.CategoryID equals Cat.CategoryID into Cat_join
from Cat in Cat_join.DefaultIfEmpty()
join Sub in Subjects on Cat.SubjectID equals Sub.SubjectID into Sub_join
from Sub in Sub_join.DefaultIfEmpty()
group new {Sub, AOU, Act,ActNA} by new {
Sub.SubjectTitle,
Sub.SubjectImageName
} into g
select new {
ActivityCount = g.Distinct().Count(), //g.Count(),
ActivityComplete = g.Sum(p => (
(p.AOU.OutcomeID == 1 ||
p.AOU.OutcomeID == 3)&&
p.AOU.ActivityUserID == 23 ? 1 : 0)),
ImagePath = g.Key.SubjectImageName,
SubjectTitle = g.Key.SubjectTitle
}))
select new {
x.ActivityCount,
x.ActivityComplete,
x.ImagePath,
x.SubjectTitle
}
LINQ full outer join is tricky. The only way it could be simulated is by union of left outer join and right antijoin. Here is IMO the LINQ equivalent of your SQL query:
var query =
// AOU full join Act
(from e in (from AOU in UserActivityOutcomes
join Act in Activities on AOU.ActivityID equals Act.ActivityID into Act_join
from Act in Act_join.DefaultIfEmpty()
select new { AOU, Act })
.Concat
(from Act in Activities
join AOU in UserActivityOutcomes on Act.ActivityID equals AOU.ActivityID into AOU_join
from AOU in AOU_join.DefaultIfEmpty()
where AOU == null
select new { AOU, Act })
let AOU = e.AOU
let Act = e.Act
// left join Cat
join Cat in Categories on Act.CategoryID equals Cat.CategoryID into Cat_join
from Cat in Cat_join.DefaultIfEmpty()
// left join Sub
join Sub in Subjects on Cat.SubjectID equals Sub.SubjectID into Sub_join
from Sub in Sub_join.DefaultIfEmpty()
group new { Sub, AOU, Act } by new { Sub.SubjectTitle, Sub.SubjectImageName } into g
select new
{
ActivityCount = g.Where(e => e.Act != null).Select(e => e.Act.ActivityID).Distinct().Count(),
ActivityComplete = g.Sum(e => (e.AOU.OutcomeID == 1 || e.AOU.OutcomeID == 3) && e.AOU.ActivityUserID == 1 ? 1 : 0),
ImagePath = g.Key.SubjectImageName,
SubjectTitle = g.Key.SubjectTitle
};

SQL Query to LINQ Lambda expression

I am trying to convert a bit of SQL containing a LEFT OUTER JOIN with a GROUP BY clause into a LINQ Lambda expression.
The SQL I need to convert is:-
SELECT m.MemberExternalPK
FROM Member.Member AS m LEFT OUTER JOIN Member.Account AS a ON m.MemberID = a.MemberID
GROUP BY MemberExternalPK
HAVING COUNT(AccountID) = 0
I have managed to get it working correctly with an INNER JOIN between Member and Accounts like this (for a count of Account = 1) but this does not work for Accounts with a count of 0 (hence the LEFT OUTER JOIN is required):-
Members.Join(Accounts, m => m.MemberID, a => a.MemberID, (m, a) => new {m, a})
.GroupBy(t => t.m.MemberExternalPK, t => t.a)
.Where(grp => grp.Count(p => p.AccountID != null) == 1)
.Select(grp => grp.Key)
I have been trying to experiment with the .DefaultIfEmpty() keyword but have so far been unsuccessful. Any help would be greatly appreciated :)
I think this is what you're after:
Query syntax
var members = new List<Member>
{
new Member {MemberId = 1, MemberExternalPk = 100},
new Member {MemberId = 2, MemberExternalPk = 200}
};
var accounts = new List<Account>
{
new Account {AccountId = 1, MemberId = 1}
};
var query =
from m in members
join a in accounts on m.MemberId equals a.MemberId into ma
from ma2 in ma.DefaultIfEmpty()
where ma2 == null
group ma2 by m.MemberExternalPk into grouped
select new {grouped.Key};
The result of this example is that only the number 200 is returned. As it will only return the MemberExternalPk for members that don't have an account. i.e. As MemberId 2 doesn't have a related Account object, it is not included in the reuslts.
var list = members.GroupJoin(accounts,m=>m.MemberId, a => a.MemberId,
(m,a) => new {m,a})
.Where(x=>x.a.Count()==0)
.Select(s=>s.m.MemberExternalPk);

Complex SQL to LINQ conversion with subquery

I am trying to convert this expression into LINQ from SQL, but a bit too difficult for me, maybe you can help me with this!
SELECT TOP (2) RecipeID, UserID, Name, Servings, PreparationTime, TotalTime, DifficultyLevelID, CuisineID, DishID, MainIngredientID, PriceLevelID, FlavourID, Instructions,
Notes, Thumbnail, VideoLink
FROM dbo.Recipes
WHERE (RecipeID NOT IN
(SELECT DISTINCT Recipes_1.RecipeID
FROM dbo.Allergies INNER JOIN
dbo.UsersAllergies ON dbo.Allergies.AllergyID = dbo.UsersAllergies.AllergyID INNER JOIN
dbo.IngredientsAllergies ON dbo.Allergies.AllergyID = dbo.IngredientsAllergies.AllergyID INNER JOIN
dbo.Ingredients ON dbo.IngredientsAllergies.IngredientID = dbo.Ingredients.IngredientID INNER JOIN
dbo.RecipesIngredients ON dbo.Ingredients.IngredientID = dbo.RecipesIngredients.IngredientID INNER JOIN
dbo.Recipes AS Recipes_1 ON dbo.RecipesIngredients.RecipeID = Recipes_1.RecipeID INNER JOIN
dbo.Users ON dbo.UsersAllergies.UserID = dbo.Users.UserID INNER JOIN
dbo.AllergyFactors ON dbo.IngredientsAllergies.AllergyFactorID = dbo.AllergyFactors.AllergyFactorID
WHERE (dbo.Users.UserID = 3) AND (dbo.AllergyFactors.AllergyFactorID < 3)))
It would be easier to help you if you showed us what you have already tried, but a Linq expression like this should give you the same result set
var query = (from rec in context.Recipes
where !(from al in context.Allergies
from ua in context.UsersAllergies.Where(x => al.AllergyID == x.AllergyID)
from ia in context.IngredientsAllergies.Where(x => al.AllergyID == x.AllergyID)
from in in context.Ingredients.Where(x => ia.IngredientID == x.IngredientID)
from ri in context.RecipesIngredients.Where(x => in.IngredientID == x.IngredientID)
from re in context.Recipes.Where(x => ri.RecipeID == x.RecipeID)
from us in context.Users.Where(x => ua.UserID == x.UserID)
from af in context.AllergyFactors.Where(x => ia.AllergyFactorID == x.AllergyFactorID)
where us.UserID == 3 && af.AllergyFactorID < 3
select re.RecipeID)
.Distinct()
.Contains(rec.RecipeID)
select new
{
rec.RecipeID,
rec.UserID,
rec.Name,
rec.Servings,
rec.PreparationTime,
rec.TotalTime,
rec.DifficultyLevelID,
rec.CuisineID,
rec.DishID,
rec.MainIngredientID,
rec.PriceLevelID,
rec.FlavourID,
rec.Instructions,
rec.Notes,
rec.Thumbnail,
rec.VideoLink
}).Take(2);