Filter in select using Slick - sql

How can I do the following sql statement in Slick. The issue is that in the select statement there is filter and I don't know how to do that in Slick.
SELECT Sellers.ID,
COALESCE(count(DISTINCT Produce.IMPORTERID) FILTER (WHERE Produce.CREATED > '2019-04-30 16:38:00'), 0::int) AS AFTERDATE,
COALESCE(count(DISTINCT Produce.IMPORTERID) FILTER (WHERE Produce.NAME::text = 'Apple'::text AND Produce.CREATED > '2018-01-30 16:38:00'), 0::bigint) AS APPLES
FROM Sellers
JOIN Produce ON Produce.SellersID = Sellers.ID
WHERE Sellers.ID = 276
GROUP BY Sellers.ID;

Try
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter
import slick.jdbc.PostgresProfile.api._
case class Seller(id: Long)
case class Produce(name: String, sellerId: Long, importerId: Long, created: LocalDateTime)
class Sellers(tag: Tag) extends Table[Seller](tag, "Sellers") {
def id = column[Long]("ID", O.PrimaryKey)
def * = id <> (Seller.apply, Seller.unapply)
}
class Produces(tag: Tag) extends Table[Produce](tag, "Produce") {
def name = column[String]("NAME", O.PrimaryKey)
def sellerId = column[Long]("SellersID")
def importerId = column[Long]("IMPORTERID")
def created = column[LocalDateTime]("CREATED")
def * = (name, sellerId, importerId, created) <> (Produce.tupled, Produce.unapply)
}
val sellers = TableQuery[Sellers]
val produces = TableQuery[Produces]
val dtf = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")
val ldt2019 = LocalDateTime.parse("2019-04-30 16:38:00", dtf)
val ldt2018 = LocalDateTime.parse("2018-01-30 16:38:00", dtf)
sellers.join(produces).on(_.id === _.sellerId)
.filter { case (s, p) => p.sellerId === 276L }
.groupBy { case (s, p) => s.id }
.map { case (sid, group) =>
(
sid,
group
.filter { case (s, p) => p.created > ldt2019 }
.map { case (s, p) => p.importerId }
.distinct.length,
group
.filter { case (s, p) => p.name === "Apple" && p.created > ldt2018 }
.map { case (s, p) => p.importerId }
.distinct.length
)
}
libraryDependencies += "com.github.tminglei" %% "slick-pg" % "0.18.0"

I really hope something like #Dymytro's answer can work, but from my testing it all comes down to limitations with the GROUP BY, and here are the issues you will run into:
Trying to use just Slick with a Postgres driver won't work because Slick doesn't support aggregate functions with a FILTER clause. Postgres is one of the few databases that supports FILTER! So you won't get far:
someQuery
.groupBy { a => a.pivot }
.map{ case (pivot, query) =>
(
pivot,
query
.filter(_.condition === "stuff")
.map(_.column).distinct.length
)
}
Although it compiles, you'll get some kind of runtime error like:
[ERROR] slick.SlickTreeException: Cannot convert node to SQL Comprehension
Then, if you check out slick-pg you'll notice it has support for Postgres aggregate functions! Including the FILTER clause! But... there's an open issue for aggregate functions with GROUP BY so this sort of attempt will fail too:
import com.github.tminglei.slickpg.agg.PgAggFuncSupport.GeneralAggFunctions._
...
someQuery
.groupBy { a => a.pivot }
.map{ case (pivot, query) =>
(
pivot,
query
.map(a => count(a.column.distinct).filter(a.condition === "stuff"))
)
}
No matching Shape found.
[error] Slick does not know how to map the given types.
So until those issues are resolved or someone posts a work around, luckily simple single column FILTER expressions can be equivalently implemented with the more primitive CASE statements. Though not as pretty, it will work!
val caseApproach = someQuery
.groupBy { a => a.pivot }
.map{ case (pivot, query) =>
(
pivot,
query
.map{ a =>
Case If a.condition === "stuff" Then a.column
}.min //here's where you add the aggregate, e.g. "min"
)
}
println(caseApproach.result.statements.headOption)
select pivot, min((case when ("condition" = 'stuff') then "column" end)) from table group by pivot;

Related

Order by in Quill SQL produces Subquery

I have an employee class.
case class Employee(name: String, age: Int, company: String)
and I write a Group-By query in Quill.
val q = quote {
query[Employee]
.filter(_.age == lift(100))
.groupBy(_.company)
.map { a =>
val tot = a._2.size
val totCond = (a._2.map { i =>
if (i.name == "software") 1 else 0
}.sum)
(tot, totCond)
}
}
ctx.run(q)
This query translates to
SELECT COUNT(*), SUM(CASE WHEN x36.name = 'software' THEN 1 ELSE 0 END) FROM employee x36
WHERE x36.age = ? GROUP BY x36.company
which is fine.
Now I want to sort the query like
quote {
query[Employee]
.filter(_.age == lift(100))
.groupBy(_.company)
.map { a =>
val tot = a._2.size
val totCond = (a._2.map { i =>
if (i.name == "software") 1 else 0
}.sum)
(tot, totCond)
}.sortBy(_._1)(Ord.desc)
}
But this translates to
SELECT x36._1, x36._2 FROM (SELECT COUNT(*) AS _1, SUM(CASE WHEN x36.name = 'software' THEN 1
ELSE 0 END) AS _2 FROM employee x36 WHERE x36.age = ? GROUP BY x36.company) AS x36 ORDER BY
x36._1 DESC
this appears as a subquery which does all logic and the main query which just selects the record from main query.
I want to get rid of this and do everything in only one query. Is there a way to do it ??

How to simplify multiple equals checks in if condition?

How can I do this easier with Kotlin?
if (translation.equals(TRANSLATION_X) ||
translation.equals(TRANSLATION_Y) ||
translation.equals(TRANSLATION_Z)
) {
return
} else {
translation = TRANSLATION_X
}
First, you can use the structural equality operator ==, which is translated to the .equals(...) calls automatically: translation == TRANSLATION_X instead of translation.equals(TRANSLATION_X).
Then, you can use the when statement:
when (translation) {
TRANSLATION_X, TRANSLATION_Y, TRANSLATION_Z -> return
else -> translation = TRANSLATION_X
}
Another alternative that may be more efficient than a when expression is to use a Set:
val options = setOf(TRANSLATION_X, TRANSLATION_Y, TRANSLATION_Z)
if (translation in options) return
else translation = TRANSLATION_X
A when statement seems appropriated in this situation :
val translation = when( translation ) {
TRANSLATION_X -> translation
TRANSLATION_Y -> translation
TRANSLATION_Z -> translation
else TRANSLATION_X
}
I think you can also group the three similar cases in one sentence like this :
val translation = when( translation ) {
TRANSLATION_X, TRANSLATION_Y, TRANSLATION_Z -> translation
else TRANSLATION_X
}

Remove items from a collection in entity framework

I have a function as below :
IEnumerable<Group> GetAllChildren(int parentID)
{
using (Entities db = new Entities())
{
var result = (from x in db.Groups
where x.ParentID == parentID
select x).ToList();
foreach (Group child in result.ToList())
{
result.AddRange(GetAllChildren(child.GroupID));
}
return result;
}
}
In the above function if I pass a group name I get all the children at all levels.
It works as expected.
Now my query looks like something like :
GroupNamesWithCorrespondingEffects
= new ObservableCollection<GroupNameWithCorrespondingEffect>
(from g in db.Groups
select new GroupNameWithCorrespondingEffect
{
GroupID = g.GroupID,
GroupName = g.GroupName,
CorrespondingEffect = g.Master_Effects.Effect
}
);
The above query will give me all the groups.
Now I want to remove all the groups from GroupNamesWithCorrespondingEffects that are children of a group with id == 25.
I have tried .Remove(GetAllChildren(25)) in 2nd query. but I get following error.
Collection.Remove(GroupNameWithCorrespondingEffect) has some invalid arguments.
hope this help you:
var childs = GetAllChildren(25).ToList();
var childIDList = childs.select(u => u.GroupID).ToList();
GroupNamesWithCorrespondingEffects = GroupNamesWithCorrespondingEffects
.Where(u => !childIDList.Contains(u.GroupID)).ToList();

Scalding: retaining all fields after groupBy

I'm doing a groupBy for calculating a value, but it seems that when I group by, I lose all the fields that are not in the aggregation keys:
filtered.filterNot('site) {s:String => ...}
.filterNot('date) {s:String => ...}
aggr = filtered.groupBy('id, 'contentHost) { group =>
group.min('timestamp -> 'min)
//how do I keep original fields? (eg: site, date)
}
aggr.store(Tsv(...)) //eg: field "site" won't be here
in pig, it would be something like this:
aggr = group filtered by concat('id, 'contentHost);
result = foreach aggr {
generate flatten(filtered), //how to do this in scalding?
min(filtered.timestamp) as min;
}
I had the same problem with the tuple API and could only solve it by using the typed API.
You can either use Scala tuples or define your own case class outside your job. E.g.:
case class Data(id: String, site: String, date: String, contentHost: String)
Then you'd process it like this:
val filtered: TypedPipe[Data] = TypedPipe.from(Seq(Data("...", "2014-04-14", "...", "...")))
filtered
.filterNot ( data => data.site == "fr" )
.filterNot ( data => data.date == "2014-02-01" )
.groupBy (data => (data.id, data.contentHost)) // (String,String) -> Data
.min // or .minBy { ... }
.toTypedPipe
.write(TypedTsv[((String, String), Data)]("/path/"))

dynamically change LINQ to Entity query

int year = 2009; // get summ of TONS2009 column
var query = from ODInfo in DataContext.CIMS_TRUCKS
where pLocationIDs.Contains(ODInfo.OID)
group ODInfo by ODInfo.OID into g
select new
{
OID = g.Key,
TotalTons = g.Sum( ODInfo => ODInfo.TONS2009)
};
IN the expression 'ODInfo => ODInfo.TONS2009', how do I change TONS2009 to TONS2010 or TONS2011 based on the method parameter 'int year' ?
K06a's answer is close but won't work server-side. Try this:
IEnumerable<OutputType> myQuery(IEnumerable<InputType> data, Expression<Func<InputType,decimal>> expr)
{
return from ODInfo in DataContext.CIMS_TRUCKS
where pLocationIDs.Contains(ODInfo.OID)
group ODInfo by ODInfo.OID into g
select new OutputType
{
OID = g.Key,
TotalTons = g.AsQueryable().Sum(expr)
};
}
var query = myQuery(DataContext.CIMS_TRUCKS, ODInfo => ODInfo.TONS2009);
I haven't tried this, but did something similar here.
UPDATE
If you really need to translate input strings (like "2009") to expressions, it's still possible:
string year = "2009";
Type ODInfoType = typeof(ODINFOTYPE); // substitute with the type of ODInfo
ParameterExpression pe = ParameterExpression.Parameter(ODInfoType, "ODInfo");
MemberInfo mi = ODInfoType.GetProperty("TONS" + year);
MemberExpression me = Expression.MakeMemberAccess(pe, mi);
var expr = Expression.Lambda<Func<ODINFOTYPE, decimal>>(me, pe);
Be aware that this is a patch to the extremly evil structure of your database.
You can try something like that:
TotalTons = g.Sum( ODInfo => (year == 2009) ? ODInfo.TONS2009 : ((year == 2010)
? ODInfo.TONS2010 : ODInfo.TONS2011))
Or make it more readable and use { } to split that lambda expression into more then one line and use eg. switch statement.
The best solution is to break this up into multiple querys that you can compose to a final query:
int year = 2009; // get summ of TONS2009 column
var odInfos =
year == 2009 ? DataContext.CIMS_TRUCKS.Select(x => new { x.OID, TONS = x.TONS2009 })
year == 2010 ? DataContext.CIMS_TRUCKS.Select(x => new { x.OID, TONS = x.TONS2010 })
year == 2011 ? DataContext.CIMS_TRUCKS.Select(x => new { x.OID, TONS = x.TONS2011 })
: null;
var query = from ODInfo in odInfos
where pLocationIDs.Contains(ODInfo.OID)
group ODInfo by ODInfo.OID into g
select new
{
OID = g.Key,
TotalTons = g.Sum(ODInfo => ODInfo.TONS)
};
This will specialize to three possible queries at runtime, thereby giving the best possible performance. It is better than a case-switch.
Try this way:
IEnumerable<OutputType> myQuery(IEnumerable<InputType> data, Func<InputType,decimal> func)
{
return from ODInfo in data
where pLocationIDs.Contains(ODInfo.OID)
group ODInfo by ODInfo.OID into g
select new OutputType
{
OID = g.Key,
TotalTons = g.Sum(func)
};
}
var query = myQuery(DataContext.CIMS_TRUCKS, ODInfo => ODInfo.TONS2009);
Using DynamicLinq which works with EF also:
int year = 2009; // get summ of TONS2009 column
var query = from ODInfo in DataContext.CIMS_TRUCKS
where pLocationIDs.Contains(ODInfo.OID)
group ODInfo by ODInfo.OID into g
select g;
var projectedGroups = query.Select("new (Key as OID, Sum(TONS" + year + ") as TotalTons)");