Union between optional and non-optional tables - sql

I have two queries that select records where a union needs to be taken, one of which is a left join and one of which is a regular (i.e. inner) join.
Here's the left join case:
def regularAccountRecords = for {
(customer, account) <- customers joinLeft accounts on (_.accountId === _.accountId) // + some other special conditions
} yield (customer, account)
Here's the regular join case:
def specialAccountRecords = for {
(customer, account) <- customers join accounts on (_.accountId === _.accountId) // + some other special conditions
} yield (customer, account)
Now I want to take a union of the two record sets:
regularAccountRecords ++ specialAccountRecords
Obviously this doesn't work because in the regular join case it returns Query[(Customer, Account),...] and in the left join case it returns Query[(Customer, Rep[Option[Account]]),...] and this results in a Type Mismatch error.
Now, If this were a regular column type (e.g. Rep[String]) I could convert it to an optional via the ? operator (i.e. record.?) and get Rep[Option[String]] but using it on a table (i.e. the accounts table) causes:
Error:(62, 85) value ? is not a member of com.test.Account
How do I work around this issue and do the union properly?

Okay, looks like this is what the '?' projection is for but I didn't realize it because I disabled the optionEnabled option in the Codegen. Here's what your codegen extension is supposed to look like:
class MyCodegen extends SourceCodeGenerator(inputModel) {
override def TableClass = new TableClassDef {
override def optionEnabled = true
}
}
Alternatively, you can use implicit classes to tack this thing onto the generated TableClass yourself. Here is how that would look:
implicit class AccountExtensions(account:Account) {
def ? = (Rep.Some(account.id), account.name).shaped.<>({r=>r._1.map(_=> Account.tupled((r._2, r._1.get)))}, (_:Any) => throw new Exception("Inserting into ? projection not supported."))
}
NOTE: be sure to check the field ordering, depending on how this
projection is done, the union query might put the ID field in the wrong
place in the output, use
println(query.result.statements.headOption) to debug the output
SQL to be sure.
Once you do that, you will be able to use account.? in the yield statement:
def specialAccountRecords = for {
(customer, account) <- customers join accounts on (_.accountId === _.accountId)
} yield (customer, account.?)
...and then you will be able to unionize the tables correctly
regularAccountRecords ++ specialAccountRecords
I really wish the Slick people would put a note on how the '?' projection is useful in the documentation beyond the vague statement 'useful for outer joins'.

Related

ActiveRecord OR query with two scopes in SQL where one scope has a join

I want to write a query which combines two scopes with an "OR" query (using Rails 4 though the issue is still there with rails 5).
Model1
scope :association_has_email, -> { joins(:model2).where.not(model2s:{email:nil}) }
scope :description_has_email, -> { where("description ~* ?", email_regex) }
(where email_regex is a regular expression picking out an email).
Which gives us SQL like:
SELECT \"model1s\".* FROM \"model1s\" WHERE (description ~* '[[:<:]][A-Z0-9._%+-]+#[A-Z0-9.-]+.[A-Z]{2,4}[[:>:]]')
SELECT \"model1s\".* FROM \"model1s\" INNER JOIN \"model2s\" ON \"model2s\".\"id\" = \"model1s\".\"model2_id\" WHERE (\"model2s\".\"email\" IS NOT NULL)
Create a scope to pick out those opportunities where either the email is in the association or an embedded email in the text.
How do you write an "OR" query where one side needs the join and the other doesn't?
You can create one more scope like following:
scope :check_email, -> {
if email is embedded
association_has_email
else
description_has_email
end
}
Ok.
For Rails 4, the where-or gem provides a fix for generating or queries as in rails 5, however if you pass the scopes above you end up with a
ArgumentError: Relation passed to #or must be structurally compatible
You can use the following:
scope :email_somewhere, -> { association_has_email.or(Model1.joins(:model2).description_has_email }
SELECT \"model1s\".* FROM \"model1s\" INNER JOIN \"model2s\" ON \"model2s\".\"id\" = \"model1s\".\"model2_id\" WHERE ((\"model2s\".\"email\" IS NOT NULL) OR (description ~* '[[:<:]][A-Z0-9._%+-]+#[A-Z0-9.-]+.[A-Z]{2,4}[[:>:]]'))
Works, but it will exclude anything that has an email in the description but doesn't have a model2, because it uses an inner join.
But by using includes, you can get the desired result.
scope :association_has_email, -> { includes(:model2).where.not(model2s:{email:nil}) }
scope :description_has_email, -> { where("description ~* ?", email_regex) }
Means you can use
scope :email_somewhere, -> { association_has_email.or(Model1.includes(:model2).description_has_email }
The difference is the SQL pulls all of the attributes from model1, and then adds a left outer join for the model2 query.
SELECT [...all the model1 attributes...] LEFT OUTER JOIN \"model2s\" ON \"model2s\".\"id\" = \"model1s\".\"model2_id\" WHERE ((\"model2s\".\"email\" IS NOT NULL) OR (description ~* '[[:<:]][A-Z0-9._%+-]+#[A-Z0-9.-]+.[A-Z]{2,4}[[:>:]]'))
Irritating if you really needed to use an inner join for the first one, but works for me.

Count one-to-one relationship in grails

I have a problem doing a query. That I want to do is access with a query to the data in my "String code" in MyDomainA through a query from MyDomainB. The relationship between the tables is unidirectional one to one.
Is possible use a gorm way to do this??
Domain A:
class LicenceType {
String code
String description
Double price
static constraints = {
}
}
TABLE DOMAIN A
code description
A this is A
B this is B
C this is C
Domain B: (have the unidirectional relationship)
class VoiceUser {
LicenceType licenceType
String username
String email
String nameID
}
TABLE DOMAIN B
User
1
2
3
4
That I want to do is know how many users have the same code(code is a column of DomainA and both tables have a unidirectional relationship as I indicated before).
This is that I'm trying to do that is wrong...
Controller:
def resulta = VoiceUser.executeQuery('SELECT a.code, COUNT(b.nameID) FROM VoiceUser AS b INNER JOIN b.licenceType AS a GROUP BY a.code')
def resultCount = resulta[0]
This is some example result that I hope...
Users with code A = 2
Users with code B = 2
Users with code C = o
The trick is to do a group by on the code and a count() on the user. You can do this using either HQL or a criteria query.
HQL
Here's an example in HQL:
VoiceUser.executeQuery('SELECT licence.code, COUNT(user) FROM VoiceUser AS user INNER JOIN user.licenceType AS licence GROUP BY licence.code')
If you're familiar with SQL, most of this should make sense right away. An important difference is the syntax for joining domain classes. HQL deals with domain classes, not tables.
Criteria query
And here's the equivalent criteria query.
VoiceUser.withCriteria {
projections {
licenceType {
groupProperty('code')
}
count('id')
}
}
Alternative queries
The queries shown above return a List<List> like this:
[
['A', 2],
['B', 2],
['C', 0]
]
If you provide a LicenceType (or its code) as input to the query, then you can get the count for just that LicenceType. For instance, here are examples which retrieve the user count for licence code 'A'.
HQL
def result = VoiceUser.executeQuery('SELECT COUNT(user) FROM VoiceUser AS user INNER JOIN user.licenceType AS licence WHERE licence.code = :code', [code: 'A'])[0]
Criteria query
def result = VoiceUser.createCriteria().get {
licenceType {
eq('code', 'A')
}
projections {
count('id')
}
}
Additional resources
I've got a series of articles which explain HQL, criteria, and where queries in detail; such as how to use projections and joins. Feel free to check them out.

Can I get a relational intersection or difference using Slick 2?

I'm using Slick versian 2.0.0-M3. If I have two Querys representing relations of the same type, I see there is a union operator to inclusively disjoin them, but I don't see a comparable operator for obtaining their intersection nor their difference. Do such operators not exist in Slick?
I think the foregoing explains what I'm looking for, but if not, here's an example. I have the suppliers table:
case class Supplier(snum: String, sname: String, status: Int, city: String)
class Suppliers(tag: Tag) extends Table[Supplier](tag, "suppliers") {
def snum = column[String]("snum")
def sname = column[String]("sname")
def status = column[Int]("status")
def city = column[String]("city")
def * = (snum, sname, status, city) <> (Supplier.tupled, Supplier.unapply _)
}
val suppliers = TableQuery[Suppliers]
If I want to know about suppliers that either are in a particular city or have a particular status, I see how to use Query.union for that:
scala> val thirtySuppliers = suppliers.filter(_.status === 30)
thirtySuppliers: scala.slick.lifted.Query[Suppliers,Suppliers#TableElementType] = scala.slick.lifted.WrappingQuery#166f63a
scala> val londonSuppliers = suppliers.filter(_.city === "London")
londonSuppliers: scala.slick.lifted.Query[Suppliers,Suppliers#TableElementType] = scala.slick.lifted.WrappingQuery#1bea855
scala> (thirtySuppliers union londonSuppliers).foreach(println)
Supplier(S1,Smith,20,London)
Supplier(S4,Clark,20,London)
Supplier(S3,Blake,30,Paris)
Supplier(S5,Adams,30,Athens)
No problem. But what if I want only the suppliers that are both in a particular city and have a particular status? Seems as if I ought to be able to do something like:
(thirtySuppliers intersect londonSuppliers).foreach(println)
Or if I want the suppliers in a particular city except the ones that have a particular status. Can I not do something like:
(thirtySuppliers except londonSuppliers).foreach(println)
SQL has UNION, INTERSECT, and EXCEPT operations, and Slick's Query class has a union method that builds an SQL query using SQL's UNION, but I'm not seeing Query methods in Slick for deriving intersections nor differences. Am I missing them?
There is a pull request that implements this. It will likely make it into 2.0 or 2.1. https://github.com/slick/slick/pull/242 We still need to figure out some details and clean up a bit.
The operations are pretty much composable in that an intersect can just be two filters. For instance
val intersect = suppliers.filter(_.status === 30).filter(_.city === "London")
or except:
val except= suppliers.filter(_.city === "London").filterNot(_.status === 30)

Can I do a NATURAL JOIN is Slick v2?

The title is self-explanatory. Using 2.0.0-M3, I'd like to avoid unnecessary verbosity is the form of explicitly naming the columns to be joined on, since they are appropriately named, and since NATURAL JOIN is part of the SQL standard. Not to mention, Wikipedia itself even says that "The natural join is arguably one of the most important operators since it is the relational counterpart of logical AND."
I think the foregoing ought to be clear enough, but just if not, read on. Suppose I want to know the supplier-name and part-number of each part. Assuming appropriate case classes not shown:
class Suppliers(tag: Tag) extends Table[Supplier](tag, "suppliers") {
def snum = column[String]("snum")
def sname = column[String]("sname")
def * = (snum, sname) <> (Supplier.tupled, Supplier.unapply _)
}
class Shipments(tag: Tag) extends Table[Shipment](tag, "shipments") {
def snum = column[String]("snum")
def pnum = column[String]("pnum")
def * = (snum, pnum) <> (Shipment.tupled, Shipment.unapply _)
}
val suppliers = TableQuery[Suppliers]
val shipments = TableQuery[Shipments]
Given that both tables have the snum column I want to join on, seems as if
( suppliers join shipments ).run
ought to return a Vector with my desired data, but I get a failed attempt at an INNER JOIN, failing (at run-time) since it's missing any join condition.
I know I can do
suppliers.flatMap( s => shipments filter (sp => sp.snum === s.snum) map (sp => (s.sname, sp.pnum)) )
but, even without the names of all the columns I omitted for clarity of this question, it's still quite a lot more typing (and proofreading) than simply
suppliers join shipments
or, for that matter
SELECT * FROM suppliers NATURAL JOIN shipments;
If the Scala code is messier than the SQL code, then I really start questioning things. Is there no way simply to do a natural join in Slick?
Currently not supported by Slick. Please submit a ticket or pull request.
To improve readability of query code, you can put your join conditions into re-usable values. Or you can put the whole join in a function or method extension of Query[Suppliers,Supplier].
Alternatively you could look at the AutoJoin pattern (which basically makes your join conditions implicit) described here http://slick.typesafe.com/docs/#20130612_slick_vs_orm_scaladays_2013 and implemented here https://github.com/cvogt/play-slick/blob/scaladays2013/samples/computer-database/app/util/autojoin.scala

How do I flatten a hierarchy in LINQ to Entities?

I have a class Org, which has ParentId (which points to a Consumer) and Orgs properties, to enable a hierarchy of Org instances. I also have a class Customer, which has a OrgId property. Given any Org instance, named Owner, how can I retrieve all Customer instances for that org? That is, before LINQ I would do a 'manual' traversal of the Org tree with Owner as its root. I'm sure something simpler exists though.
Example: If I have a root level Org called 'Film', with Id '1', and sub-Org called 'Horror' with ParentId of '1', and Id of 23, I want to query for all Customers under Film, so I must get all customers with OrgId's of both 1 and 23.
Linq won't help you with this but SQL Server will.
Create a CTE to generate a flattened list of Org Ids, something like:
CREATE PROCEDURE [dbo].[OrganizationIds]
#rootId int
AS
WITH OrgCte AS
(
SELECT OrganizationId FROM Organizations where OrganizationId = #rootId
UNION ALL
SELECT parent.OrganizationId FROM Organizations parent
INNER JOIN OrgCte child ON parent.Parent_OrganizationId = Child.OrganizationId
)
SELECT * FROM OrgCte
RETURN 0
Now add a function import to your context mapped to this stored procedure. This results in a method on your context (the returned values are nullable int since the original Parent_OrganizationId is declared as INT NULL):
public partial class TestEntities : ObjectContext
{
public ObjectResult<int?> OrganizationIds(int? rootId)
{
...
Now you can use a query like this:
// get all org ids for specific root. This needs to be a separate
// query or LtoE throws an exception regarding nullable int.
var ids = OrganizationIds(2);
// now find all customers
Customers.Where (c => ids.Contains(c.Organization.OrganizationId)).Dump();
Unfortunately, not natively in Entity Framework. You need to build your own solution. Probably you need to iterate up to the root. You can optimize this algorithm by asking EF to get a certain number of parents in one go like this:
...
select new { x.Customer, x.Parent.Customer, x.Parent.Parent.Customer }
You are limited to a statically fixed number of parent with this approach (here: 3), but it will save you 2/3 of the database roundtrips.
Edit: I think I did not get your data model right but I hope the idea is clear.
Edit 2: In response to your comment and edit I have adapted the approach like this:
var rootOrg = ...;
var orgLevels = new [] {
select o from db.Orgs where o == rootOrg, //level 0
select o from db.Orgs where o.ParentOrg == rootOrg, //level 1
select o from db.Orgs where o.ParentOrg.ParentOrg == rootOrg, //level 2
select o from db.Orgs where o.ParentOrg.ParentOrg.ParentOrg == rootOrg, //level 3
};
var setOfAllOrgsInSubtree = orgLevels.Aggregate((a, b) => a.Union(b)); //query for all org levels
var customers = from c in db.Customers where setOfAllOrgsInSubtree.Contains(c.Org) select c;
Notice that this only works for a bounded maximum tree depth. In practice, this is usually the case (like 10 or 20).
Performance will not be great but it is a LINQ-to-Entities-only solution.