Yesod Esqueleto - How can I express selects with inner pagination? - sql

I am doing a paginated resource, which will require an inner select, which I've already designed in sql terms. It has the following structure:
select *
from (
select w.*, d.distance
from `Work` w
inner join AdrDistance d on d.nhood2 = w.nhood
where d.nhood1 = 1 -- this will be a variable
order by d.distance
limit 0, 10 -- this will be pagination
) w
inner join WImage wi on wi.`work` = w.id
My entity definitions:
Work
...
WImage
work WorkId
url Text
AdrNhood
city AdrCityId
name Text maxlen=100
lat Double
lng Double
-- This is a view with a computed column I used for ordering
AdrDistance
nhood1 AdrNhoodId
nhood2 AdrNhoodId
distance Distance -- type Distance = Int - in Meters
How can I define such a select in Esqueleto, which would resemble such structure (by doing one single query of course)?
Update
I tried to follow this path:
worksByNhood nId offset' limit' =
from $ \wi -> do
(w, d) <- from $ \(w `InnerJoin` d) -> do
on $ d ^. AdrDistanceNhood2 ==. w ^. WorkNhood
where_ (d ^. AdrDistanceNhood1 ==. val nId)
orderBy [asc (d ^. AdrDistanceDistance)]
offset offset'
limit limit'
return (w, d)
where_ (wi ^. WImageWork ==. w ^. WorkId)
return (w, d ^. AdrDistanceDistance, wi)
But it didn't drive me to the correct solution. If someone can help me (even saying that I would be better doing several selects because what I am trying is not viable in Esqueleto), please, comment or answer my question.

I have read this issue on github
and I concluded Esqueleto wasn't designed to support selects in froms the way I was trying, so I did differently:
worksByNhood nId offset' limit' = do
works <- select $ from $ \(w `InnerJoin` d) -> do
on $ d ^. AdrDistanceNhood2 ==. w ^. WorkNhood
where_ (d ^. AdrDistanceNhood1 ==. val nId)
orderBy [asc (d ^. AdrDistanceDistance)]
offset offset'
limit limit'
return (w, d ^. AdrDistanceDistance)
works' <- forM works $ \(w#(Entity wId _), d) -> do
images <- select $ from $ \wi -> do
where_ (wi ^. WImageWork ==. val wId)
return wi
return (w, d, images);
return works'
It is not exactly what I was looking for, but for now I will use it. If somebody have a better approach, please, tell me.

Related

Error: Cannot find any non-recursive equality over Q0 in Coq

Theorem preservation : forall Pi e T1 S e' Q S', hasType empty Pi e T1 -> storeWellTyped Pi S -> step (conf (e :: Q) S) (conf (e' :: Q) S') -> exists Pi', (extends Pi' Pi /\ hasType empty Pi' e' T1 /\ storeWellTyped Pi' S'). – Amit 3 mins ago edit
When I am trying to prove this theorem e get replaced by various expressions in my syntax but e' does not get replaced by corresponding stepped expression. For example if e is e1;e2 then e' should be e1';e2 in the hypothesis. But e' does not changes so it becomes difficult to prove.
I was trying to do induction on the hypothesis hasType empty Pi e T1 and got the various cases for each expression to prove but due to above problem I could not proceed further.
Error: Cannot find any non-recursive equality over Q0.
I was trying to prove preservation theorem where I got this error. I don't really understand what it means and how it can be eliminated. This error pops out after using subst with inversion.

Yesod - Persistent - ODBC issue

My relevant models:
ProductCharacteristic
productInstance ProductInstanceId sql=productInstanceId
characteristicInstance CharacteristicInstanceId sql=characteristicInstanceId
deriving Show
Product
codePrefix Text
defaultPrice Double
defaultBuyingPrice Double
name Text
obs Text
disabled Bool
deriving Show
ProductInstance
product ProductId sql=productId
code Text
barCode Text
price Double
buyingPrice Double
proportional Bool
obs Text
disabled Bool
deriving Show
Now my Esqueleto expression:
f' :: Handler [(Entity ProductInstance, Entity Product, Entity ProductCharacteristic)]
f' = runDB
$ E.select
$ E.from $ \(pi `E.InnerJoin` p `E.InnerJoin` pc) -> do
E.on $ pi ^. ProductInstanceProduct E.==. p ^. ProductId
E.on $ pc ^. ProductCharacteristicProductInstance E.==. pi ^. ProductInstanceId
return ( pi, p, pc)
The code I showed generated the following select instruction (which is wrong):
SELECT `ProductInstance`.`id`, `ProductInstance`.`productId`, `ProductInstance`.`code`, `ProductInstance`.`barCode`, `ProductInstance`.`price`, `ProductInstance`.`buyingPrice`, `ProductInstance`.`proportional`, `ProductInstance`.`obs`, `ProductInstance`.`disabled`, `Product`.`id`, `Product`.`codePrefix`, `Product`.`defaultPrice`, `Product`.`defaultBuyingPrice`, `Product`.`name`, `Product`.`obs`, `Product`.`disabled`, `ProductCharacteristic`.`id`, `ProductCharacteristic`.`productInstanceId`, `ProductCharacteristic`.`characteristicInstanceId`
FROM `ProductInstance` INNER JOIN `Product` ON `ProductCharacteristic`.`productInstanceId` = `ProductInstance`.`id` INNER JOIN `ProductCharacteristic` ON `ProductInstance`.`productId` = `Product`.`id`;
It is flipping the on clause in the JOIN!! How can I make a workaround???
The correct select instruction should be:
SELECT `ProductInstance`.`id`, `ProductInstance`.`productId`, `ProductInstance`.`code`, `ProductInstance`.`barCode`, `ProductInstance`.`price`, `ProductInstance`.`buyingPrice`, `ProductInstance`.`proportional`, `ProductInstance`.`obs`, `ProductInstance`.`disabled`, `Product`.`id`, `Product`.`codePrefix`, `Product`.`defaultPrice`, `Product`.`defaultBuyingPrice`, `Product`.`name`, `Product`.`obs`, `Product`.`disabled`, `ProductCharacteristic`.`id`, `ProductCharacteristic`.`productInstanceId`, `ProductCharacteristic`.`characteristicInstanceId`
FROM `ProductInstance` INNER JOIN `Product` ON `ProductInstance`.`productId` = `Product`.`id` INNER JOIN `ProductCharacteristic` ON `ProductCharacteristic`.`productInstanceId` = `ProductInstance`.`id`;

How do I do a DIFF in PIG

I have 2 data sets on which I am trying to find the difference. I am aware that there are other ways to do the same. What I am interested in is why this snippet of code is failing.
A = LOAD 'raw.people1' using org.apache.hive.hcatalog.pig.HCatLoader();
B = LOAD 'raw.people2' using org.apache.hive.hcatalog.pig.HCatLoader();
C = COGROUP A BY (name, place, animal, thing) , B BY (name, place, animal, thing) ;
D = FOREACH C DIFF(A, B);
A, B and C work correctly. But D fails with the error:
Failed to parse: Syntax error, unexpected symbol at or near 'DIFF'
Now this should not be the case. The pig docs (http://pig.apache.org/docs/r0.9.1/func.html#diff) state the DIFF takes two pags as params and A and B are bags of tuples.
What am I missing here?
Thanks
You missed GENERATE keyword before DIFF stmt, that is the reason for this error. Can you change like this?
D = FOREACH C GENERATE DIFF(A, B);

Composing Database.Esqueleto queries, conditional joins and counting

How can I compose Database.Esqueleto queries in a modular way such that after defining a "base" query and the corresponding result set, I can restrict the result set by adding additional inner joins and where expressions.
Also, how can I convert the base query that returns a list of entities (or field tuples) into a query that counts the result set since the base query is not executed as such, but a modified version of it with LIMIT and OFFSET.
The following incorrect Haskell code snippet adopted from the Yesod Book hopefully clarifies what I'm aiming at.
{-# LANGUAGE QuasiQuotes, TemplateHaskell, TypeFamilies, OverloadedStrings #-}
{-# LANGUAGE GADTs, FlexibleContexts #-}
import qualified Database.Persist as P
import qualified Database.Persist.Sqlite as PS
import Database.Persist.TH
import Control.Monad.IO.Class (liftIO)
import Data.Conduit
import Control.Monad.Logger
import Database.Esqueleto
import Control.Applicative
share [mkPersist sqlSettings, mkMigrate "migrateAll"] [persistLowerCase|
Person
name String
age Int Maybe
deriving Show
BlogPost
title String
authorId PersonId
deriving Show
Comment
comment String
blogPostId BlogPostId
|]
main :: IO ()
main = runStdoutLoggingT $ runResourceT $ PS.withSqliteConn ":memory:" $ PS.runSqlConn $ do
runMigration migrateAll
johnId <- P.insert $ Person "John Doe" $ Just 35
janeId <- P.insert $ Person "Jane Doe" Nothing
jackId <- P.insert $ Person "Jack Black" $ Just 45
jillId <- P.insert $ Person "Jill Black" Nothing
blogPostId <- P.insert $ BlogPost "My fr1st p0st" johnId
P.insert $ BlogPost "One more for good measure" johnId
P.insert $ BlogPost "Jane's" janeId
P.insert $ Comment "great!" blogPostId
let baseQuery = select $ from $ \(p `InnerJoin` b) -> do 
on (p ^. PersonId ==. b ^. BlogPostAuthorId)
where_ (p ^. PersonName `like` (val "J%"))
return (p,b)
-- Does not compile
let baseQueryLimited = (,) <$> baseQuery <*> (limit 2)
-- Does not compile
let countingQuery = (,) <$> baseQuery <*> (return countRows)
-- Results in invalid SQL
let commentsQuery = (,) <$> baseQuery
<*> (select $ from $ \(b `InnerJoin` c) -> do
on (b ^. BlogPostId ==. c ^. CommentBlogPostId)
return ())
somePosts <- baseQueryLimited
count <- countingQuery
withComments <- commentsQuery
liftIO $ print somePosts
liftIO $ print ((head count) :: Value Int)
liftIO $ print withComments
return ()
Looking at the documentation and the type of select:
select :: (...) => SqlQuery a -> SqlPersistT m [r]
It's clear that upon calling select, we leave the world of pure composable queries (SqlQuery a) and enter the world of side effects (SqlPersistT m [r]). So we simply need to compose before we select.
let baseQuery = from $ \(p `InnerJoin` b) -> do
on (p ^. PersonId ==. b ^. BlogPostAuthorId)
where_ (p ^. PersonName `like` (val "J%"))
return (p,b)
let baseQueryLimited = do r <- baseQuery; limit 2; return r
let countingQuery = do baseQuery; return countRows
somePosts <- select baseQueryLimited
count <- select countingQuery
This works for limiting and counting. I haven't figured out how to do it for joins yet, but it looks like it should be possible.
For LIMIT and COUNT, hammar's answer is entirely correct so I'll not delve into them. I'll just reiterate that once you use select you'll not be able to change the query in any way again.
For JOINs, currently you are not able to do a INNER JOIN with a query that was defined in a different from (nor (FULL|LEFT|RIGHT) OUTER JOINs). However, you can do implicit joins. For example, if you have defined:
baseQuery =
from $ \(p `InnerJoin` b) -> do
on (p ^. PersonId ==. b ^. BlogPostAuthorId)
where_ (p ^. PersonName `like` val "J%")
return (p, b)
Then you may just say:
commentsQuery =
from $ \c -> do
(p, b) <- baseQuery
where_ (b ^. BlogPostId ==. c ^. CommentBlogPostId)
return (p, b, c)
Esqueleto then will generate something along the lines of:
SELECT ...
FROM Comment, Person INNER JOIN BlogPost
ON Person.id = BlogPost.authorId
WHERE Person.name LIKE "J%"
AND BlogPost.id = Comment.blogPostId
Not pretty but gets the job done for INNER JOINs. If you need to do a OUTER JOIN then you'll have to refactor your code so that all the OUTER JOINs are in the same from (note that you can do an implicit join between OUTER JOINs just fine).

Partial SQL insert in haskelldb

I just started a new project and wanted to use HaskellDB in the beginning. I created a database with 2 columns:
create table sensor (
service text,
name text
);
..found out how to do the basic HaskellDB machinery (ohhh..the documentation) and wanted to do an insert. However, I wanted to do a partial insert (there are supposed to be more columns), something like:
insert into sensor (service) values ('myservice');
Translated into HaskellDB:
transaction db $ insert db SE.sensor (SE.service <<- (Just $ senService sensor))
But...that simply doesn't work. What also does not work is if I specify the column names in different order, which is not exactly conenient as well. Is there a way to do a partial insert in haskelldb?
The error codes I get are - when I just inserted a different column (the 'name') as the first one:
Couldn't match expected type `SEI.Service'
against inferred type `SEI.Name'
Expected type: SEI.Intsensor
Inferred type: Database.HaskellDB.HDBRec.RecCons
SEI.Name (Expr String) er
When using functional dependencies to combine
Database.HaskellDB.Query.InsertRec
(Database.HaskellDB.HDBRec.RecCons f (e a) r)
(Database.HaskellDB.HDBRec.RecCons f (Expr a) er),
etc..
And when I do the 'service' as the first - and only - field, I get:
Couldn't match expected type `Database.HaskellDB.HDBRec.RecCons
SEI.Name
(Expr String)
(Database.HaskellDB.HDBRec.RecCons
SEI.Time
(Expr Int)
(Database.HaskellDB.HDBRec.RecCons
SEI.Intval (Expr Int) Database.HaskellDB.HDBRec.RecNil))'
against inferred type `Database.HaskellDB.HDBRec.RecNil'
(I have a couple of other columns in the table)
This looks really like 'by design', unfortunately :(
You're right, that does look intentional. The HaskellDB.Query docs show that insert has a type of:
insert :: (ToPrimExprs r, ShowRecRow r, InsertRec r er) => Database -> Table er -> Record r -> IO ()
In particular, the relation InsertRec r er must hold. That's defined elsewhere by the recursive type program:
InsertRec RecNil RecNil
(InsertExpr e, InsertRec r er) => InsertRec (RecCons f (e a) r) (RecCons f (Expr a) er)
The first line is the base case. The second line is an inductive case. It really does want to walk every element of er, the table. There's no short-circuit, and no support for re-ordering. But in my own tests, I have seen this work, using _default:
insQ db = insert db test_tbl1 (c1 <<- (Just 5) # c2 << _default)
So if you want a partial insert, you can always say:
insC1 db x = insert db test_tbl1 (c1 <<- (Just x) # c2 << _default)
insC2 db x = insert db test_tbl2 (c1 << _default # c2 <<- (Just x))
I realize this isn't everything you're looking for. It looks like InsertRec can be re-written in the style of HList, to permit more generalization. That would be an excellent contribution.