My relevant models:
ProductCharacteristic
productInstance ProductInstanceId sql=productInstanceId
characteristicInstance CharacteristicInstanceId sql=characteristicInstanceId
deriving Show
Product
codePrefix Text
defaultPrice Double
defaultBuyingPrice Double
name Text
obs Text
disabled Bool
deriving Show
ProductInstance
product ProductId sql=productId
code Text
barCode Text
price Double
buyingPrice Double
proportional Bool
obs Text
disabled Bool
deriving Show
Now my Esqueleto expression:
f' :: Handler [(Entity ProductInstance, Entity Product, Entity ProductCharacteristic)]
f' = runDB
$ E.select
$ E.from $ \(pi `E.InnerJoin` p `E.InnerJoin` pc) -> do
E.on $ pi ^. ProductInstanceProduct E.==. p ^. ProductId
E.on $ pc ^. ProductCharacteristicProductInstance E.==. pi ^. ProductInstanceId
return ( pi, p, pc)
The code I showed generated the following select instruction (which is wrong):
SELECT `ProductInstance`.`id`, `ProductInstance`.`productId`, `ProductInstance`.`code`, `ProductInstance`.`barCode`, `ProductInstance`.`price`, `ProductInstance`.`buyingPrice`, `ProductInstance`.`proportional`, `ProductInstance`.`obs`, `ProductInstance`.`disabled`, `Product`.`id`, `Product`.`codePrefix`, `Product`.`defaultPrice`, `Product`.`defaultBuyingPrice`, `Product`.`name`, `Product`.`obs`, `Product`.`disabled`, `ProductCharacteristic`.`id`, `ProductCharacteristic`.`productInstanceId`, `ProductCharacteristic`.`characteristicInstanceId`
FROM `ProductInstance` INNER JOIN `Product` ON `ProductCharacteristic`.`productInstanceId` = `ProductInstance`.`id` INNER JOIN `ProductCharacteristic` ON `ProductInstance`.`productId` = `Product`.`id`;
It is flipping the on clause in the JOIN!! How can I make a workaround???
The correct select instruction should be:
SELECT `ProductInstance`.`id`, `ProductInstance`.`productId`, `ProductInstance`.`code`, `ProductInstance`.`barCode`, `ProductInstance`.`price`, `ProductInstance`.`buyingPrice`, `ProductInstance`.`proportional`, `ProductInstance`.`obs`, `ProductInstance`.`disabled`, `Product`.`id`, `Product`.`codePrefix`, `Product`.`defaultPrice`, `Product`.`defaultBuyingPrice`, `Product`.`name`, `Product`.`obs`, `Product`.`disabled`, `ProductCharacteristic`.`id`, `ProductCharacteristic`.`productInstanceId`, `ProductCharacteristic`.`characteristicInstanceId`
FROM `ProductInstance` INNER JOIN `Product` ON `ProductInstance`.`productId` = `Product`.`id` INNER JOIN `ProductCharacteristic` ON `ProductCharacteristic`.`productInstanceId` = `ProductInstance`.`id`;
Related
I'm working on a project resource management application and my resource table has several fields, one of which is an enum as below:
CREATE TYPE "clearance" AS ENUM (
'None',
'Baseline',
'NV1',
'NV2',
'TSPV'
);
Then, my resource table includes that enum:
CREATE TABLE "resource" (
"employee_id" integer PRIMARY KEY,
"name" varchar NOT NULL,
"email" varchar NOT NULL,
"job_title_id" integer NOT NULL,
"manager_id" integer NOT NULL,
"workgroup_id" integer NOT NULL,
"clearance_level" clearance,
"specialties" text[],
"certifications" text[],
"active" boolean DEFAULT 't'
);
When querying the data, I want to be able to provide query string parameters in the url, that then apply filters to the database query.
For example (using a local dev machine):
curl localhost:6543/v1/resources # returns all resources in a paginated query
curl localhost:6543/v1/resources?specialties=NSX # returns all resources with NSX as a specialty
curl localhost:6543/v1/resources?manager=John+Smith # returns resources that report to John Smith
curl localhost:6543/v1/resources?jobTitle=Senior+Consultant # returns all Senior Consultants
etc.
Where I'm running into an issue though is that I also want to be able to filter on the security clearance level like this:
curl localhost:6543/v1/resources?clearance=NV2
When I provide a clearance filter I can get the query to work fine:
query := fmt.Sprintf(`
SELECT count(*) OVER(), r.employee_id, r.name, r.email, job_title.title, m.name AS manager, workgroup.workgroup_name, r.clearance_level, r.specialties, r.certifications, r.active
FROM (((resource r
INNER JOIN job_title ON r.job_title_id=job_title.title_id)
INNER JOIN resource m ON r.manager_id=m.employee_id)
INNER JOIN workgroup ON workgroup.workgroup_id=r.workgroup_id)
WHERE (workgroup.workgroup_name = ANY($1) OR $1 = '{}')
AND (r.clearance_level = $2::clearance)
AND (r.specialties #> $3 OR $3 = '{}')
AND (r.certifications #> $4 OR $4 = '{}')
AND (m.name = $5 OR $5 = '')
AND (r.active = $6)
AND (r.name = $7 OR $7 = '')
ORDER BY %s %s, r.employee_id ASC
LIMIT $8 OFFSET $9`, clearance_filter, fmt.Sprintf("r.%s", filters.sortColumn()), filters.sortDirection())
However, I can't figure out a reasonably way to implement the filtering, so that all results are returned when no clearance filter is provided.
The poor way I have made it work is to just apply an empty string filter on another field when no clearance is filtered for and substitute in the correct filter when a clearance argument is provided.
It works, but smells really bad:
func (m *ResourceModel) GetAll(name string, workgroups []string, clearance string, specialties []string,
certifications []string, manager string, active bool, filters Filters) ([]*Resource, Metadata, error) {
// THIS IS A SMELL
// Needed to provide a blank filter parameter if all clearance levels should be returned.
// Have not found a good way to filter on enums to include all values when no filter argument is provided
var clearance_filter = `AND (r.name = $2 OR $2 = '')`
if clearance != "" {
clearance_filter = `AND (r.clearance_level = $2::clearance)`
}
query := fmt.Sprintf(`
SELECT count(*) OVER(), r.employee_id, r.name, r.email, job_title.title, m.name AS manager, workgroup.workgroup_name, r.clearance_level, r.specialties, r.certifications, r.active
FROM (((resource r
INNER JOIN job_title ON r.job_title_id=job_title.title_id)
INNER JOIN resource m ON r.manager_id=m.employee_id)
INNER JOIN workgroup ON workgroup.workgroup_id=r.workgroup_id)
WHERE (workgroup.workgroup_name = ANY($1) OR $1 = '{}')
%s
AND (r.specialties #> $3 OR $3 = '{}')
AND (r.certifications #> $4 OR $4 = '{}')
AND (m.name = $5 OR $5 = '')
AND (r.active = $6)
AND (r.name = $7 OR $7 = '')
ORDER BY %s %s, r.employee_id ASC
LIMIT $8 OFFSET $9`, clearance_filter, fmt.Sprintf("r.%s", filters.sortColumn()), filters.sortDirection())
...
...
}
Is there a better way to approach this?
It feels like a really poor solution to the point that I'm thinking of dropping the enum and making it another table that just establishes a domain of values:
CREATE TABLE clearance (
"level" varchar NOT NULL
);
For anyone that needs this very niche use case in the future, the answer was built on the initial hint from #mkopriva
The approach was to cast the clearance_level to text, so the filter is:
...
AND(r.clearance_level::text = $2 OR $2 = '')
...
This returns all results, regardless of clearance when no clearance filter is provided and returns only the result that match the provided clearance_level when a filter is provided.
Must appreciated to #mkopriva for the assistance.
Retrieve the product number, name, and list price of products whose product number begins 'BK-' followed by any character other than 'R’, and ends with a '-' followed by any two numerals. Question belongs to Lab file i'm working on. Below is what i'v tried:
Select p.ProductNumber, p.Name,p.ListPrice,p.ProductNumber
From SalesLT.Product as p
Where p.ProductNumber Like 'BK-%[^r]%-[0-9][0-9]'
Column Name
FR-R32B-78
FR-R32R-78
HL-U703-R
HL-U703
SO-B303-M
SO-B303-L
HL-U703-B
CA-1038
LJ-0132-S
LJ-0132-M
BK-M82S-32
BK-M82S-33
BK-M82S-38
BK-R33R-62
BK-R33R-44
The problem is you have % before [^r]
Select p.ProductNumber, p.Name,p.ListPrice,p.ProductNumber
From SalesLT.Product as p
Where p.ProductNumber Like 'BK-[^r]%-[0-9][0-9]'
This should work fine i think.
The reason is %[^r]% means - Any symbols followed by not r followed by any symbols. Which is true for any of them.
Example R33R -> R is any symbols , 3 is not r and 3R is any symbols.
declare #t table (ProductNumber varchar(100));
insert into #t
values
('FR-R32B-78'),
('FR-R32R-78'),
('HL-U703-R'),
('HL-U703'),
('SO-B303-M'),
('SO-B303-L'),
('HL-U703-B'),
('CA-1038'),
('LJ-0132-S'),
('LJ-0132-M'),
('BK-M82S-32'),
('BK-M82S-33'),
('BK-M82S-38'),
('BK-R33R-62'),
('BK-R33R-44')
Select *
From #t
Where ProductNumber Like 'BK-%-[0-9][0-9]' and ProductNumber not like '___R%';
I am inside a function in a controller.
So from the Form, I get a value for a variable, say:
$x = "whatever";
Then I need to embed that variable (so, its value), in the WHERE statement. If I hardcode the value, it brings a correct result, but I have tried in all ways to insert that variable without success. Well, supposing that I manage to use that variable, then I will have to look into binding to avoid sql injection, but so far, I would say, see if that variable can get used in the query.
I have tried, double quotes, concatenation . $vx . , curly braces {$x}, the variable plain like this $variable, but either gives syntax errors in some cases, (concatenation), or if I just embed the variable like this where author = $x, it tells me that it can't find the column named $x
$x = "whatever";
$results = DB::select(DB::raw('SELECT
t.id, t.AvgStyle, r.RateDesc
FROM (
SELECT
p.id, ROUND(AVG(s.Value)) AS AvgStyle
FROM posts p
INNER JOIN styles s
ON s.post_id = p.id
WHERE author = $x
GROUP BY p.id
) t
INNER JOIN rates r
ON r.digit = t.AvgStyle'
));
This appears to be a simple PHP variable interpolation issue.
DB::raw() wants literally raw SQL. So there are a couple of issues that need to be fixed in the SQL string you are passing.
PHP Variable interpolation (injecting variables into a string) only happens if you use double quotes around the string. With single quotes it becomes a string constant.
If Author is a char/varchar, then SQL syntax requires quotes around the string in your raw SQL statement. Query builders typically take care of these issues for you, but you are going around them.
So the "fixed" version of this would be:
$x = "whatever";
$results = DB::select(DB::raw("SELECT
t.id, t.AvgStyle, r.RateDesc
FROM (
SELECT
p.id, ROUND(AVG(s.Value)) AS AvgStyle
FROM posts p
INNER JOIN styles s
ON s.post_id = p.id
WHERE author = '$x'
GROUP BY p.id
) t
INNER JOIN rates r
ON r.digit = t.AvgStyle"
));
Like all interpolation, this opens you up to the possibility of SQL injection if the variable being interpolated comes from user input. From the original question it is unclear whether this is a problem.
DB::select() has an option that allows you to pass an array of parameters that is inherently safe from SQL injection. In that case the solution would be:
$x = "whatever";
$results = DB::select(DB::raw("SELECT
t.id, t.AvgStyle, r.RateDesc
FROM (
SELECT
p.id, ROUND(AVG(s.Value)) AS AvgStyle
FROM posts p
INNER JOIN styles s
ON s.post_id = p.id
WHERE author = :author
GROUP BY p.id
) t
INNER JOIN rates r
ON r.digit = t.AvgStyle"
),
array('author' => $x)
);
Regarding this tutorial
$results = DB::select( DB::raw("SELECT * FROM some_table WHERE some_col = :somevariable"), array(
'somevariable' => $someVariable,
));
This is one example for you to insert variable in a raw sql laravel
$query_result = Event::select(
DB::raw('(CASE WHEN status = "draft" THEN "draft"
WHEN events.end_time <= \''.$now.'\' THEN "closed"
ELSE "available"
END) AS status'))
->orderBy('status')
->get();
I am doing a paginated resource, which will require an inner select, which I've already designed in sql terms. It has the following structure:
select *
from (
select w.*, d.distance
from `Work` w
inner join AdrDistance d on d.nhood2 = w.nhood
where d.nhood1 = 1 -- this will be a variable
order by d.distance
limit 0, 10 -- this will be pagination
) w
inner join WImage wi on wi.`work` = w.id
My entity definitions:
Work
...
WImage
work WorkId
url Text
AdrNhood
city AdrCityId
name Text maxlen=100
lat Double
lng Double
-- This is a view with a computed column I used for ordering
AdrDistance
nhood1 AdrNhoodId
nhood2 AdrNhoodId
distance Distance -- type Distance = Int - in Meters
How can I define such a select in Esqueleto, which would resemble such structure (by doing one single query of course)?
Update
I tried to follow this path:
worksByNhood nId offset' limit' =
from $ \wi -> do
(w, d) <- from $ \(w `InnerJoin` d) -> do
on $ d ^. AdrDistanceNhood2 ==. w ^. WorkNhood
where_ (d ^. AdrDistanceNhood1 ==. val nId)
orderBy [asc (d ^. AdrDistanceDistance)]
offset offset'
limit limit'
return (w, d)
where_ (wi ^. WImageWork ==. w ^. WorkId)
return (w, d ^. AdrDistanceDistance, wi)
But it didn't drive me to the correct solution. If someone can help me (even saying that I would be better doing several selects because what I am trying is not viable in Esqueleto), please, comment or answer my question.
I have read this issue on github
and I concluded Esqueleto wasn't designed to support selects in froms the way I was trying, so I did differently:
worksByNhood nId offset' limit' = do
works <- select $ from $ \(w `InnerJoin` d) -> do
on $ d ^. AdrDistanceNhood2 ==. w ^. WorkNhood
where_ (d ^. AdrDistanceNhood1 ==. val nId)
orderBy [asc (d ^. AdrDistanceDistance)]
offset offset'
limit limit'
return (w, d ^. AdrDistanceDistance)
works' <- forM works $ \(w#(Entity wId _), d) -> do
images <- select $ from $ \wi -> do
where_ (wi ^. WImageWork ==. val wId)
return wi
return (w, d, images);
return works'
It is not exactly what I was looking for, but for now I will use it. If somebody have a better approach, please, tell me.
How can I compose Database.Esqueleto queries in a modular way such that after defining a "base" query and the corresponding result set, I can restrict the result set by adding additional inner joins and where expressions.
Also, how can I convert the base query that returns a list of entities (or field tuples) into a query that counts the result set since the base query is not executed as such, but a modified version of it with LIMIT and OFFSET.
The following incorrect Haskell code snippet adopted from the Yesod Book hopefully clarifies what I'm aiming at.
{-# LANGUAGE QuasiQuotes, TemplateHaskell, TypeFamilies, OverloadedStrings #-}
{-# LANGUAGE GADTs, FlexibleContexts #-}
import qualified Database.Persist as P
import qualified Database.Persist.Sqlite as PS
import Database.Persist.TH
import Control.Monad.IO.Class (liftIO)
import Data.Conduit
import Control.Monad.Logger
import Database.Esqueleto
import Control.Applicative
share [mkPersist sqlSettings, mkMigrate "migrateAll"] [persistLowerCase|
Person
name String
age Int Maybe
deriving Show
BlogPost
title String
authorId PersonId
deriving Show
Comment
comment String
blogPostId BlogPostId
|]
main :: IO ()
main = runStdoutLoggingT $ runResourceT $ PS.withSqliteConn ":memory:" $ PS.runSqlConn $ do
runMigration migrateAll
johnId <- P.insert $ Person "John Doe" $ Just 35
janeId <- P.insert $ Person "Jane Doe" Nothing
jackId <- P.insert $ Person "Jack Black" $ Just 45
jillId <- P.insert $ Person "Jill Black" Nothing
blogPostId <- P.insert $ BlogPost "My fr1st p0st" johnId
P.insert $ BlogPost "One more for good measure" johnId
P.insert $ BlogPost "Jane's" janeId
P.insert $ Comment "great!" blogPostId
let baseQuery = select $ from $ \(p `InnerJoin` b) -> do
on (p ^. PersonId ==. b ^. BlogPostAuthorId)
where_ (p ^. PersonName `like` (val "J%"))
return (p,b)
-- Does not compile
let baseQueryLimited = (,) <$> baseQuery <*> (limit 2)
-- Does not compile
let countingQuery = (,) <$> baseQuery <*> (return countRows)
-- Results in invalid SQL
let commentsQuery = (,) <$> baseQuery
<*> (select $ from $ \(b `InnerJoin` c) -> do
on (b ^. BlogPostId ==. c ^. CommentBlogPostId)
return ())
somePosts <- baseQueryLimited
count <- countingQuery
withComments <- commentsQuery
liftIO $ print somePosts
liftIO $ print ((head count) :: Value Int)
liftIO $ print withComments
return ()
Looking at the documentation and the type of select:
select :: (...) => SqlQuery a -> SqlPersistT m [r]
It's clear that upon calling select, we leave the world of pure composable queries (SqlQuery a) and enter the world of side effects (SqlPersistT m [r]). So we simply need to compose before we select.
let baseQuery = from $ \(p `InnerJoin` b) -> do
on (p ^. PersonId ==. b ^. BlogPostAuthorId)
where_ (p ^. PersonName `like` (val "J%"))
return (p,b)
let baseQueryLimited = do r <- baseQuery; limit 2; return r
let countingQuery = do baseQuery; return countRows
somePosts <- select baseQueryLimited
count <- select countingQuery
This works for limiting and counting. I haven't figured out how to do it for joins yet, but it looks like it should be possible.
For LIMIT and COUNT, hammar's answer is entirely correct so I'll not delve into them. I'll just reiterate that once you use select you'll not be able to change the query in any way again.
For JOINs, currently you are not able to do a INNER JOIN with a query that was defined in a different from (nor (FULL|LEFT|RIGHT) OUTER JOINs). However, you can do implicit joins. For example, if you have defined:
baseQuery =
from $ \(p `InnerJoin` b) -> do
on (p ^. PersonId ==. b ^. BlogPostAuthorId)
where_ (p ^. PersonName `like` val "J%")
return (p, b)
Then you may just say:
commentsQuery =
from $ \c -> do
(p, b) <- baseQuery
where_ (b ^. BlogPostId ==. c ^. CommentBlogPostId)
return (p, b, c)
Esqueleto then will generate something along the lines of:
SELECT ...
FROM Comment, Person INNER JOIN BlogPost
ON Person.id = BlogPost.authorId
WHERE Person.name LIKE "J%"
AND BlogPost.id = Comment.blogPostId
Not pretty but gets the job done for INNER JOINs. If you need to do a OUTER JOIN then you'll have to refactor your code so that all the OUTER JOINs are in the same from (note that you can do an implicit join between OUTER JOINs just fine).