I'm building a Node.js app that needs to query a Redshift database (based on postgres 8.0.2) using CTEs. Unfortunately, the SQL query builders I've looked at thus far (node-sql, knex.js and sequelize) don't seem to support common table expressions (CTEs).
I had great success forming common table expressions in Ruby using Jeremy Evans' Sequel gem, which has a with method that takes two arguments to define the tables' aliased name and the dataset reference. I'd like something similar in Node.
Have I missed any obvious contenders for Node.js SQL query builders? From what I can tell these are the four most obvious:
node-sql
nodesql (no postgres support?)
knex.js
sequelize
knex.js now supports WITH clauses:
knex.with('with_alias', (qb) => {
qb.select('*').from('books').where('author', 'Test')
}).select('*').from('with_alias')
Outputs:
with "with_alias" as (select * from "books" where "author" = 'Test') select * from "with_alias"
I was able to use common table expressions (CTEs) with knex.js and it was pretty easy.
Assuming you're using socket.io along with knex.js,
knex-example.js:
function knexExample (io, knex) {
io.on('connection', function (socket) {
var this_cte = knex('this_table').select('this_column');
var that_cte = knex('that_table').select('that_column');
knex.raw('with t1 as (' + this_cte +
'), t2 as (' + that_cte + ')' +
knex.select(['this', 'that'])
.from(['t1', 't2'])
)
.then(function (rows) {
socket.emit('this_that:update', rows);
});
})
}
module.exports = knexExample;
From this issue and this issue I understand that you can use CTEs with Sequelize.
You need to use the raw queries and maybe precise the type, to be sure that Sequelize understands it's a Select query. See first link.
Sample code would be:
sequelize.query(
query, //raw SQL
tableName,
{raw: true, type: Sequelize.QueryTypes.SELECT}
).success(function (rows) {
// ...
})
See here for mode details.
xql.js supports WITH clause from version 1.4.12. A small example:
const xql = require("xql");
const SELECT = xql.SELECT;
return SELECT()
.WITH("with_alias", SELECT().FROM("books").WHERE("author", "=", "Test"))
.FROM("with_alias");
The WITH clause is supported for SELECT, INSERT, UPDATE, DELETE, and compound queries as well.
Related
I am trying to build a case query with distinct count in cakephp 3.
This is the query in SQL:
select COUNT(distinct CASE WHEN type = 'abc' THEN app_num END) as "count_abc",COUNT(distinct CASE WHEN type = 'xyz' THEN app_num END) as "count_xyz" from table;
Currently, I got this far:
$query = $this->find();
$abc_case = $query->newExpr()->addCase($query->newExpr()->add(['type' => 'abc']),' app_num','string');
$xyz_case = $query->newExpr()->addCase($query->newExpr()->add(['type' => 'xyz']),'app_num','string');
$query->select([
"count_abc" => $query->func()->count($abc_case),
"count_xyz" => $query->func()->count($xyz_case),
]);
But I can't apply distinct in this code.
Using keywords in functions has been a problem for quite some time, see for example this issue ticket: https://github.com/cakephp/cakephp/issues/10454.
This has been somewhat improved in https://github.com/cakephp/cakephp/pull/11410, so that it's now possible to (mis)use a function expression for DISTINCT as kind of a workaround, ie generate code like DISTINCT(expression), which works because the parentheses are being ignored, so to speak, as DISTINCT is not a function!
I'm not sure if this works because the SQL specifications explicitly allow parentheses to be used like that (also acting as a whitespace substitute), or because it's a side-effect, so maybe check that out before relying on it!
That being said, you can use the workaround from the linked PR until real aggregate function keyword support is being added, ie do something like this:
"count_abc" => $query->func()->count(
$query->func()->DISTINCT([$abc_case])
)
This would generate SQL similar to:
(COUNT(DISTINCT(CASE WHEN ... END)))
I have a SQL query (Postgres) containing several joins that I'm having trouble converting into a single Knex.js statement. Here's the SQL query:
SELECT
User.id, User.name, User.email,
Role.name AS r_name,
UserPofile.id AS p_id, UserPofile.date_of_birth AS p_dob,
AuthToken.id AS at_id, AuthToken.token AS at_token, AuthToken.platform AS at_platform
FROM public."user" User
LEFT JOIN public."user_role" UserRole ON User.id = UserRole.user_id
LEFT JOIN public."role" Role ON UserRole.role_id = Role.id
LEFT JOIN public."application" Application ON UserProfile.app_id = Application.id
LEFT JOIN public."user_profile" UserProfile ON User.id = UserProfile.user_id
LEFT JOIN public."auth_token" AuthToken ON User.id = AuthToken.user_id
WHERE
User.email LIKE 'some#email.com' AND
Application.name LIKE 'awesome-application' AND
AuthToken.platform LIKE 'mobile';
Here's my Knex.js code:
return knex('user').where({ email:'some#email.com' })
.select([
'user.id', 'user.name', 'user.email' // User
'role.name AS rName' // Roles
'user_profile.id AS pId', 'user_profile.date_of_birth AS pDob' // UserProfiles
'auth_token.id AS atId' 'auth_token.platform AS atPlatform', 'auth_token.token AS atToken' // AuthTokens
])
.leftJoin('user_profile', 'user_profile.user_id', 'user.id')
.leftJoin('user_role', 'user_role.user_id', 'user.id')
.leftJoin('role', 'role.id', 'user_role.role_id')
.leftJoin('auth_token', 'auth_token.user_id', 'user.id')
.then(users => {
users = users.filter(user => {
return user.pApp_id === appId && user.atApp_id === appId && user.atPlatform === platform;
});
return users;
});
This produces the same result that the SQL query does, but the problem is that I have to filter the returned users in the .then() clause of the Knex call because I don't know how to add WHERE conditions for the Application.name and AuthToken.platform.
Question:
Can someone please help me figure out how to structure my Knex code's .where() clause to have it be equivalent to the SQL query?
Notes:
I don't know how to console.log the SQL queries that Knex produces, therefore I'm not entirely sure that my current Knex code will produce the SQL query above it (minus the correct WHERE clause). I have checked though that it does in fact return the same results, by running the query in PgAdmin and console.log()ing the users returned in from the Knex function.
I haven't included the CREATE TABLE / Knex migrations that defined the tables and columns used in this question, because to me it didn't feel necessary, and I don't want to make an already long question even longer. But if you need to see it, please don't hesitate to let me know. I'll gladly include it.
To debug you can use .toSQL() to debug your knex queries documentation.
Also, nice cheatsheet
As hot fix solution you can use .raw() and paste your SQL code there.
About WHERE conditions, you can just chain them in the end of your knex query.
Something like this:
return knex('user')
.select('user.id', 'user.name', 'user.email',
'role.name AS rName'
'user_profile.id AS pId', 'user_profile.date_of_birth AS pDob',
'auth_token.id AS atId' 'auth_token.platform AS atPlatform', 'auth_token.token AS atToken'
)
.leftJoin('user_profile', 'user_profile.user_id', 'user.id')
.leftJoin('user_role', 'user_role.user_id', 'user.id')
.leftJoin('role', 'role.id', 'user_role.role_id')
.leftJoin('auth_token', 'auth_token.user_id', 'user.id')
.where('user.email', 'like', '%some#email.com%')
.andWhere('application.name', 'like' '%awesome-application%')
//...etc
Simplest solution is use .whereRaw():
knex whereRaw
Example:
.leftJoin('auth_token', 'auth_token.user_id', 'user.id')
.whereRaw('User.email LIKE ?', `${your_param_here}`)
.andWhereRaw('AuthToken.platform LIKE ?', `${your_param_here}`)
I want to retrieve all rows matched on multiple partial prase against with a column. My situation can be explained as following raw sql:
SELECT *
FROM TABLENAME
WHERE COLUMN1 LIKE %abc%
OR COLUMN1 LIKE %bcd%
OR COLUMN1 LIKE %def%;
Here, abc, bcd, def are array elements. i.e: array('abc','bcd','def'). Is there any way to write code passing this array to form the above raw sql using cakephp 3?
N.B: I am using mysql as DBMS.
probably you can use Collection to create a proper array, but I think a foreach loop will do the job in the same amount of code. So here is my solution supposing $query stores your Query object
$a = ['abc','bcd','def'];
foreach($a as $value)
{
$or[] = function ($exp, $q) use ($value) {
return $exp->like('column1', '%'.$value.'%');
};
}
$query->where(['or' => $or]);
you could also use orWhere() but I see it will be deprecated in 3.6
I am trying to query query the current month, here is my query:
$clients = $this->Clients;
$query = $clients->find();
if($this->Auth->user('role') !== 'admin'){
$query->where(['user_id =' => $this->Auth->user('id')]);
$query->where(['MONTH(dob) = ' => 'EXTRACT(month FROM (NOW()))']);
$query->order(['dob' => 'ASC']);
}
It returns 0 records (my field is a date type), however this query in phpmyadmin works:
SELECT * FROM `clients` WHERE MONTH(dob) = EXTRACT(month FROM (NOW()))
What am I doing wrong?
Just look at the actual generated query (check out your DBMS query log, or try DebugKit), it will look different, as the right hand side value in a key => value condition set is subject to parameter-binding/casting/quoting/escaping. In your case it will be treated as a string, so the condition will finally look something like:
WHERE MONTH(dob) = 'EXTRACT(month FROM (NOW()))'
That will of course not match anything.
You could pass the whole SQL snippet as a single array value, or as an expression object, that way it would be inserted into the query as is (do not insert user values that way, that would create an SQL injection vulnerability!), but I'd suggest to use portable function expressions instead.
CakePHP ships with functions expressions for EXTRACT and NOW, so you can simply do something like:
use Cake\Database\Expression\IdentifierExpression;
use Cake\Database\Expression\QueryExpression;
use Cake\ORM\Query;
// ...
$query->where(function (QueryExpression $exp, Query $query) {
return $exp->eq(
$query->func()->extract('MONTH', new IdentifierExpression('dob')),
$query->func()->extract('MONTH', $query->func()->now())
);
});
Looks a bit complicated, but it's worth it, it's cross DBMS portable as well as auto-quoting compatible. The generated SQL will look something like
WHERE EXTRACT(MONTH FROM (dob)) = (EXTRACT(MONTH FROM (NOW())))
See also
Cookbook > Database Access & ORM > Query Builder > Advanced Conditions
Cookbook > Database Access & ORM > Query Builder > Using SQL Functions
API > \Cake\Database\Expression\QueryExpression::eq()
API > \Cake\Database\FunctionsBuilder::extract()
API > \Cake\Database\FunctionsBuilder::now()
I'd like to get value by the following SQL using Eloquent ORM.
- SQL
SELECT COUNT(*) FROM
(SELECT * FROM abc GROUP BY col1) AS a;
Then I considered the following.
- Code
$sql = Abc::from('abc AS a')->groupBy('col1')->toSql();
$num = Abc::from(\DB::raw($sql))->count();
print $num;
I'm looking for a better solution.
Please tell me simplest solution.
In addition to #delmadord's answer and your comments:
Currently there is no method to create subquery in FROM clause, so you need to manually use raw statement, then, if necessary, you will merge all the bindings:
$sub = Abc::where(..)->groupBy(..); // Eloquent Builder instance
$count = DB::table( DB::raw("({$sub->toSql()}) as sub") )
->mergeBindings($sub->getQuery()) // you need to get underlying Query Builder
->count();
Mind that you need to merge bindings in correct order. If you have other bound clauses, you must put them after mergeBindings:
$count = DB::table( DB::raw("({$sub->toSql()}) as sub") )
// ->where(..) wrong
->mergeBindings($sub->getQuery()) // you need to get underlying Query Builder
// ->where(..) correct
->count();
Laravel v5.6.12 (2018-03-14) added fromSub() and fromRaw() methods to query builder (#23476).
The accepted answer is correct but can be simplified into:
DB::query()->fromSub(function ($query) {
$query->from('abc')->groupBy('col1');
}, 'a')->count();
The above snippet produces the following SQL:
select count(*) as aggregate from (select * from `abc` group by `col1`) as `a`
The solution of #JarekTkaczyk it is exactly what I was looking for. The only thing I miss is how to do it when you are using
DB::table() queries. In this case, this is how I do it:
$other = DB::table( DB::raw("({$sub->toSql()}) as sub") )->select(
'something',
DB::raw('sum( qty ) as qty'),
'foo',
'bar'
);
$other->mergeBindings( $sub );
$other->groupBy('something');
$other->groupBy('foo');
$other->groupBy('bar');
print $other->toSql();
$other->get();
Special atention how to make the mergeBindings without using the getQuery() method
From laravel 5.5 there is a dedicated method for subqueries and you can use it like this:
Abc::selectSub(function($q) {
$q->select('*')->groupBy('col1');
}, 'a')->count('a.*');
or
Abc::selectSub(Abc::select('*')->groupBy('col1'), 'a')->count('a.*');
There are many readable ways to do these kinds of queries at the moment (Laravel 8).
// option 1: DB::table(Closure, alias) for subquery
$count = DB::table(function ($sub) {
$sub->from('abc')
->groupBy('col1');
}, 'a')
->count();
// option 2: DB::table(Builder, alias) for subquery
$sub = DB::table('abc')->groupBy('col1');
$count = DB::table($sub, 'a')->count();
// option 3: DB::query()->from(Closure, alias)
$count = DB::query()
->from(function ($sub) {
$sub->from('abc')
->groupBy('col1')
}, 'a')
->count();
// option 4: DB::query()->from(Builder, alias)
$sub = DB::table('abc')->groupBy('col1');
$count = DB::query()->from($sub, 'a')->count();
For such small subqueries, you could even try fitting them in a single line with PHP 7.4's short closures but this approach can be harder to mantain.
$count = DB::table(fn($sub) => $sub->from('abc')->groupBy('col1'), 'a')->count();
Note that I'm using count() instead of explicitly writing the count(*) statement and using get() or first() for the results (which you can easily do by replacing count() with selectRaw(count(*))->first()).
The reason for this is simple: It returns the number instead of an object with an awkwardly named property (count(*) unless you used an alias in the query)
Which looks better?
// using count() in the builder
echo $count;
// using selectRaw('count(*)')->first() in the builder
echo $count->{'count(*)'};
Correct way described in this answer: https://stackoverflow.com/a/52772444/2519714
Most popular answer at current moment is not totally correct.
This way https://stackoverflow.com/a/24838367/2519714 is not correct in some cases like: sub select has where bindings, then joining table to sub select, then other wheres added to all query. For example query:
select * from (select * from t1 where col1 = ?) join t2 on col1 = col2 and col3 = ? where t2.col4 = ?
To make this query you will write code like:
$subQuery = DB::query()->from('t1')->where('t1.col1', 'val1');
$query = DB::query()->from(DB::raw('('. $subQuery->toSql() . ') AS subquery'))
->mergeBindings($subQuery->getBindings());
$query->join('t2', function(JoinClause $join) {
$join->on('subquery.col1', 't2.col2');
$join->where('t2.col3', 'val3');
})->where('t2.col4', 'val4');
During executing this query, his method $query->getBindings() will return bindings in incorrect order like ['val3', 'val1', 'val4'] in this case instead correct ['val1', 'val3', 'val4'] for raw sql described above.
One more time correct way to do this:
$subQuery = DB::query()->from('t1')->where('t1.col1', 'val1');
$query = DB::query()->fromSub($subQuery, 'subquery');
$query->join('t2', function(JoinClause $join) {
$join->on('subquery.col1', 't2.col2');
$join->where('t2.col3', 'val3');
})->where('t2.col4', 'val4');
Also bindings will be automatically and correctly merged to new query.
I like doing something like this:
Message::select('*')
->from(DB::raw("( SELECT * FROM `messages`
WHERE `to_id` = ".Auth::id()." AND `isseen` = 0
GROUP BY `from_id` asc) as `sub`"))
->count();
It's not very elegant, but it's simple.
This works fine
$q1 = DB::table('tableA')->groupBy('col');
$data = DB::table(DB::raw("({$q1->toSql()}) as sub"))->mergeBindings($q1)->get();
I could not made your code to do the desired query, the AS is an alias only for the table abc, not for the derived table.
Laravel Query Builder does not implicitly support derived table aliases, DB::raw is most likely needed for this.
The most straight solution I could came up with is almost identical to yours, however produces the query as you asked for:
$sql = Abc::groupBy('col1')->toSql();
$count = DB::table(DB::raw("($sql) AS a"))->count();
The produced query is
select count(*) as aggregate from (select * from `abc` group by `col1`) AS a;
->selectRaw('your subquery as somefield')
Deriving off mpskovvang's answer, here is what it would look like using eloquent model. (I tried updating mpskovvang answer to include this, but there's too many edit requests for it.)
$qry = Abc::where('col2', 'value')->groupBy('col1')->selectRaw('1');
$num = Abc::from($qry, 'q1')->count();
print $num;
Produces...
SELECT COUNT(*) as aggregate FROM (SELECT 1 FROM Abc WHERE col2='value' GROUP BY col1) as q1