Upsert in KnexJS - sql

I have an upsert query in PostgreSQL like:
INSERT INTO table
(id, name)
values
(1, 'Gabbar')
ON CONFLICT (id) DO UPDATE SET
name = 'Gabbar'
WHERE
table.id = 1
I need to use knex to this upsert query. How to go about this?

So I solved this using the following suggestion from Dotnil's answer on Knex Issues Page:
var data = {id: 1, name: 'Gabbar'};
var insert = knex('table').insert(data);
var dataClone = {id: 1, name: 'Gabbar'};
delete dataClone.id;
var update = knex('table').update(dataClone).whereRaw('table.id = ' + data.id);
var query = `${ insert.toString() } ON CONFLICT (id) DO UPDATE SET ${ update.toString().replace(/^update\s.*\sset\s/i, '') }`;
return knex.raw(query)
.then(function(dbRes){
// stuff
});
Hope this helps someone.

As of knex#v0.21.10+ a new method onConflict was introduced.
Official documentation says:
Implemented for the PostgreSQL, MySQL, and SQLite databases. A
modifier for insert queries that specifies alternative behaviour in
the case of a conflict. A conflict occurs when a table has a PRIMARY
KEY or a UNIQUE index on a column (or a composite index on a set of
columns) and a row being inserted has the same value as a row which
already exists in the table in those column(s). The default behaviour
in case of conflict is to raise an error and abort the query. Using
this method you can change this behaviour to either silently ignore
the error by using .onConflict().ignore() or to update the existing
row with new data (perform an "UPSERT") by using
.onConflict().merge().
So in your case, the implementation would be:
knex('table')
.insert({
id: id,
name: name
})
.onConflict('id')
.merge()

I've created a function for doing this and described it on the knex github issues page (along with some of the gotchas for dealing with composite unique indices).
const upsert = (params) => {
const {table, object, constraint} = params;
const insert = knex(table).insert(object);
const update = knex.queryBuilder().update(object);
return knex.raw(`? ON CONFLICT ${constraint} DO ? returning *`, [insert, update]).get('rows').get(0);
};
Example usage:
const objToUpsert = {a:1, b:2, c:3}
upsert({
table: 'test',
object: objToUpsert,
constraint: '(a, b)',
})
A note about composite nullable indices
If you have a composite index (a,b) and b is nullable, then values (1, NULL) and (1, NULL) are considered mutually unique by Postgres (I don't get it either).

Yet another approach I could think of!
exports.upsert = (t, tableName, columnsToRetain, conflictOn) => {
const insert = knex(tableName)
.insert(t)
.toString();
const update = knex(tableName)
.update(t)
.toString();
const keepValues = columnsToRetain.map((c) => `"${c}"=${tableName}."${c}"`).join(',');
const conflictColumns = conflictOn.map((c) => `"${c.toString()}"`).join(',');
let insertOrUpdateQuery = `${insert} ON CONFLICT( ${conflictColumns}) DO ${update}`;
insertOrUpdateQuery = keepValues ? `${insertOrUpdateQuery}, ${keepValues}` : insertOrUpdateQuery;
insertOrUpdateQuery = insertOrUpdateQuery.replace(`update "${tableName}"`, 'update');
insertOrUpdateQuery = insertOrUpdateQuery.replace(`"${tableName}"`, tableName);
return Promise.resolve(knex.raw(insertOrUpdateQuery));
};

very simple.
Adding onto Dorad's answer, you can choose specific columns to upsert using merge keyword.
knex('table')
.insert({
id: id,
name: name
})
.onConflict('id')
.merge(['name']); // put column names inside an array which you want to merge.

Related

How to count number of user id in a list without duplicate in sequelize [duplicate]

I am trying to get a distinct count of a particular column using sequelize. My initial attempt is using the 'count' method of my model, however it doesn't look like this is possible.
The DISTINCT feature is needed because I am joining other tables and filtering the rows of the parent based on the related tables.
here's the query I would like:
SELECT COUNT(DISTINCT Product.id) as `count`
FROM `Product`
LEFT OUTER JOIN `Vendor` AS `vendor` ON `vendor`.`id` = `Product`.`vendorId`
WHERE (`vendor`.`isEnabled`=true );
using the following query against my Product model:
Product.count({
include: [{model: models.Vendor, as: 'vendor'}],
where: [{ 'vendor.isEnabled' : true }]
})
Generates the following query:
SELECT COUNT(*) as `count`
FROM `Product`
LEFT OUTER JOIN `Vendor` AS `vendor` ON `vendor`.`id` = `Product`.`vendorId`
WHERE (`vendor`.`isEnabled`=true );
UPDATE: New version
There are now separate distinct and col options. The docs for distinct state:
Apply COUNT(DISTINCT(col)) on primary key or on options.col.
You want something along the lines of:
MyModel.count({
include: ...,
where: ...,
distinct: true,
col: 'Product.id'
})
.then(function(count) {
// count is an integer
});
Original Post
(As mentioned in the comments, things have changed since my original post, so you probably want to ignore this part.)
After looking at Model.count method in lib/model.js, and tracing some code, I found that when using Model.count, you can just add any kind of aggregate function arguments supported by MYSQL to your options object. The following code will give you the amount of different values in MyModel's someColumn:
MyModel.count({distinct: 'someColumn', where: {...}})
.then(function(count) {
// count is an integer
});
That code effectively generates a query of this kind: SELECT COUNT(args) FROM MyModel WHERE ..., where args are all properties in the options object that are not reserved (such as DISTINCT, LIMIT and so on).
The Sequelize documentation on count links to a count method that doesn't let you specify which column to get the count of distinct values:
Model.prototype.count = function(options) {
options = Utils._.clone(options || {});
conformOptions(options, this);
Model.$injectScope(this.$scope, options);
var col = '*';
if (options.include) {
col = this.name + '.' + this.primaryKeyField;
expandIncludeAll.call(this, options);
validateIncludedElements.call(this, options);
}
Utils.mapOptionFieldNames(options, this);
options.plain = options.group ? false : true;
options.dataType = new DataTypes.INTEGER();
options.includeIgnoreAttributes = false;
options.limit = null;
options.offset = null;
options.order = null;
return this.aggregate(col, 'count', options);
};
Basically SELECT COUNT(DISTINCT(*)) or SELECT COUNT(DISTINCT(primaryKey)) if you've got a primary key defined.
To do the Sequelize equivalent of SELECT category, COUNT(DISTINCT(product)) as 'countOfProducts' GROUP BY category, you'd do:
model.findAll({
attributes: [
'category',
[Sequelize.literal('COUNT(DISTINCT(product))'), 'countOfProducts']
],
group: 'category'
})
Looks like this is now supported in Sequelize versions 1.7.0+.
the count and findAndCountAll methods of a model will give you 'real' or 'distinct' count of your parent model.
I was searching for SELECT COUNT(0) query for sequelize, below is the answer for that.
let existingUsers = await Users.count({
where: whereClouser,
attributes: [[sequelize.fn('COUNT', 0), 'count']]
});
This helped me to get distinct count from another table rows,
dataModel.findAll({
attributes: {
include: [[Sequelize.literal("COUNT(DISTINCT(history.data_id))"), "historyModelCount"]]
},
include: [{
model: historyModel, attributes: []
}],
group: ['data.id']
});
Ref 1, Ref 2.
With respect to your question in order to get the distinct counts of products based on the id of product
you just need to pass the key 'distinct' with value 'id' to your count object , Here is the example
To generate this sql query as you asked
SELECT COUNT(DISTINCT(`Product`.`id`)) as `count`
FROM `Product`
LEFT OUTER JOIN `Vendor` AS `vendor` ON `vendor`.`id` = `Product`.`vendorId`
WHERE (`vendor`.`isEnabled`=true );
Add 'distinct' key in your Sequelize query
Product.count({
include: [{model: models.Vendor, as: 'vendor'}],
where: [{ 'vendor.isEnabled' : true }],
distinct: 'id' // since count is applied on Product model and distinct is directly passed to its object so Product.id will be selected
});
This way of using 'distinct' key to filter out distinct counts or rows , I tested in Sequelize Version 6.
Hope this will help you or somebody else!

How to 'replace' table name in raw SQL query?

I have the following SQL query, which works:
await sequelize.query(
"DELETE FROM `table_name` WHERE (?) IN (?)",
{
replacements: ["project_id", projectIds],
type: QueryTypes.DELETE,
}
);
But I also want to use a replacement for table_name like this:
await sequelize.query(
"DELETE FROM (?) WHERE (?) IN (?)",
{
replacements: ["table_name", "project_id", projectIds],
type: QueryTypes.DELETE,
}
);
But this doesn't work and generates an error about SQL syntax. How can I make this work?
You are mixing data value binding and quoting identifiers.
There is ancient issue in the repo: https://github.com/sequelize/sequelize/issues/4494, which sounds like the problem above.
I believe you can create a workaround that respects different sql dialects like this:
const queryInterface = sequelize.getQueryInterface();
const tableName = queryInterface.quoteIdentifier("projects");
const columnName = queryInterface.quoteIdentifier("project_id");
await sequelize.query(`DELETE FROM ${tableName} WHERE ${columnName} IN (?)`, {
replacements: [project_ids],
type: QueryTypes.DELETE,
});
Assuming you are using sequelize 6.x.

How to delete object with Id in Realm or Primary Key?

I have below object :
obj1 = [{ id = 1, name = "abc"}, {id=2, name="pqr"}, {id=3, name="xyz"}]
I need to delete an object with id=2 where id is Primary Key too.
Below method using to delete the object
const collection = RealmDB.realm
.objects("StudentName")
.filtered(`id= $0`, '65');
RealmDB.realm.write(() => {
RealmDB.realm.delete(collection);
});
But it is not working with id object can anyone please suggest better way to do this ?
But still that object is there so may I know what is wrong here.
const id = 1;
realm.write(()=>{
realm.delete(realm.objectForPrimaryKey('Baby',id));
})
Try this.
Hello resolved an issue by following query with no error
const collection = RealmDB.realm
.objects('StudentName')
.filtered('id= $0', `65`);
RealmDB.realm.write(() => {
RealmDB.realm.delete(collection);
Thank you

Linq2DB can't translate a mapped column in Where clause

I'm working with a legacy Oracle database that has a column on a table which stores boolean values as 'Y' or 'N' characters.
I have mapped/converted this column out like so:
MappingSchema.Default.SetConverter<char, bool>(ConvertToBoolean);
MappingSchema.Default.SetConverter<bool, char>(ConvertToChar);
ConvertToBoolean & ConvertToChar are simply functions that map between the types.
Here's the field:
private char hasDog;
[Column("HAS_DOG")]
public bool HasDog
{
get => ConvertToBoolean(hasDog);
set => hasDog = ConvertToChar(value);
}
This has worked well for simply retrieving data, however, it seems the translation of the following:
var humanQuery = (from human in database.Humans
join vetVisit in database.VetVisits on human.Identifier equals vetVisit.Identifier
select new HumanModel(
human.Identifier
human.Name,
human.HasDog,
vetVisit.Date,
vetVisit.Year,
vetVisit.PaymentDue
));
// humanQuery is filtered by year here
var query = from vetVisits in database.VetVisits
select new VetPaymentModel(
(humanQuery).First().Year,
(humanQuery).Where(q => q.HasDog).Sum(q => q.PaymentDue), -- These 2 lines aren't correctly translated to Y/N
(humanQuery).Where(q => !q.HasDog).Sum(q => q.PaymentDue)
);
As pointed out above, the .Where clause here doesn't translate the boolean comparison of HasDog being true/false to the relevant Y/N values, but instead a 0/1 and results in the error
ORA-01722: invalid number
Is there any way to handle this case? I'd like the generated SQL to check that HAS_DOG = 'Y' for instance with the specified Where clause :)
Notes
I'm not using EntityFramework here, the application module that this query exists in doesn't use EF/EFCore
You can define new mapping schema for your particular DataConnection:
var ms = new MappingSchema();
builder = ms.GetFluentMappingBuilder();
builder.Entity<Human>()
.Property(e => e.HasDog)
.HasConversion(v => v ? 'Y' : 'N', p => p == 'Y');
Create this schema ONCE and use when creating DataConnection

Optional column update if provided value for column is not null

I have following table:
CREATE TABLE IF NOT EXISTS categories
(
id SERIAL PRIMARY KEY,
title CHARACTER VARYING(100) NOT NULL,
description CHARACTER VARYING(200) NULL,
category_type CHARACTER VARYING(100) NOT NULL
);
I am using pg-promise, and I want to provide optional update of columns:
categories.update = function (categoryTitle, toUpdateCategory) {
return this.db.oneOrNone(sql.update, [
categoryTitle,
toUpdateCategory.title, toUpdateCategory.category_type, toUpdateCategory.description,
])
}
categoryName - is required
toUpdateCategory.title - is required
toUpdateCategory.category_type - is optional (can be passed or undefined)
toUpdateCategory.description - is optional (can be passed or undefined)
I want to build UPDATE query for updating only provided columns:
UPDATE categories
SET title=$2,
// ... SET category_type=$3 if $3 is no NULL otherwise keep old category_type value
// ... SET description=$4 if $4 is no NULL otherwise keep old description value
WHERE title = $1
RETURNING *;
How can I achieve this optional column update in Postgres?
You could coalesce between the old and the new values:
UPDATE categories
SET title=$2,
category_type = COALESCE($3, category_type),
description = COALESCE($4, description) -- etc...
WHERE title = $1
The helpers syntax is best for any sort of dynamic logic with pg-promise:
/* logic for skipping columns: */
const skip = c => c.value === null || c.value === undefined;
/* reusable/static ColumnSet object: */
const cs = new pgp.helpers.ColumnSet(
[
'title',
{name: 'category_type', skip},
{name: 'description', skip}
],
{table: 'categories'});
categories.update = function(title, category) {
const condition = pgp.as.format(' WHERE title = $1', title);
const update = () => pgp.helpers.update(category, cs) + condition;
return this.db.none(update);
}
And if your optional column-properties do not even exist on the object when they are not specified, you can simplify the skip logic to just this (see Column logic):
const skip = c => !c.exists;
Used API: ColumnSet, helpers.update.
See also a very similar question: Skip update columns with pg-promise.