I have a database with is interfaced through sequelize. I have multiple environments, and for some reason a specific migration did not run on production, but seemingly went trough as expected on our development, staging and test databases. Since then, multiple migrations has been run. Is it possible to safely rerun a specific migration after it is recognized as having been run?
The migration name is 20210316102540-delete_clothes_columns_expired_in_webshop_and_in_store.js
The migration has no down function, so maybe I would be able to db:migrate:undo --name 20210316102540-delete_clothes_columns_expired_in_webshop_and_in_store.js and db:migrate after? Is that a safe approach?
20210316102540-delete_clothes_columns_expired_in_webshop_and_in_store.js
'use strict';
module.exports = {
up: async (queryInterface, Sequelize) => {
queryInterface.sequelize.transaction(async (t) => {
await queryInterface.sequelize.query(`UPDATE clothes SET status = 'ACTIVE' WHERE in_store = TRUE AND status = 'IDLE';`, {transaction: t})
await queryInterface.removeColumn('clothes', 'in_webshop', {transaction: t})
await queryInterface.removeColumn('clothes', 'expired', {transaction: t})
await queryInterface.removeColumn('clothes', 'in_store', {transaction: t})
})
},
down: async (queryInterface, Sequelize) => {
/**
* Add reverting commands here.
*
* Example:
* await queryInterface.dropTable('users');
*/
}
};
To make sure it has been run
production=# select * from "SequelizeMeta";
name
-----------------------------------------------------------------------------------------
...
20210316102540-delete_clothes_columns_expired_in_webshop_and_in_store.js
...
(11 rows)
I am still not sure why the migration had not run. But I ended up deleting the row in SequelizeMeta and rerunning it.
delete from "SequelizeMeta" where name = '20210316102540-delete_clothes_columns_expired_in_webshop_and_in_store.js';
sequelize-cli db:migrate
Related
I know, it sounds really wierd, but it's true.
I was sitting and making an API on node.js, but when I began to test it, I was surprised to find out that nearly in beginning of query treatment when the first res.status().send() reached, VS Code drop a "Cannot set headers after they are sent to the client" error.
How I realize that database query is guilty? I used "res.finished" property to check out when it "sent to client" and discovered that res.finished changes from "false" to "true" exactly after db query.
Who knows what it can be? I had done the same API thing with MySQL database and it went nice, but now I'm using PostgreSQL, so things start to happen.
I export "PostgreSQL manager" class from typescript file
PostgreSQL_Manager.ts:
module.exports = {
PostgreSQL_db_manager
}
Import and initialize it in index.ts:
index.ts
const PostgreSQL_mngr = require('./PostgreSQL_Manager.ts').PostgreSQL_db_manager;
const db = new PostgreSQL_mngr;
And then, if I comment the statement with query to database, res.finished stay false (I tried it with readRows (SELECT) and with createRows(INSERT INTO)):
Piece of index.ts code:
console.log('res.finished : ', res.finished);
//let nickval : any = await db.readRows('users', 'nickname', `nickname = \'${nickname}\'`);
//await db.createRows('test', '(color, odor, taste, quantity)', '(\'meaningless\', \'absent\', \'sadness\', 0)');
console.log('res.finished : ', res.finished);
Piece of Terminal:
res.finished : false
res.finished : false
But when I uncomment database query, it becomes this:
Piece of index.ts code:
console.log('res.finished : ', res.finished);
//let nickval : any = await db.readRows('users', 'nickname', `nickname = \'${nickname}\'`);
await db.createRows('test', '(color, odor, taste, quantity)', '(\'meaningless\', \'absent\', \'sadness\', 0)');
console.log('res.finished : ', res.finished);
Piece of Terminal:
res.finished : false
Postgres: Rows were created...
res.finished : true
Code of db.createRows in postgres manager class looks like this:
public async createRows(table : string, columns: string | string[], values: string | string[]) : Promise<void> {
let createPromise = new Promise<void> ((resolve, reject) => {
this.db.query(`INSERT INTO ${table} ${columns} VALUES ${values};`, (err) => {
if(err) throw err;
console.log('Postgres: Rows were created...');
resolve();
});
});
await createPromise;
}
Edit 1:
There is error occurs (This function called from app.post, nickname, email and password has string values):
async function validationUsers (res : any, email : string = undefined, password : string = undefined, nickname : string = undefined) : Promise<boolean> {
console.log('f:validationUsers email : ', email);
console.log('f:validationUsers password : ', password);
console.log('f:validationUsers : nickname', nickname);
//validation: nickname and email
if(nickname) {
console.log('res.finished : ', res.finished);
//let nickval : any = await db.readRows('users', 'nickname', `nickname = \'${nickname}\'`);
await db.createRows('test', '(color, odor, taste, quantity)', '(\'meaningless\', \'absent\', \'sadness\', 0)');
console.log('res.finished : ', res.finished);
/* if(nickval[0] !== undefined) {
console.log('frofkofro');
res.status(410).send('Nickname already exists');
res.end();
return false;
} */
}
//validation: email
if(email) {
let emailval : any = await db.readRows('users', 'email', `email = \'${email}\'`);
console.log('f:validationUsers if(email) emailval[0] : ', emailval[0]);
if(emailval[0] !== undefined) {
console.log("?00");
res.send('Email already exists');
res.end();
return false;
}
}
//validation: password
if(password) {
let passwordval = /^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{8,100}$/;
if(!password.match(passwordval)) {
console.log('password BAM!!!');
res.status(412).send('Password does not match the criteria'); <-- **THIS STRING**
console.log('password BOOM!!!');
//res.end();
return false;
}
}
console.log('End of f:validationUsers');
return true;
}
Edit 2:
Can it be some problem with pool.query or pool connection to database from "pg" library for PostgreSQL? Or maybe problem with ts-node compiler?
So, I really don't understand what's going on.
I don't know if it's important, but I use ts-node for compile and render typescript
Edit 3:
OKAY, so I started in new ts file new server with the same 5000 port and run THIS:
app1.get('/db', async (req : any, res : any) => {
console.log('res.finished : ', res.finished);
await db1.createRows('test', '(color, odor, taste, quantity)', '(\'meaningless\', \'absent\', \'sadness\', 0)');
console.log('res.finished : ', res.finished);
res.status(200).send('All is fine this is send');
res.end();
});
And result in console:
Connected to database as pool successfully...
Server started on port 5000
res.finished : false
Postgres: Rows were created...
res.finished : false
And POSTMAN received res.send(). wtf??????
The error 'Headers already sent...' happens when your PostgreSQL code sends multiple results using res.send().
I assume the res.send() part is within PostgreSQL manager, which looks like a tested and true library - so what is happening to make it send two answers to one query?
I have no experience with Typescript and pSQL, but I have worked with pSQL and remember hitting this same snag years ago.
PostgreSQL supported (and I imagine it still supports) multiple query mode, such as, UPDATE b SET a=2 WHERE c; SELECT a FROM b. Those are two statements, and the reason why some exploits can even work.
And, just like it happened to me once, even if the second one has zero length and apparently is not even a query, in your code
`INSERT INTO ${table} ${columns} VALUES ${values};`
your PostgreSQL Manager just might think that there are two statements.
So, try removing that apparently harmless ';' at the end and see whether it solves the problem. I wasn't using your libraries, but for me, that did it.
I would say that the error is comming from here :
res.send('Email already exists');
res.end();
Indeed if you node's doc reads :
The res.end() function is used to end the response process. This method actually comes from the Node core, specifically the response.end() method of HTTP.ServerResponse. Use to quickly end the response without any data.
given that you already responsed 'Email already exists' express.js, you recieve the error message Cannot set headers after they are sent to the client (it just means you have already sent a response)
I think just removing res.end(); would fix your issue.
Okay, it's really strange and wierd. I have middleware function app.use and it looked something like that:
app.use(async function (req : any, res : any, next : any) {
console.log(smthng);
get('header') stuff;
if (cond) {
// this is not executed because condition was false in all my situation
} else {
// this is executed in all cases of this thread
req.name = undefined;
next();
}
next()
})
As you see, in the end of middleware function was next(). So, I removed JUST THIS NEXT AND:
Terminal:
res.finished : false
Postgres: Rows were read...
[]
res.finished : false
res.finished : false
Postgres: Rows were read...
[]
f:validationUsers if(email) emailval[0] : undefined
res.finished : false
password BAM!!!
password BOOM!!!
I do not know what happened, I think this is deepsea shizophrenic hyperfluid flows under complier with some crossroad between db promise, middleware and res.send()
I'm using libgit2 via git-rs. I'm trying to commit things which is working, however, old files are showing up as deleted and/or staged even after the commit. How can I clean this up?
let repo: Repository = ...
let head_commit = match repo.head() {
Ok(head) => head.peel_to_commit().ok(),
Err(_) => None,
};
let head_tree = match head_commit {
Some(ref commit) => commit.tree().ok(),
None => None,
};
let mut diff_options = DiffOptions::new();
diff_options
.include_untracked(true)
.recurse_untracked_dirs(true);
let diff_result = self
.repo
.diff_tree_to_workdir_with_index(head_tree.as_ref(), Some(&mut diff_options));
let diff_deltas: Vec<_> = match diff_result {
Ok(ref diff) => diff.deltas().collect(),
Err(_) => Vec::new(),
};
if diff_deltas.is_empty() {
info!("no files changed");
return Ok(());
}
let mut index = .repo.index()?;
for diff_delta in diff_deltas {
let delta = diff_delta.status();
match delta {
Delta::Added
| Delta::Copied
| Delta::Modified
| Delta::Renamed
| Delta::Untracked
| Delta::Unmodified => {
let path = diff_delta.new_file().path().unwrap();
debug!("Staging {:?} file: {:?}", delta, path);
index.add_path(path)?;
}
Delta::Deleted => {
let path = diff_delta.old_file().path().unwrap();
debug!("Unstaging {:?} file: {:?}", delta, path);
index.remove_path(path)?;
}
_ => debug!("skipping {:?} file", delta),
}
}
let index_oid = index.write_tree()?;
let index_tree = self.repo.find_tree(index_oid)?;
let sig = Signature::new(&self.committer.name, &self.committer.email, &time)?;
let parents: Vec<_> = [&head_commit].iter().flat_map(|c| c.as_ref()).collect();
repo.commit(Some("HEAD"), &sig, &sig, message, &index_tree, &parents)?;
index.clear().unwrap();
As #user2722968 pointed out, you must call both index.write() and index.write_tree().
index.write_tree() will create a tree (or trees) from the index, returning the root tree object. This can be used to create a commit.
The trouble here is that you have updated HEAD with the commit that you've created. When you run git status, then that will compare the working directory to the index to the HEAD commit.
In this case, you have made changes to the index, you've updated the index in memory and used that to update HEAD. But you haven't actually written the index itself to disk, so you will see that the working directory does not match the index for any modified files, nor does the index match HEAD. (The working directory and HEAD contents will match, though that is never actually tested by git.)
Once you call index.write() then the working directory, the index and the HEAD commit will all be identical, and thus you'll see the expected (empty) status.
I have multiple queries that I created using QueryBuilder, I wanted to execute all of these queries inside a single transaction.
I read the documentation but it lacks information on how to use a QueryBuilder with transactions. I tried to create my queries using the same QueryRunner manager but I really feel like this does nothing.
I also tried to wrap my method using a #Transaction decoration but I get a socket hang up error.
This is my current attempt, I only added 2 queries in the example but I have 5 in total.
let user, membersInstituition
const connection = getConnection()
const queryRunner = connection.createQueryRunner()
console.log(queryRunner.isTransactionActive) // false
await queryRunner.startTransaction()
try {
console.log(queryRunner.isTransactionActive) // true
user = await queryRunner.manager
.createQueryBuilder(Usuarios, 'user')
.leftJoin(UltimaAtualizacaoMobile, 'att', 'user.id_usuario = att.id_usuario')
.select(['user.id_usuario as idUser, user.nome as name, att.ultima_atualizacao as ultimaAtualizacaoMobile'])
.where('user.id_usuario = :idUsuario', { idUsuario: idUsuario })
.execute()
membersInstituition = await queryRunner.manager
.createQueryBuilder(Instituicoes, 'insti')
.leftJoin(Amostras, 'am', 'am.id_instituicao = insti.id_instituicao')
.leftJoin(MembrosInstituicao, 'mi', 'mi.id_instituicao = insti.id_instituicao')
.leftJoin(Permissoes, 'perm', 'perm.id_permissao = mi.id_permissao')
.leftJoin(MembrosAmostra, 'mam', 'mam.id_amostra = am.id_instituicao AND mam.id_membro_instituicao = mi.id_membro_instituicao')
.where('am.id_amostra IN (:...idAmostras)', { idAmostras: idAmostras })
.andWhere('mi.id_usuario = :idUsuario', { idUsuario: idUsuario })
.select('mi.id_membro_instituicao as idMember, mam.id_membro_amostra as idMemberAmostra, insti.id_instituicao as idInstituition, perm.permissao as userPermission')
.orderBy('insti.id_instituicao', 'ASC')
.execute()
await queryRunner.commitTransaction()
console.log(queryRunner.isTransactionActive) // false
} catch (error) {
// Handle error
} finally {
await queryRunner.release()
}
Above code is indeed working, confirmed using TypeORM's logging, it shows the transaction starting and all the following queries before committing.
More info on how to enable Logging and to verify if your queries are under the same transaction: https://github.com/typeorm/typeorm/blob/master/docs/logging.md
To start off, here's my main bot.js code (Sorry if it seems like code spaghetti. I'm not the best with javascript):
const fs = require('fs');
const Discord = require('discord.js');
const client = new Discord.Client();
const auth = require('./auth.json');
const pack = require('./package.json');
const SQLite = require('better-sqlite3');
const sql = new SQLite('./scores.sqlite');
client.commands = new Discord.Collection();
const commandFiles = fs.readdirSync('./commands').filter(file => file.endsWith('.js'));
for (const file of commandFiles) {
const command = require(`./commands/${file}`);
client.commands.set(command.name, command);
}
client.on('ready', () => {
console.log(` Bot: ${client.user.tag}
Version: ${pack.version}
Logged in and clear for takeoff.`);
});
client.on("ready", () => {
const battle_leaderboard = sql.prepare("SELECT count(*) FROM sqlite_master WHERE type='table' AND name = 'scores';").get();
if (!battle_leaderboard['count(*)']) {
sql.prepare("CREATE TABLE scores (id TEXT PRIMARY KEY, user TEXT, guild TEXT, points INTEGER;").run();
sql.prepare("CREATE UNIQUE INDEX idx_scores_id ON scores (id);").run();
sql.pragma("synchronous = 1");
sql.pragma("journal_mode = wal");
}
client.getScore = sql.prepare("SELECT * FROM scores WHERE user = ? AND guild = ?");
client.setScore = sql.prepare("INSERT OR REPLACE INTO scores (id, user, guild, points) VALUES (#id, #user, #guild, #points);");
client.removeScore = sql.prepare("DELETE FROM scores WHERE user=?");
});
client.on('message', message => {
if (!message.content.startsWith(auth.prefix) || message.author.bot) return;
const args = message.content.slice(auth.prefix.length).split(/ +/);
const command = args.shift().toLowerCase();
switch (command) {
case 'ping':
client.commands.get('ping').execute(message, args);
break;
case 'mimic':
client.commands.get('mimic').execute(message, args);
break;
case 'add_role':
client.commands.get('add_role').execute(message, args);
break;
case 'remove_role':
client.commands.get('remove_role').execute(message, args);
break;
case 'bl_add_point':
if (message.member.roles.some(r=>["Fight Officiator"])) {
let score;
let member = message.mentions.members.first() || client.members.get(args[0]);
score = client.getScore.get(member.id, member.guild.id);
if (!score) {
score = { id: `${member.guild.id}-${member.id}`, user: member.username, guild: member.guild.id, points: 0}
}
score.points ++;
client.setScore.run(score);
message.channel.send(`Looks like ${member} got a win! Good one.`);
} else {
message.channel.send(`Sorry, this is only usable by GFOs.`);
};
break;
case '!bl_delete':
if (message.member.roles.some(r=>["Fight Officiator"])) {
let member = message.mentions.users.first();
client.removeScore.run(`${member.guild.id}-${member.id}`)
message.channel.send(`${member} has been set to zero on the leaderboard!`);
} else {
message.channel.send(`Sorry, this is only usable by GFOs.`);
};
break;
case 'bl':
const top5 = sql.prepare("SELECT * FROM scores WHERE guild = ? ORDER BY points DESC LIMIT 5;").all(message.guild.id);
const battleleaderboard = new Discord.RichEmbed()
.setColor('#FFD700')
.setTitle('The Battle Leaderboard - Current Rankings')
.setAuthor(`RankBot`)
.setDescription('The top fighters are displayed here.')
.setTimestamp()
.setFooter(`DM a Gang Fight Officiator to set up a spar with another member and possibly get your name registered onto RankBot's leaderboard!`);
for (const data of top5) {
battleleaderboard.addField(client.users.get(data.user), `${data.points} Win(s)`);
}
message.channel.send(battleleaderboard);
break;
};
});
client.login(auth.token);
I'm creating a bot that should be pretty simple:
The bot is a RankBot (at least that's what i dubbed it as), which is supposed to track the amount of wins in battles (since this is for a discord about a fighting game) using a special role that manually increments those wins with a command, then being able to pull up a leaderboard embed showing the people with the most wins. I'm testing it in a discord separate from the one it's for with a friend, and it seemingly worked until we commanded the bot to open the leaderboard. Nothing was shown. We thought it was strange, since it worked before when only one user was in the database, and I went to the PowerShell Command Line to see if there was an error message, and this showed up.
C:\Users\USER\Documents\OF_RankBot\bot.js:89
battleleaderboard.addField(client.users.get(data.user).tag || client.members.get(data.user).tag, `${data.points} Win(s)`);
^
TypeError: Cannot read property 'tag' of undefined
We were confused by what happened, so I removed the tag part of the command and used it again. The embed showed up this time, but it looked like this.
I assumed that what went wrong was that since they both had the same number of wins listed, the part of the bl command that sorts the users was confused. I then went and used bl_add_point on me and tried again. The near exact same embed was shown.
My guess as to what went wrong was that I messed up at some point when setting up the bl_add_point command, but I'm not sure how to fix the issue.
This isn't part of the main problem, but I can't seem to get bl_delete working either. No errors are shown when I use it. In fact, seemingly nothing happens. I'm not sure if these issues are connected, but I really only need the main issue fixed.
Thanks in advance.
edit: I found out partially what happened with the bl_delete issue, which was I simply put the prefix into the case when the prefix was already defined. Now when I put in the command, the PowerShell responds with:
C:\Users\USER\Documents\OF_RankBot\bot.js:72
client.removeScore.run(`${member.guild.id}-${member.id}`)
^
TypeError: Cannot read property 'id' of undefined
I'm printing SQL queries executed in my application using below code
var chalk = require('chalk');
module.exports = (app) => {
var connector = app.datasources.mysqlDs.connector;
connector.observe('after execute', function(ctx, next) {
console.log(chalk.green(ctx.req.sql));
next();
});
}
The above code prints sql query in console like this,
SELECT `users`.* FROM `users` WHERE `users`.`id` = 1 LIMIT 1
I'm interested to print time take to execute the sql query.
Ruby on rails applications prints sql query along with timing, similar to this given below
User Load (0.2ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 1 LIMIT 1
Is there any way to achieve this in loopback 3?
I am afraid LoopBack does not provide timing information out of the box. You can use before execute and after execute hooks to gather timing data yourself.
module.exports = (app) => {
var connector = app.datasources.mysqlDs.connector;
connector.observe('before execute', function(ctx, next) {
// store the start time in the context
ctx.__started = process.hrtime();
next();
});
connector.observe('after execute', function(ctx, next) {
// compute the time difference as [seconds, nanoseconds]
const delta = process.hrtime(ctx.__started);
// convert the two-part value into number of milliseconds elapsed
// and round it to a single decimal place
const durationInMs = 10 * Math.round((delta[0]*1000 + delta[1]/1e6)/10);
console.log('(%s ms) %s', durationInMs, chalk.green(ctx.req.sql));
next();
});
}