I have a few .sql files which contain the SQL commands to GET data from my app-db.
I am using Cypress to automate reading data from DB and passing that as an array for example an array of ids to an API which in turn would use the id to fetch me a response.
So far, I have been able to connect to the dB and execute a code when it is hardcoded into the automation script.
The next step for me is to create a few utility functions which would read the query from a .sql file based on the API that needs to be called.
What I have done is :
In /plugins/index.js
const mysql = require('mysql');
const connections = {
dbSource : {
host: <myHostIP>,
port: <myPort>,
user: <myUserName>,
password: <myPassword>,
database: <myDbName>
}
}
function queryDB(connectionInfo, query) {
const connection = mysql.createConnection(connectionInfo);
connection.connect();
return new Promise((resolve, reject) => {
connection.query(query, (error, results) => {
if (error) {
return reject(error);
}
connection.end();
return resolve(results);
})
})
}
module.exports = (on, config) => {
on('task', {
queryDatabase({ dbName, query }) {
const connectionInfo = connections[dbName];
if (!connectionInfo) {
throw new Error(`Do not have DB connection under name ${dbName}`);
}
return queryDB(connectionInfo, query);
},
})
In /integration/test-code.js
import getQuery from "../query-generator/get-query";
const dbName = 'myDbName';
describe('Connect and fetch data from myDb', () => {
it('Fetches all data from my_table table in myDb', () => {
var query = getQuery.queryToFetchDataFromMyDb();
cy.log(query);
cy.task('queryDatabase', { dbName, query }).then((res) => {
expect(res).to.have.lengthOf(1);
});
});
})
In /query-generator/get-query.js
class getQuery {
static queryToFetchDataFromMyDb()() {
cy.readFile('My Queries.sql').then((queryString) => {
return queryString;
});
}
}
export default getQuery;
However, I am getting an error when I run this code, which says:
task
queryDatabase, {dbname: amsQANew, query: undefined}
**CypressError**
cy.task('queryDatabase') failed with the following error:
> ER_EMPTY_QUERY: Query was empty
Am I reading the file wrong? How do I get around this?
I think the problem lies in getQuery(),
class getQuery {
static queryToFetchDataFromMyDb()() {
return cy.readFile('My Queries.sql') // .then() does nothing here
}
}
The readFile is asynchronous, so in the test
getQuery.queryToFetchDataFromMyDb().then(query => {
cy.log(query);
cy.task('queryDatabase', { dbName, query }).then((res) => {
...
});
})
The getQuery class clutters up the logic, you would figure it out more easily by just using the Cypress commands directly in the test
cy.readFile('My Queries.sql').then(query => {
cy.log(query);
cy.task('queryDatabase', { dbName, query }).then((res) => {
...
});
})
Related
Right now im trying to write a query deconstruction, it should look like this
describe("Deconstruct query params", () => {
it("should deconstruct the desired query param such as id=1,2,3,4 into id=1&id=2&id=3&id=4", async () => {
const req: RequestCustom = {
extra : {
user: {}
},
query: {
id: "1,2,3,4",
},
};
const res = {};
const next = jest.fn();
await deconstructQueryParams(["id"])(
(req as unknown) as express.Request,
res as express.Response,
next
);
expect(req.query).toEqual(
"id=1&id=2&id=3&id=4"
);
});
});
For this I try and use
export const deconstructQueryParams = (params: Array<string>) => async (
req: Request,
res: Response,
next: NextFunction
) => {
params.forEach((param) => {
if (req.query[param]) {
const paramArr = req.query[param].split(",");
delete req.query[param]
paramArr.forEach((value: string) => {
req.query.append(param, value); //append doesnt exist
});
}
});
next();
}
The problem with this, is that I cant use
req.query.id=1
req.query.id=2
.....
Because those queries will be replaced, when I need "id=1&id=2&id=3&id=4"
But apparently req.query.append doesnt exist, so I cant duplicate the query properties? How can I do so?
I'm trying to fetch data from a dog API and I want to add to my database only their temperaments. I tried using some loops and a split to isolate the data and then using
findOrCreate() to add only those who are not already in the DB, after that I use findAll()
to get that info from the DB to send it using expressJS.
The unexpected behavior comes when I go to the route that executes all this and the route
only gives about half of the temperaments (they are 124 and it displays arround 54), then when I refresh the page it shows all 124 of them. The DB gets populated with all the 124 in one go so the problem is with findAll()
This is the function that isolates the temperaments and append them to the DB:
module.exports = async () => {
const info = await getAllDogs();
info.forEach(async (element) => {
const { temperament } = element;
if (temperament) {
const eachOne = temperament.split(", ");
for (i in eachOne) {
await Temperament.findOrCreate({
where: { name: eachOne[i] },
});
}
}
});
};
And this is the code that gets executed when I hit my expressJS sv to get the info
exports.temperaments = async (req, res) => {
try {
await getTemperaments(); //this function is the above function
} catch (error) {
res.status(500).send("something gone wrong", error);
}
const temperamentsDB = await Temperament.findAll();
res.json(temperamentsDB);
};
So as you can see the last function executes the function that appends all the data to the DB and then sends it with findAll and res.json()
forEach is a synchronous method so it doesn't await a result of the async callback. You need to do for of in order to get wait for all results:
module.exports = async () => {
const info = await getAllDogs();
for (element of info) {
const { temperament } = element;
if (temperament) {
const eachOne = temperament.split(", ");
for (i in eachOne) {
await Temperament.findOrCreate({
where: { name: eachOne[i] },
});
}
}
}
};
I'm relatively new to SQLite and I've been trying to modify a project made by people over at glitch. The app is supposed to save text to a database, and my project link is this. Although the logs don't show any errors. I can't get any data to save no matter what I do. (I changed the database file saving location from .data/sqlite.db to /data.sqlite.db). I assume this is some small issue I'm just new about, but I can't find anything about it on the internet.
app.use(express.static("public"));
// init sqlite db
const dbFile = "./data/sqlite.db";
const exists = fs.existsSync(dbFile);
const sqlite3 = require("sqlite3").verbose();
const db = new sqlite3.Database(dbFile);
// if ./.data/sqlite.db does not exist, create it, otherwise print records to console
db.serialize(() => {
if (!exists) {
db.run(
"CREATE TABLE Dreams (id INTEGER PRIMARY KEY AUTOINCREMENT, dream TEXT)"
);
console.log("New table Dreams created!");
// insert default dreams
db.serialize(() => {
db.run(
'INSERT INTO Dreams (dream) VALUES ("test"), ("hi sam"), ("fortnite gaming chair 3d")'
);
});
} else {
console.log('Database "Dreams" ready to go!');
db.each("SELECT * from Dreams", (err, row) => {
if (row) {
console.log(`record: ${row.dream}`);
}
});
}
});
// http://expressjs.com/en/starter/basic-routing.html
app.get("/", (request, response) => {
response.sendFile(`${__dirname}/views/index.html`);
});
// endpoint to get all the dreams in the database
app.get("/getDreams", (request, response) => {
db.all("SELECT * from Dreams", (err, rows) => {
response.send(JSON.stringify(rows));
});
});
// endpoint to add a dream to the database
app.post("/addDream", (request, response) => {
console.log(`add to dreams ${request.body.dream}`);
// DISALLOW_WRITE is an ENV variable that gets reset for new projects
// so they can write to the database
if (!process.env.DISALLOW_WRITE) {
const cleansedDream = request.body.dream;
db.run(`INSERT INTO Dreams (dream) VALUES (?)`, cleansedDream, error => {
if (error) {
response.send({ message: "error!" });
} else {
response.send({ message: cleansedDream+"success" });
}
});
}
});
// endpoint to clear dreams from the database
app.get("/clearDreams", (request, response) => {
// DISALLOW_WRITE is an ENV variable that gets reset for new projects so you can write to the database
if (!process.env.DISALLOW_WRITE) {
db.each(
"SELECT * from Dreams",
(err, row) => {
console.log("row", row);
db.run(`DELETE FROM Dreams WHERE ID=?`, row.id, error => {
if (row) {
console.log(`deleted row ${row.id}`);
}
});
},
err => {
if (err) {
response.send({ message: "error!" });
} else {
response.send({ message: "success" });
}
}
);
}
});
// listen for requests :)
var listener = app.listen(process.env.PORT, () => {
console.log(`Your app is listening on port ${listener.address().port}`);
});
I am new to Node.js (3 days total experience). I am using Node.js and the tedious package to query a database (azure SQL). I use the example as explained here: https://learn.microsoft.com/en-us/azure/azure-sql/database/connect-query-nodejs?tabs=macos
const connection = new Connection(config);
// Attempt to connect and execute queries if connection goes through
connection.on("connect", err => {
if (err) {
console.error(err.message);
} else {
console.log("Reading rows from the Table...");
// Read all rows from table
const request = new Request(
"SELECT * FROM clients",
(err, rowCount, columns) => {
if (err) {
console.error(err.message);
} else {
console.log(`${rowCount} row(s) returned`);
}
}
);
request.on("row", columns => {
columns.forEach(column => {
console.log("%s\t%s", column.metadata.colName, column.value);
});
});
connection.execSql(request);
}
});
I have two issues:
I do not know how to get the queried data into an object and
If I run the script it does print the items to the console, but it doesn't close the connection after it has done so. If I add a connection.close() at the bottom, it will close the connection before its done. I get the feeling that node.js executes everything at the same time (I am used to Python..).
Update
I found a way to close the connection, to my understanding the request object has several "events" that are predefined by the library. It seems I need to add the event "done" through request.on('done', ...) in order to make sure that it can even BE done. My updated code looks like this:
var connection = new Connection(config);
connection.connect(function(err) {
// If no error, then good to go...
executeStatement();
}
);
connection.on('debug', function(text) {
//remove commenting below to get full debugging.
//console.log(text);
}
);
function executeStatement() {
request = new Request("SELECT * FROM clients", function(err, rowCount) {
if (err) {
console.log(err);
} else {
console.log(rowCount + ' rows');
}
connection.close();
});
request.on('row', function(rows) {
_.forEach(rows, function(value, collection){
console.log(value)
console.log(value.value);
console.log(value.metadata.colName)
console.log(collection)
})
});
request.on('done', function(rowCount, more) {
console.log(rowCount + ' rows returned');
});
// In SQL Server 2000 you may need: connection.execSqlBatch(request);
connection.execSql(request);
}
Anyways, your help would be much appreciated!
Regards
Pieter
The package tedious is synchronous package, it uses the callback to return results. So when we call connection.close(), it will disable connection and stop the callback function. If will want to close the connection, I suggest you use async package to implement it.
For example
const { Connection, Request } = require("tedious");
const async = require("async");
const config = {
authentication: {
options: {
userName: "username", // update me
password: "password", // update me
},
type: "default",
},
server: "your_server.database.windows.net", // update me
options: {
database: "your_database", //update me
encrypt: true,
validateBulkLoadParameters: true,
},
};
const connection = new Connection(config);
let results=[]
function queryDatabase(callback) {
console.log("Reading rows from the Table...");
// Read all rows from table
const request = new Request("SELECT * FROM Person", (err, rowCount) => {
if (err) {
callback(err);
} else {
console.log(`${rowCount} row(s) returned`);
callback(null);
}
});
request.on("row", (columns) => {
let result={}
columns.forEach((column) => {
result[column.metadata.colName]=column.value
console.log("%s\t%s", column.metadata.colName, column.value);
});
// save result into an array
results.push(result)
});
connection.execSql(request);
}
function Complete(err, result) {
if (err) {
callback(err);
} else {
connection.close();
console.log("close connection");
}
}
connection.on("connect", function (err) {
if (err) {
console.log(err);
} else {
console.log("Connected");
// Execute all functions in the array serially
async.waterfall([queryDatabase], Complete);
}
});
connection.connect();
Besides, you also can use the package mssql. It supports asynchronous methods and depends on package tedious. We can directly call close after querying.
For example
const mssql = require("mssql");
const config = {
user: "username",
password: "password",
server: "your_server.database.windows.net",
database: "your_database",
options: {
encrypt: true,
enableArithAbort: true,
},
};
let pool = new mssql.ConnectionPool(config);
async function query() {
try {
await pool.connect();
const request = pool.request();
const result = await request.query("SELECT * FROM Person");
console.dir(result.recordset);
await pool.close();
console.log(pool.connected);
} catch (error) {
throw error;
}
}
query().catch((err) => {
throw err;
});
You can custom a class first and declare an Array to save ojects such as:
let sales = new Array();
class SalesLT{
constructor(catagryName,productName){
this.catagryName = catagryName;
this.productName = productName;
}
Here my sql statement returns 2 properties, so every time the loop takes out two elements from the ColumnValue[].
request.on("row", columns => {
for(let i=0; i<columns.length; i=i+2){
let sale = new SalesLT(columns[i].value,columns[i+1].value);
sales.push(sale);
}
sales.forEach( item => {
console.log("%s\t%s",item.catagryName, item.productName)
})
});
The code is as follows:
const { Connection, Request } = require("tedious");
let sales = new Array();
class SalesLT{
constructor(catagryName,productName){
this.catagryName = catagryName;
this.productName = productName;
}
}
// Create connection to database
const config = {
authentication: {
options: {
userName: "<***>", // update me
password: "<***>" // update me
},
type: "default"
},
server: "<****>.database.windows.net", // update me
options: {
database: "<***>", //update me
encrypt: true
}
};
const connection = new Connection(config);
// Attempt to connect and execute queries if connection goes through
connection.on ("connect", err => {
if (err) {
console.error(err.message);
} else {
queryDatabase();
}
});
function queryDatabase() {
console.log("Reading rows from the Table...");
// Read all rows from table
const request = new Request(
`SELECT TOP 2 pc.Name as CategoryName,
p.name as ProductName
FROM [SalesLT].[ProductCategory] pc
JOIN [SalesLT].[Product] p ON pc.productcategoryid = p.productcategoryid`,
(err, rowCount) => {
if (err) {
console.error(err.message);
} else {
console.log(`${rowCount} row(s) returned`);
}
connection.close();
}
);
request.on("row", columns => {
for(let i=0; i<columns.length; i=i+2){
let sale = new SalesLT(columns[i].value,columns[i+1].value);
sales.push(sale);
}
sales.forEach( item => {
console.log("%s\t%s",item.catagryName, item.productName)
})
});
connection.execSql(request);
}
this article should help you, to solve all the issues you are facing...which were the same I had when I started using Node :)
https://devblogs.microsoft.com/azure-sql/promises-node-tedious-azure-sql-oh-my/
I'm testing a simple rule:
match /users/{userId} {
allow write, get: if isSignedIn() && userOwner(userId);
}
function isSignedIn(){
return request.auth != null
}
function userOwner(userId){
return userId == request.auth.uid
}
Here is my test:
test("read succeed only if requested user is authenticated user", async () => {
const db = await setup(
{
uid: "testid",
email: "test#test.com"
},
{
"users/testid": {},
"users/anotherid": {}
}
);
const userRef = db.collection("users");
expect(await assertSucceeds(userRef.doc("testid").get()));
expect(await assertFails(userRef.doc("anotherid").get()));
})
And the setup method:
export const setup = async (auth?: any, data?: any) => {
const projectId = `rules-spec-${Date.now()}`;
const app = firebase.initializeTestApp({
projectId,
auth
});
const db = app.firestore();
if (data) {
for (const key in data) {
const ref = db.doc(key);
await ref.set(data[key]);
}
}
await firebase.loadFirestoreRules({
projectId,
rules: fs.readFileSync("firestore.rules").toString()
});
return db;
};
It throws the following error :
FirebaseError: 7 PERMISSION_DENIED:
false for 'create' # L5, Null value error. for 'create' # L9
It seems that when it tries to set the mock data given in setup, it can't because of the write rule. but I don't understand, I load the rules after the database being set.
Any idea what's going on here?
You can try setting the rules to be open before you populate the database.
After the data is set you load the rules you are trying to test.
export const setup = async(auth ? : any, data ? : any) => {
const projectId = `rules-spec-${Date.now()}`;
const app = firebase.initializeTestApp({
projectId,
auth
});
const db = app.firestore();
await firebase.loadFirestoreRules({
projectId,
rules:
"service cloud.firestore {match/databases/{database}/documents" +
"{match /{document=**} {" +
"allow read, write: if true;" +
"}}}",
});
if (data) {
for (const key in data) {
const ref = db.doc(key);
await ref.set(data[key]);
}
}
await firebase.loadFirestoreRules({
projectId,
rules: fs.readFileSync("firestore.rules").toString()
});
return db;
};