I know, it sounds really wierd, but it's true.
I was sitting and making an API on node.js, but when I began to test it, I was surprised to find out that nearly in beginning of query treatment when the first res.status().send() reached, VS Code drop a "Cannot set headers after they are sent to the client" error.
How I realize that database query is guilty? I used "res.finished" property to check out when it "sent to client" and discovered that res.finished changes from "false" to "true" exactly after db query.
Who knows what it can be? I had done the same API thing with MySQL database and it went nice, but now I'm using PostgreSQL, so things start to happen.
I export "PostgreSQL manager" class from typescript file
PostgreSQL_Manager.ts:
module.exports = {
PostgreSQL_db_manager
}
Import and initialize it in index.ts:
index.ts
const PostgreSQL_mngr = require('./PostgreSQL_Manager.ts').PostgreSQL_db_manager;
const db = new PostgreSQL_mngr;
And then, if I comment the statement with query to database, res.finished stay false (I tried it with readRows (SELECT) and with createRows(INSERT INTO)):
Piece of index.ts code:
console.log('res.finished : ', res.finished);
//let nickval : any = await db.readRows('users', 'nickname', `nickname = \'${nickname}\'`);
//await db.createRows('test', '(color, odor, taste, quantity)', '(\'meaningless\', \'absent\', \'sadness\', 0)');
console.log('res.finished : ', res.finished);
Piece of Terminal:
res.finished : false
res.finished : false
But when I uncomment database query, it becomes this:
Piece of index.ts code:
console.log('res.finished : ', res.finished);
//let nickval : any = await db.readRows('users', 'nickname', `nickname = \'${nickname}\'`);
await db.createRows('test', '(color, odor, taste, quantity)', '(\'meaningless\', \'absent\', \'sadness\', 0)');
console.log('res.finished : ', res.finished);
Piece of Terminal:
res.finished : false
Postgres: Rows were created...
res.finished : true
Code of db.createRows in postgres manager class looks like this:
public async createRows(table : string, columns: string | string[], values: string | string[]) : Promise<void> {
let createPromise = new Promise<void> ((resolve, reject) => {
this.db.query(`INSERT INTO ${table} ${columns} VALUES ${values};`, (err) => {
if(err) throw err;
console.log('Postgres: Rows were created...');
resolve();
});
});
await createPromise;
}
Edit 1:
There is error occurs (This function called from app.post, nickname, email and password has string values):
async function validationUsers (res : any, email : string = undefined, password : string = undefined, nickname : string = undefined) : Promise<boolean> {
console.log('f:validationUsers email : ', email);
console.log('f:validationUsers password : ', password);
console.log('f:validationUsers : nickname', nickname);
//validation: nickname and email
if(nickname) {
console.log('res.finished : ', res.finished);
//let nickval : any = await db.readRows('users', 'nickname', `nickname = \'${nickname}\'`);
await db.createRows('test', '(color, odor, taste, quantity)', '(\'meaningless\', \'absent\', \'sadness\', 0)');
console.log('res.finished : ', res.finished);
/* if(nickval[0] !== undefined) {
console.log('frofkofro');
res.status(410).send('Nickname already exists');
res.end();
return false;
} */
}
//validation: email
if(email) {
let emailval : any = await db.readRows('users', 'email', `email = \'${email}\'`);
console.log('f:validationUsers if(email) emailval[0] : ', emailval[0]);
if(emailval[0] !== undefined) {
console.log("?00");
res.send('Email already exists');
res.end();
return false;
}
}
//validation: password
if(password) {
let passwordval = /^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{8,100}$/;
if(!password.match(passwordval)) {
console.log('password BAM!!!');
res.status(412).send('Password does not match the criteria'); <-- **THIS STRING**
console.log('password BOOM!!!');
//res.end();
return false;
}
}
console.log('End of f:validationUsers');
return true;
}
Edit 2:
Can it be some problem with pool.query or pool connection to database from "pg" library for PostgreSQL? Or maybe problem with ts-node compiler?
So, I really don't understand what's going on.
I don't know if it's important, but I use ts-node for compile and render typescript
Edit 3:
OKAY, so I started in new ts file new server with the same 5000 port and run THIS:
app1.get('/db', async (req : any, res : any) => {
console.log('res.finished : ', res.finished);
await db1.createRows('test', '(color, odor, taste, quantity)', '(\'meaningless\', \'absent\', \'sadness\', 0)');
console.log('res.finished : ', res.finished);
res.status(200).send('All is fine this is send');
res.end();
});
And result in console:
Connected to database as pool successfully...
Server started on port 5000
res.finished : false
Postgres: Rows were created...
res.finished : false
And POSTMAN received res.send(). wtf??????
The error 'Headers already sent...' happens when your PostgreSQL code sends multiple results using res.send().
I assume the res.send() part is within PostgreSQL manager, which looks like a tested and true library - so what is happening to make it send two answers to one query?
I have no experience with Typescript and pSQL, but I have worked with pSQL and remember hitting this same snag years ago.
PostgreSQL supported (and I imagine it still supports) multiple query mode, such as, UPDATE b SET a=2 WHERE c; SELECT a FROM b. Those are two statements, and the reason why some exploits can even work.
And, just like it happened to me once, even if the second one has zero length and apparently is not even a query, in your code
`INSERT INTO ${table} ${columns} VALUES ${values};`
your PostgreSQL Manager just might think that there are two statements.
So, try removing that apparently harmless ';' at the end and see whether it solves the problem. I wasn't using your libraries, but for me, that did it.
I would say that the error is comming from here :
res.send('Email already exists');
res.end();
Indeed if you node's doc reads :
The res.end() function is used to end the response process. This method actually comes from the Node core, specifically the response.end() method of HTTP.ServerResponse. Use to quickly end the response without any data.
given that you already responsed 'Email already exists' express.js, you recieve the error message Cannot set headers after they are sent to the client (it just means you have already sent a response)
I think just removing res.end(); would fix your issue.
Okay, it's really strange and wierd. I have middleware function app.use and it looked something like that:
app.use(async function (req : any, res : any, next : any) {
console.log(smthng);
get('header') stuff;
if (cond) {
// this is not executed because condition was false in all my situation
} else {
// this is executed in all cases of this thread
req.name = undefined;
next();
}
next()
})
As you see, in the end of middleware function was next(). So, I removed JUST THIS NEXT AND:
Terminal:
res.finished : false
Postgres: Rows were read...
[]
res.finished : false
res.finished : false
Postgres: Rows were read...
[]
f:validationUsers if(email) emailval[0] : undefined
res.finished : false
password BAM!!!
password BOOM!!!
I do not know what happened, I think this is deepsea shizophrenic hyperfluid flows under complier with some crossroad between db promise, middleware and res.send()
I have the following function which gives me an array called URLs
const storageRef = this.$fire.storage.ref().child(fileName)
try {
const snapshot = storageRef.put(element).then((snapshot) => {
snapshot.ref.getDownloadURL().then((url) => {
urls.push(url)
})
})
console.log('File uploaded.')
} catch (e) {
console.log(e.message)
}
});
console.log(urls)
console.log("about to run enter time with imageurls length " + urls.length)
When I run console.log(URLs) initially I do see the array like the following
[]
0: "testvalue"
length: 1
__proto__: Array(0)
However, there is a small information icon stating
This value was evaluated upon first expanding. The value may have changed since.
Because of this, when I try to get the length of URLs, I get zero, meaning the value is being updated.
Does anyone know what's happening? I am using Vue.JS/Nuxt.
I have an array of users as below
let usersarr = ["'SAC_XSA_HDB_USER_ABC','SAC_XSA_HDB_USER_DEF'"]
I want to fetch data about the above users(if exists) from Hana database. I am using sap-hdbext-promisfied library in node.js.
My database connection is working fine. So, I am trying to execute a select query as below
async function readUsers(xsaDbConn){
try{
let usersarr = ["'SAC_XSA_HDB_USER_ABC','SAC_XSA_HDB_USER_DEF'"]
const checkuserexiststatement = await xsaDbConn.preparePromisified("SELECT USER_NAME FROM USERS WHERE USER_NAME IN (?)")
let checkuserexistresult = await xsaDbConn.statementExecPromisified(checkuserexiststatement, [usersarr])
console.log(checkuserexistresult)
return checkuserexistresult
}catch(err){
console.log(err)
return;
}
}
Below is the output I get
PS C:\Users\Documents\XSA\SAC_POC\cap_njs> npm start
> cap_njs#1.0.0 start C:\Users\Documents\XSA\SAC_POC\cap_njs
> node server.js
myapp is using Node.js version: v12.18.3
myapp listening on port 3000
[]
I get an empty array object as output. This is not the expected output, instead it should provide details about the users as they exist in the database.
The above code works when I provide single user value instead of multiple users in an array as shown below
async function readUsers(xsaDbConn, tempxsahdbusers){
try{
let usersarr = 'SAC_XSA_HDB_USER_ABC'
const checkuserexiststatement = await xsaDbConn.preparePromisified("SELECT USER_NAME FROM USERS WHERE USER_NAME IN (?)")
let checkuserexistresult = await xsaDbConn.statementExecPromisified(checkuserexiststatement, [usersarr])
console.log(checkuserexistresult)
return checkuserexistresult
}catch(err){
console.log(err)
return;
}
}
Output Of Above Code -
PS C:\Users\Documents\XSA\SAC_POC\cap_njs> npm start
> cap_njs#1.0.0 start C:\Users\Documents\XSA\SAC_POC\cap_njs
> node server.js
myapp is using Node.js version: v12.18.3
myapp listening on port 3000
[ 'SAC_XSA_HDB_USER_ABC' ]
So, why is it giving an empty array object when I provide an array as a parameter instead of a variable? Is it possible to provide an array as a parameter to the function statementExecPromisified(statement, []) of sap-hdbext-promisfied library in node.js ?
Your
let usersarr = ["'SAC_XSA_HDB_USER_ABC','SAC_XSA_HDB_USER_DEF'"]
has exactly one value, the String:
"'SAC_XSA_HDB_USER_ABC','SAC_XSA_HDB_USER_DEF'"
When passing the userarr in the statementExecPromisified function as a parameter you are actually passing a nested array in an array. You could either try
xsaDbConn.statementExecPromisified(checkuserexiststatement, [usersarr[0]])
or separate the values in the userarr and add multiple ? in the prepared statement and reference each single value with userarr[x].
I am building a Community Connector between Google Data Studio and SpyFu.com, in order to funnel SEO information for a specific url into the GDS Dashboard.
However, My getData() request only contains the first two fields from my Schema. As you can see, I have four listed in the code. The result is only the first two fields in the schema are printed to GDS.
I've been through tutorials, official documentation, YouTube videos, looked this issue up on google and checked out the community resources on GitHub.
//Step Two: Define getConfig()
function getConfig(request) {
var cc = DataStudioApp.createCommunityConnector();
var config = cc.getConfig();
config.newInfo()
.setId('instructions')
.setText('Give me SpyFu information on the following domain:');
config.newTextInput()
.setId('domain')
.setName('Enter the domain to search')
.setHelpText('e.g. ebay.com')
.setPlaceholder('ebay.com');
config.newTextInput()
.setId('SECRET_KEY')
.setName('Enter your API Secret Key')
.setHelpText('e.g. A1B2C3D4')
.setPlaceholder('A1B2C3D4');
config.setDateRangeRequired(false);
return config.build();
}
//Step Three: Define getSchema()
function getFields(request) {
var cc = DataStudioApp.createCommunityConnector();
var fields = cc.getFields();
var types = cc.FieldType;
var aggregations = cc.AggregationType;
fields.newDimension()
.setId('Keyword')
.setName('Keywords')
.setDescription('The keywords most often attributed to this domain.')
.setType(types.TEXT);
fields.newMetric()
.setId('Rank')
.setName('Rankings')
.setDescription('The ranking of the target site keyword on the Google Search Page.')
.setType(types.NUMBER);
fields.newMetric()
.setId('Local_Monthly_Searches')
.setName('Local Searches per Month')
.setDescription('Number of times, locally, that people have searched for this term within in the last month.')
.setType(types.NUMBER);
fields.newMetric()
.setId('Global_Monthly_Searches')
.setName('Global Searches per Month')
.setDescription('Number of times, globally, that people have searched for this term within in the last month.')
.setType(types.NUMBER);
return fields;
}
function getSchema(request) {
var fields = getFields(request).build();
return { schema: fields };
}
//Step Four: Define getData()
function responseToRows(requestedFields, response, domain) {
// Transform parsed data and filter for requested fields
return response.map(function(Array) {
var row = [];
requestedFields.asArray().forEach(function (field) {
switch (field.getId()) {
case 'Keyword':
return row.push(Array.term);
case 'Rank':
return row.push(Array.position);
case 'Local_Monthly_Searches':
return row.push(Array.exact_local_monthly_search_volume);
case 'Global_Monthly_Searches':
return row.push(Array.exact_global_monthly_search_volume);
case 'domain':
return row.push(domain);
default:
return row.push('');
}
});
return { values: row };
});
}
function getData(request) {
console.log("Request from Data Studio");
console.log(request);
var requestedFieldIds = request.fields.map(function(field) {
return field.name;
});
var requestedFields = getFields().forIds(requestedFieldIds);
// Fetch data from API
var url = [
'https://www.spyfu.com/apis/url_api/organic_kws?q='
+ request.configParams.domain
+ '&r=20'
+ '&p=[1 TO 10]'
+ '&api_key='
+ request.configParams.SECRET_KEY,
];
try {
var response = UrlFetchApp.fetch(url.join(''));
} catch (e) {
DataStudioApp.createCommunityConnector()
.newUserError()
.setDebugText('Failed URL Fetch Attempt. Exception details: ' + e)
.setText('There was an error accessing this domain. Try again later, or file an issue if this error persists.')
.throwException();
}
console.log("Response from API");
console.log(response);
//Parse data from the API
try {
var parsedResponse = JSON.parse(response);
} catch (e) {
DataStudioApp.createCommunityConnector()
.newUserError()
.setDebugText('Error parsing the JSON data. Exception details: ' + e)
.setText('There was an error parsing the JSON data. Try again later, or file an issue if this error persists.')
.throwException();
}
var rows = responseToRows(requestedFields, parsedResponse);
return {
schema: requestedFields.build(),
rows: rows
};
}
I need the GDS to post four columns of data. They are, "Keyword", "Rank", "Local Monthly Searches" and "Global Monthly searches".
I cannot figure out how to create a "fixed schema" so that the system always prints these four columns of data at every request. The tutorials and various documentation say it's possible, but not how to do it. Please help!
The number of metrics initially called up by the Google Community Connector is handled from the front-end, via Google Data Studio.
The back-end system (the Connector) only initially posts the default dimension and default metric. Getting the rest of the schemas to post should be handled when you are building a report on Google Data Studio. Simply click on the data set, select "data" on the right-hand menu, scroll down to either Metrics or Dimensions, and pick the ones you wish to add to the current set.
Note that these are the fields you established earlier in the coding process, when you were setting up your schemas.
Here, you're filtering your defined schema for fields that are present on the request object received by getData().
var requestedFieldIds = request.fields.map(function(field) {
return field.name;
});
var requestedFields = getFields().forIds(requestedFieldIds);
The visualization in Google Data Studio that is the catalyst for the request will determine which fields are requested.
I'm trying to connect to a remote Sql Anywhere 12.01 database with the following code:
let sqlanywhere = require('sqlanywhere');
let conn = sqlanywhere.createConnection();
let conn_params = {
Server : 'server:port',
UserId : 'user',
Password : 'pass'
};
conn.connect(conn_params, function() {
console.log("Connected!");
conn.exec('select * from cases', function (err, result) {
if (err) {
console.log(err);
} else {
console.log(result);
}
});
});
My console.log("Connected!") fires so I'm assuming that I've connected to the remote database? However, any query that I make results in this:
[Error: Code: -2001 Msg: Invalid Object]
I looked through the error codes online and didn't find this one. Anyone know why I may be experiencing it and how I can fix it?
The callback function should take two parameters: err and result. If err is undefined, then the connection succeeded, otherwise it indicates what error occurred. (In the case of connection, result will always be undefined.) Your code should look like this:
...
conn.connect( conn_params, function( err, result ) {
if( err ) {
console.log( "Connection failed: "+err );
} else {
console.log( "Connected!" );
}
...
});
I had to change "let" to "var" to make the javascript run. Also, your Server connection parameter should be the server name, not the location. You should change "Server" to "Host" to indicate the hostname / port.
Note: This answer was copied from the same question posted on sqlanywhere-forum.sap.com.