I have this API setup in Express but I cannot figure out how to correctly prepare the language filter with the IN statement using SQLite3 in node.js.
The first query applies the language filter correctly. I get the correct results but this could pose a sql injection risk.
The second query finds no results because of the language filter.
How do I correctly setup the db.prepare statement to accept both words, and language?
app.post('/api/languages/:language/getTextWords', (req, res) => {
let words = req.body.map(word => word.toLowerCase())
let wordMap = words.map(() => "?").join(',')
let language = req.params.language.toLowerCase();
// the language filter is applied correctly I get results but the statement is not fully prepared
// let query = db.prepare(`SELECT * FROM words WHERE word IN (${wordMap}) AND language = '${language}'`, words)
// no results are found because of the language filter
let query = db.prepare(`SELECT * FROM words WHERE word IN (${wordMap}) AND language = '?'`, words, language)
query.all((err, rows) => {
if (err) {
res.status(500).send(err.message)
throw err;
}
if (!rows) {
res.json([])
}
if (rows) {
res.json(rows)
}
})
})
Here is a screenshot of the DB table in question:
words table
pass values as an array:
let query = db.prepare(`SELECT * FROM words WHERE word IN (${wordMap}) AND language = ?`, [...words, language]);
Related
I apologize for the title, I don't exactly know how to word it. But essentially, this is a graph-type query but I know RavenDB's graph functionality will be going away so this probably needs to be solved with Javascript.
Here is the scenario:
I have a bunch of documents of different types, call them A, B, C, D. Each of these particular types of documents have some common properties. The one that I'm interested in right now is "Owner". The owner field is an ID which points to one of two other document types; it can be a Group or a User.
The Group document has a 'Members' field which contains an ID which either points to a User or another Group. Something like this
It's worth noting that the documents in play have custom IDs that begin with their entity type. For example Users and Groups begin with user: and group: respectively. Example IDs look like this: user:john#castleblack.com or group:the-nights-watch. This comes into play later.
What I want to be able to do is the following type of query:
"Given that I have either a group id or a user id, return all documents of type a, b, or c where the group/user id is equal to or is a descendant of the document's owner."
In other words, I need to be able to return all documents that are owned by a particular user or group either explicitly or implicitly through a hierarchy.
I've considered solving this a couple different ways with no luck. Here are the two approaches I've tried:
Using a function within a query
With Dejan's help in an email thread, I was able to devise a function that would walk it's way down the ownership graph. What this attempted to do was build a flat array of IDs which represented explicit and implicit owners (i.e. root + descendants):
declare function hierarchy(doc, owners){
owners = owners || [];
while(doc != null) {
let ownerId = id(doc)
if(ownerId.startsWith('user:')) {
owners.push(ownerId);
} else if(ownerId.startsWith('group:')) {
owners.push(ownerId);
doc.Members.forEach(m => {
let owner = load(m, 'Users') || load(m, 'Groups');
owners = hierarchy(owner, owners);
});
}
}
return owners;
}
I had two issues with this. 1. I don't actually know how to use this in a query lol. I tried to use it as part of the where clause but apparently that's not allowed:
from #all_docs as d
where hierarchy(d) = 'group:my-group-d'
// error: method hierarchy not allowed
Or if I tried anything in the select statement, I got an error that I have exceeded the number of allowed statements.
As a custom index
I tried the same idea through a custom index. Essentially, I tried to create an index that would produce an array of IDs using roughly the same function above, so that I could just query where my id was in that array
map('#all_docs', function(doc) {
function hierarchy(n, graph) {
while(n != null) {
let ownerId = id(n);
if(ownerId.startsWith('user:')) {
graph.push(ownerId);
return graph;
} else if(ownerId.startsWith('group:')){
graph.push(ownerId);
n.Members.forEach(g => {
let owner = load(g, 'Groups') || load(g, 'Users');
hierarchy(owner, graph);
});
return graph;
}
}
}
function distinct(value, index, self){ return self.indexOf(value) === index; }
let ownerGraph = []
if(doc.Owner) {
let owner = load(doc.Owner, 'Groups') || load(doc.Owner, 'Users');
ownerGraph = hierarchy(owner, ownerGraph).filter(distinct);
}
return { Owners: ownerGraph };
})
// error: recursion is not allowed by the javascript host
The problem with this is that I'm getting an error that recursion is not allowed.
So I'm stumped now. Am I going about this wrong? I feel like this could be a subquery of sorts or a filter by function, but I'm not sure how to do that either. Am I going to have to do this in two separate queries (i.e. two round-trips), one to get the IDs and the other to get the docs?
Update 1
I've revised my attempt at the index to the following and I'm not getting the recursion error anymore, but assuming my queries are correct, it's not returning anything
// Entity/ByOwnerGraph
map('#all_docs', function(doc) {
function walkGraph(ownerId) {
let owners = []
let idsToProcess = [ownerId]
while(idsToProcess.length > 0) {
let current = idsToProcess.shift();
if(current.startsWith('user:')){
owners.push(current);
} else if(current.startsWith('group:')) {
owners.push(current);
let group = load(current, 'Groups')
if(!group) { continue; }
idsToProcess.concat(group.Members)
}
}
return owners;
}
let owners = [];
if(doc.Owner) {
owners.concat(walkGraph(doc.Owner))
}
return { Owners: owners };
})
// query (no results)
from index Entity/ByOwnerGraph as x
where x.Owners = "group:my-group-id"
// alternate query (no results)
from index Entity/ByOwnerGraph as x
where x.Owners ALL IN ("group:my-group-id")
I still can't use this approach in a query either as I get the same error that there are too many statements.
I was trying to query my (postgres) db with a customizable statement built front end.
My resolver gets the built query inside the input param, but when I use the queryRaw method I get this error:
`"\nInvalid `prisma.queryRaw()` invocation:\n\n\n Raw query failed. Code: `42601`. Message: `db error: ERROR: syntax error at or near \"$1\"`"`
Is there a way to build a custom query and pass it like the input variable WITHOUT USING queryRawUnsafe to prisma? (queryRawUnsafe works fine, but well.. it's unsafe XD) Thanks <3
Here is my code.
getCars: (_parent, { input }, { prisma }) => {
if(input){
console.log(input) // --> SELECT * FROM car WHERE car."plate" ILIKE '%123%' //type String
const differentInput = '%123%'
// const result = prisma.$queryRaw`SELECT * FROM car WHERE car."plate" ILIKE '%123%'` // works
// const result = prisma.$queryRaw`SELECT * FROM car WHERE car."plate" ILIKE ${differentInput}` // works
// const result = prisma.$queryRawUnsafe(input) // works
const result = prisma.$queryRaw`${input}` // Doesn`t work
return result
}
// ... Other code
}
prisma.$queryRaw only accepts templated strings, not just strings. You can use the Prisma.sql helper to generate those templated strings to get the expected results. That might look like:
const sql = Prisma.sql`SELECT * FROM car WHERE car."plate" ILIKE '%123%'`
const result = prisma.$queryRaw`${sql}`
The queryRaw documentation mentions Prisma.sql with other examples but doesn't show any examples of what you are trying to do.
I'm discovering Express by creating a simple CRUD without ORM.
Issue is, I'm not able to find any record through the Model.findBy() function
model User {
static async findBy(payload) {
try {
let attr = Object.keys(payload)[0]
let value = Object.values(payload)[0]
let user = await pool.query(
`SELECT * from users WHERE $1::text = $2::text LIMIT 1;`,
[attr, value]
);
return user.rows; // empty :-(
} catch (err) {
throw err
}
}
}
User.findBy({ email: 'foo#bar.baz' }).then(console.log);
User.findBy({ name: 'Foo' }).then(console.log);
I've no issue using psql if I surround $2::text by single quote ' like:
SELECT * FROM users WHERE email = 'foo#bar.baz' LIMIT 1;
Though that's not possible inside parametrized queries. I've tried stuff like '($2::text)' (and escaped variations), but that looks far from what the documentation recommends.
I must be missing something. Is the emptiness of user.rows related to the way I fetch attr & value ? Or maybe, is some kind of escape required when passing string parameters ?
"Answer":
As stated in the comment section, issue isn't related to string escape, but to dynamic column names.
Column names are not identifiers, and therefore cannot be dynamically set using a query parameter.
See: https://stackoverflow.com/a/50813577/11509906
I am trying to make a search bar which works with multiple words, but I am worried about SQL injection.
I am using node express with the npm mssql package.
Here's the code which gets the criteria, generates the SQL and runs it:
router
.get('/search/:criteria', function (req, res) {
var criteria = req.params.criteria;
var words = criteria.split(" ");
var x = ""
words.map(word => x += `name like '%${word}%' and `);
x = x.substring(0, x.length - 5); // Remove trailing 'and'
var query = `SELECT * FROM table WHERE ${x}`
new sql.ConnectionPool(db).connect().then(pool => {
return pool.request().query(query)
}).then(result => {
})
});
A search for something to search would result in this query:
SELECT * FROM table
WHERE
name like '%something%'
and name like '%to%'
and name like '%search%'
I tried some SQL injections myself, but none of them seem to work.
Note: I am aware that we should always use inputs for this. It works fine for one word, but I don't know how to use inputs for many words. Ex:
new sql.ConnectionPool(db).connect().then(pool => {
return pool.request()
.input('input', '%'+criteria+'%')
.query(query)
})
The answer is: It's not safe. Your code does exactly nothing to make it safe, either. Don't build SQL by concatenating/interpolating user-supplied data into the statement.
In addition, you don't do any escaping for LIKE itself, either, so that is just as unclean.
If you need dynamic SQL, build a prepared SQL statement with the expected number of placeholders and then bind user-supplied values to those placeholders.
router.get('/search/:criteria', (req, res) => {
const ps = new sql.PreparedStatement();
const sqlConditions = [];
const escapedValues = {};
// set up escaped values, safe SQL bits, PS parameters
req.params.criteria.split(" ").forEach((v, i) => {
const paramName = 'val' + i;
escapedValues[paramName] = v.replace(/[\\%_]/g, '\\$&');
sqlConditions.push(`name LIKE '%' + #${paramName} + '%' ESCAPE '\'`);
ps.input(paramName, sql.VarChar);
});
// build safe SQL string, prepare statement
const sql = 'SELECT * FROM table WHERE ' + sqlConditions.join(' AND ');
ps.prepare(sql);
// connect, execute, return
ps.execute(escapedValues).then(result => {
res(result)
});
});
(Disclaimer: code is untested, as I have no SQL Server available right now, but you get the idea.)
I am trying to build some custom search logic in Umbraco 7 (7.3.6) which will search for multiple terms supplied by a user, where those terms may include phrases enclosed in quotes.
I have the following code which takes the supplied term, uses a regex to split individual terms (whilst maintaining those enclosed in quotes), then uses a series of GroupedOr calls to search against multiple fields
var searcher = Examine.ExamineManager.Instance.SearchProviderCollection[this.searchConfig.SiteSearchProviderName];
var searchCriteria = searcher.CreateSearchCriteria(Examine.SearchCriteria.BooleanOperation.Or);
var splitTerms = Regex.Matches(term, #"[\""].+?[\""]|[^ ]+")
.Cast<Match>()
.Select(m => m.Value)
.ToArray();
var query = searchCriteria.GroupedOr(
new[] { BaseContent.FIELD_NodeName },
this.GetValues(splitTerms, 3, 0.8F))
.Or()
.GroupedOr(
new[] { this.searchConfig.ContentFieldName },
this.GetValues(splitTerms, 1, 0.8F));
This is the GetValues method:
private IExamineValue[] GetValues(string[] terms, float boost, float fuzziness)
{
return terms
.Select(t => t.Boost(boost).Value.Fuzzy(fuzziness))
.ToArray();
}
I have a document in my index which contains the term "The quick brown fox jumps over the lazy dog". If I pass the string "\"brown fox\"" through the above logic then examine my query I can see my query object contains the following Lucene query:
(nodeName:"brown fox"~0.8) (_content:"brown fox"~0.8)
However, when I use this to build a search query as follows, I get no results.
var searchQuery = searcher
.Search(query.Compile(), 100)
.OrderByDescending(x => x.Score)
.TakeWhile(x => x.Score > 0.05f);
But if I run the exact same Lucene query using Luke I get the result I was expecting.
Is anyone able to help me understand this? Extra marks if you can explain why my boost values aren't being added to the Lucene query!!