websql use select in to get rows from an array - sql

in websql we can request a certain row like this:
tx.executeSql('SELECT * FROM tblSettings where id = ?', [id], function(tx, rs){
// do stuff with the resultset.
},
function errorHandler(tx, e){
// do something upon error.
console.warn('SQL Error: ', e);
});
however, I know regular SQL and figured i should be able to request
var arr = [1, 2, 3];
tx.executeSql('SELECT * FROM tblSettings where id in (?)', [arr], function(tx, rs){
// do stuff with the resultset.
},
function errorHandler(tx, e){
// do something upon error.
console.warn('SQL Error: ', e);
});
but that gives us no results, the result is always empty. if i would remove the [arr] into arr, then the sql would get a variable amount of parameters, so i figured it should be [arr]. otherwise it would require us to add a dynamic amount of question marks (as many as there are id's in the array).
so can anyone see what i'm doing wrong?

aparently, there is no other solution, than to manually add a question mark for every item in your array.
this is actually in the specs on w3.org
var q = "";
for each (var i in labels)
q += (q == "" ? "" : ", ") + "?";
// later to be used as such:
t.executeSql('SELECT id FROM docs WHERE label IN (' + q + ')', labels, function (t, d) {
// do stuff with result...
});
more info here: http://www.w3.org/TR/webdatabase/#introduction (at the end of the introduction)
however, at the moment i created a helper function that creates such a string for me
might be better than the above, might not, i haven't done any performance testing.
this is what i use now
var createParamString = function(arr){
return _(arr).map(function(){ return "?"; }).join(',');
}
// when called like this:
createparamString([1,2,3,4,5]); // >> returns ?,?,?,?,?
this however makes use of the underscore.js library we have in our project.

Good answer. It was interesting to read an explanation in the official documentation.
I see this question was answered in 2012. I tried it in Google 37 exactly as it is recommened and this is what I got.
Data on input: (I outlined them with the black pencil)
Chrome complains:
So it accepts as many question signs as many input parameters are given. (Let us pay attention that although array is passed it's treated as one parameter)
Eventually I came up to this solution:
var activeItemIds = [1,2,3];
var q = "";
for (var i=0; i< activeItemIds.length; i++) {
q += '"' + activeItemIds[i] + '", ';
}
q= q.substring(0, q.length - 2);
var query = 'SELECT "id" FROM "products" WHERE "id" IN (' + q + ')';
_db.transaction(function (tx) {
tx.executeSql(query, [], function (tx, results1) {
console.log(results1);
debugger;
}, function (a, b) {
console.warn(a);
console.warn(b);
})
})

Related

Is there a way to get back source code from antlr4ts parse tree after modifications ctx.removeLastChild/ctx.addChild? [duplicate]

I want to keep white space when I call text attribute of token, is there any way to do it?
Here is the situation:
We have the following code
IF L > 40 THEN;
ELSE
IF A = 20 THEN
PUT "HELLO";
In this case, I want to transform it into:
if (!(L>40){
if (A=20)
put "hello";
}
The rule in Antlr is that:
stmt_if_block: IF expr
THEN x=stmt
(ELSE y=stmt)?
{
if ($x.text.equalsIgnoreCase(";"))
{
WriteLn("if(!(" + $expr.text +")){");
WriteLn($stmt.text);
Writeln("}");
}
}
But the result looks like:
if(!(L>40))
{
ifA=20put"hello";
}
The reason is that the white space in $stmt was removed. I was wondering if there is anyway to keep these white space
Thank you so much
Update: If I add
SPACE: [ ] -> channel(HIDDEN);
The space will be preserved, and the result would look like below, many spaces between tokens:
IF SUBSTR(WNAME3,M-1,1) = ')' THEN M = L; ELSE M = L - 1;
This is the C# extension method I use for exactly this purpose:
public static string GetFullText(this ParserRuleContext context)
{
if (context.Start == null || context.Stop == null || context.Start.StartIndex < 0 || context.Stop.StopIndex < 0)
return context.GetText(); // Fallback
return context.Start.InputStream.GetText(Interval.Of(context.Start.StartIndex, context.Stop.StopIndex));
}
Since you're using java, you'll have to translate it, but it should be straightforward - the API is the same.
Explanation: Get the first token, get the last token, and get the text from the input stream between the first char of the first token and the last char of the last token.
#Lucas solution, but in java in case you have troubles in translating:
private String getFullText(ParserRuleContext context) {
if (context.start == null || context.stop == null || context.start.getStartIndex() < 0 || context.stop.getStopIndex() < 0)
return context.getText();
return context.start.getInputStream().getText(Interval.of(context.start.getStartIndex(), context.stop.getStopIndex()));
}
Looks like InputStream is not always updated after removeLastChild/addChild operations. This solution helped me for one grammar, but it doesn't work for another.
Works for this grammar.
Doesn't work for modern groovy grammar (for some reason inputStream.getText contains old text).
I am trying to implement function name replacement like this:
enterPostfixExpression(ctx: PostfixExpressionContext) {
// Get identifierContext from ctx
...
const token = CommonTokenFactory.DEFAULT.createSimple(GroovyParser.Identifier, 'someNewFnName');
const node = new TerminalNode(token);
identifierContext.removeLastChild();
identifierContext.addChild(node);
UPD: I used visitor pattern for the first implementation

How do I read data in one cell and write data into another cell using Google Sheets?

So let's all assume that column B is filled with multiple, short statements. These statements may be used more than once, not at all, or just once throughout the column. I want to be able to read what's in each cell of column B and assign a category to it in column F using the Google Sheets script editor. I'll include some pseudo-code of how I would do something like this normally.
for (var i = 0; i < statements.length; i++) {
if (statements[i] == 'Description One') {
category[i] = 'Category One';
}
else if (statements[i] == 'Description Two') {
category[i] = 'Category Two';
}
// and so on for all known categories....
}
How do I go about accessing a cell for a read and accessing a different cell for a write?
Thanks in advance for the help!
Ok, so after a little more thought on the subject, I've arrived at a solution. It's super simple, albeit tedious
function assignCategory(description) {
if (description == 'Description One') {
return 'Category One';
}
// and so on for all known categories
}
Hopefully someone will see this and be helped anyway, if you guys think of a more efficient and easier to maintain way of doing this, by all means do chime in.
Assuming a sheet such as this one, which has a header and six different columns (where B is the description, and F the category); you could use a dictionary to translate your values as follows:
// (description -> category) dictionary
var translations = {
"cooking": "Cooking",
"sports": "Sport",
"leisure": "Leisure",
"music": "Music",
"others": "Other"
}
function assignCategories() {
var dataRange = SpreadsheetApp.getActiveSheet().getDataRange();
for (var i=2; i<=dataRange.getNumRows(); i++) {
var description = dataRange.getCell(i, 2).getValue();
var category = translations[description];
dataRange.getCell(i, 6).setValue(category);
}
}
In case you need additional ruling (i.e. descriptions that contain cricket must be classified as sport), you could accomplish your desired results by implementing your own custom function and using string functions (such as indexOf) or regular expressions.
Using indexOf
// (description -> category) dictionary
var translations = {
"cooking": "Cooking",
"sports": "Sport",
"leisure": "Leisure",
"music": "Music",
"others": "Other"
}
function assignCategories() {
var dataRange = SpreadsheetApp.getActiveSheet().getDataRange();
for (var i=2; i<=dataRange.getNumRows(); i++) {
var description = dataRange.getCell(i, 2).getValue()
var category = assignCategory(description);
if (category) dataRange.getCell(i, 6).setValue(category);
}
}
function assignCategory(description) {
description = description.toLowerCase();
var keys = Object.keys(translations);
for (var i=0; i<categories.length; i++) {
var currentKey = keys[i];
if (description.indexOf(currentKey) > -1)
return translations[currentKey];
}
}
This version is a bit more sophisticated. It will make the 'description' of each row lowercase in order to better compare with your dictionary, and also uses indexOf for checking whether the 'translation key' appears in the description, rather than checking for an exact match.
You should be aware however that this method will be considerably slower, and that the script may timeout (see GAS Quotas). You could implement ways to 'resume' your script operations such that you can re-run it and continue where it left off, in case that this hinders your operations.

How safe is this generated query from SQL injection?

I am trying to make a search bar which works with multiple words, but I am worried about SQL injection.
I am using node express with the npm mssql package.
Here's the code which gets the criteria, generates the SQL and runs it:
router
.get('/search/:criteria', function (req, res) {
var criteria = req.params.criteria;
var words = criteria.split(" ");
var x = ""
words.map(word => x += `name like '%${word}%' and `);
x = x.substring(0, x.length - 5); // Remove trailing 'and'
var query = `SELECT * FROM table WHERE ${x}`
new sql.ConnectionPool(db).connect().then(pool => {
return pool.request().query(query)
}).then(result => {
})
});
A search for something to search would result in this query:
SELECT * FROM table
WHERE
name like '%something%'
and name like '%to%'
and name like '%search%'
I tried some SQL injections myself, but none of them seem to work.
Note: I am aware that we should always use inputs for this. It works fine for one word, but I don't know how to use inputs for many words. Ex:
new sql.ConnectionPool(db).connect().then(pool => {
return pool.request()
.input('input', '%'+criteria+'%')
.query(query)
})
The answer is: It's not safe. Your code does exactly nothing to make it safe, either. Don't build SQL by concatenating/interpolating user-supplied data into the statement.
In addition, you don't do any escaping for LIKE itself, either, so that is just as unclean.
If you need dynamic SQL, build a prepared SQL statement with the expected number of placeholders and then bind user-supplied values to those placeholders.
router.get('/search/:criteria', (req, res) => {
const ps = new sql.PreparedStatement();
const sqlConditions = [];
const escapedValues = {};
// set up escaped values, safe SQL bits, PS parameters
req.params.criteria.split(" ").forEach((v, i) => {
const paramName = 'val' + i;
escapedValues[paramName] = v.replace(/[\\%_]/g, '\\$&');
sqlConditions.push(`name LIKE '%' + #${paramName} + '%' ESCAPE '\'`);
ps.input(paramName, sql.VarChar);
});
// build safe SQL string, prepare statement
const sql = 'SELECT * FROM table WHERE ' + sqlConditions.join(' AND ');
ps.prepare(sql);
// connect, execute, return
ps.execute(escapedValues).then(result => {
res(result)
});
});
(Disclaimer: code is untested, as I have no SQL Server available right now, but you get the idea.)

BigQuery UDF memory exceeded error on multiple rows but works fine on single row

I'm writing a UDF to process Google Analytics data, and getting the "UDF out of memory" error message when I try to process multiple rows. I downloaded the raw data and found the largest record and tried running my UDF query on that, with success. Some of the rows have up to 500 nested hits, and the size of the hit record (by far the largest component of each row of the raw GA data) does seem to have an effect on how many rows I can process before getting the error.
For example, the query
select
user.ga_user_id,
ga_session_id,
...
from
temp_ga_processing(
select
fullVisitorId,
visitNumber,
...
from [79689075.ga_sessions_20160201] limit 100)
returns the error, but
from [79689075.ga_sessions_20160201] where totals.hits = 500 limit 1)
does not.
I was under the impression that any memory limitations were per-row? I've tried several techniques, such as setting row = null; before emit(return_dict); (where return_dict is the processed data) but to no avail.
The UDF itself doesn't do anything fancy; I'd paste it here but it's ~45 kB in length. It essentially does a bunch of things along the lines of:
function temp_ga_processing(row, emit) {
topic_id = -1;
hit_numbers = [];
first_page_load_hits = [];
return_dict = {};
return_dict["user"] = {};
return_dict["user"]["ga_user_id"] = row.fullVisitorId;
return_dict["ga_session_id"] = row.fullVisitorId.concat("-".concat(row.visitNumber));
for(i=0;i<row.hits.length;i++) {
hit_dict = {};
hit_dict["page"] = {};
hit_dict["time"] = row.hits[i].time;
hit_dict["type"] = row.hits[i].type;
hit_dict["page"]["engaged_10s"] = false;
hit_dict["page"]["engaged_30s"] = false;
hit_dict["page"]["engaged_60s"] = false;
add_hit = true;
for(j=0;j<row.hits[i].customMetrics.length;j++) {
if(row.hits[i].customDimensions[j] != null) {
if(row.hits[i].customMetrics[j]["index"] == 3) {
metrics = {"video_play_time": row.hits[i].customMetrics[j]["value"]};
hit_dict["metrics"] = metrics;
metrics = null;
row.hits[i].customDimensions[j] = null;
}
}
}
hit_dict["topic"] = {};
hit_dict["doctor"] = {};
hit_dict["doctor_location"] = {};
hit_dict["content"] = {};
if(row.hits[i].customDimensions != null) {
for(j=0;j<row.hits[i].customDimensions.length;j++) {
if(row.hits[i].customDimensions[j] != null) {
if(row.hits[i].customDimensions[j]["index"] == 1) {
hit_dict["topic"] = {"name": row.hits[i].customDimensions[j]["value"]};
row.hits[i].customDimensions[j] = null;
continue;
}
if(row.hits[i].customDimensions[j]["index"] == 3) {
if(row.hits[i].customDimensions[j]["value"].search("doctor") > -1) {
return_dict["logged_in_as_doctor"] = true;
}
}
// and so on...
}
}
}
if(row.hits[i]["eventInfo"]["eventCategory"] == "page load time" && row.hits[i]["eventInfo"]["eventLabel"].search("OUTLIER") == -1) {
elre = /(?:onLoad|pl|page):(\d+)/.exec(row.hits[i]["eventInfo"]["eventLabel"]);
if(elre != null) {
if(parseInt(elre[0].split(":")[1]) <= 60000) {
first_page_load_hits.push(parseFloat(row.hits[i].hitNumber));
if(hit_dict["page"]["page_load"] == null) {
hit_dict["page"]["page_load"] = {};
}
hit_dict["page"]["page_load"]["sample"] = 1;
page_load_time_re = /(?:onLoad|pl|page):(\d+)/.exec(row.hits[i]["eventInfo"]["eventLabel"]);
if(page_load_time_re != null) {
hit_dict["page"]["page_load"]["page_load_time"] = parseFloat(page_load_time_re[0].split(':')[1])/1000;
}
}
// and so on...
}
}
row = null;
emit return_dict;
}
The job ID is realself-main:bquijob_4c30bd3d_152fbfcd7fd
Update Aug 2016 : We have pushed out an update that will allow the JavaScript worker to use twice as much RAM. We will continue to monitor jobs that have failed with JS OOM to see if more increases are necessary; in the meantime, please let us know if you have further jobs failing with OOM. Thanks!
Update : this issue was related to limits we had on the size of the UDF code. It looks like V8's optimize+recompile pass of the UDF code generates a data segment that was bigger than our limits, but this was only happening when when the UDF runs over a "sufficient" number of rows. I'm meeting with the V8 team this week to dig into the details further.
#Grayson - I was able to run your job over the entire 20160201 table successfully; the query takes 1-2 minutes to execute. Could you please verify that this works on your side?
We've gotten a few reports of similar issues that seem related to # rows processed. I'm sorry for the trouble; I'll be doing some profiling on our JavaScript runtime to try to find if and where memory is being leaked. Stay tuned for the analysis.
In the meantime, if you're able to isolate any specific rows that cause the error, that would also be very helpful.
A UDF will fail on anything but very small datasets if it has a lot of if/then levels, such as:
if () {
.... if() {
.........if () {
etc
We had to track down and remove the deepest if/then statement.
But, that is not enough. In addition, when you pass the data into the UDF run a "GROUP EACH BY" on all the variables. This will force BQ to send the output to multiple "workers". Otherwise it will also fail.
I've wasted 3 days of my life on this annoying bug. Argh.
I love the concept of parsing my logs in BigQuery, but I've got the same problem, I get
Error: Resources exceeded during query execution.
The Job Id is bigquery-looker:bquijob_260be029_153dd96cfdb, if that at all helps.
I wrote a very basic parser does a simple match and returns rows. Works just fine on a 10K row data set, but I get out of resources when trying to run against a 3M row logfile.
Any suggestions for a work around?
Here is the javascript code.
function parseLogRow(row, emit) {
r = (row.logrow ? row.logrow : "") + (typeof row.l2 !== "undefined" ? " " + row.l2 : "") + (row.l3 ? " " + row.l3 : "")
ts = null
category = null
user = null
message = null
db = null
found = false
if (r) {
m = r.match(/^(\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d\.\d\d\d (\+|\-)\d\d\d\d) \[([^|]*)\|([^|]*)\|([^\]]*)\] :: (.*)/ )
if( m){
ts = new Date(m[1])/1000
category = m[3] || null
user = m[4] || null
db = m[5] || null
message = m[6] || null
found = true
}
else {
message = r
found = false
}
}
emit({
ts: ts,
category: category,
user: user,
db: db,
message: message,
found: found
});
}
bigquery.defineFunction(
'parseLogRow', // Name of the function exported to SQL
['logrow',"l2","l3"], // Names of input columns
[
{'name': 'ts', 'type': 'timestamp'}, // Output schema
{'name': 'category', 'type': 'string'},
{'name': 'user', 'type': 'string'},
{'name': 'db', 'type': 'string'},
{'name': 'message', 'type': 'string'},
{'name': 'found', 'type': 'boolean'},
],
parseLogRow // Reference to JavaScript UDF
);

Is there a way to seperate with a <br> each time the order property finds a change?

I'm working on a defect list custom app that uses a queryConfig to populate results of a defect. I'm currently sorting the results using the order property.
My question is: is it possible to somehow detect when the criteria of the order sort changes and to make a break?
For example- let's say the list that is populating is a list of food sorted by categorization. In our order property we will sort these DESC. It may look like this:
Apples
Bananas
Strawberries
Carrots
Lettuce
Now within the food categorization, there is a Fruits category and a Vegetables category. The question again, can I somehow know when the Vegetables sort is happening and then perform a to make the results look grouped?
Using the example from above, we would want to achieve:
Apples
Bananas
Strawberries
Carrots
Lettuce
Any help would be greatly appreciated.
Sorting via the Order parameter is done server-side, so I don't think there's a way to identify changes in groupings client-side without inspecting the data.
Since you're using AppSDK 1.x, this may be a situation where using an Array of queryConfig's could be useful to attain groupings of results that match your desired groupings. Then you could do something similar to the following:
var queryConfig = [];
queryConfig[0] = {
type : 'HierarchicalRequirement',
key : 'fruit_stories',
query: '(c_FoodCategory = "Fruits")',
fetch: 'Name,FormattedID,Description,c_FoodCategory'
};
queryConfig[1] = {
type : 'HierarchicalRequirement',
key : 'veggie_stories',
query: '(c_FoodCategory = "Vegetables")',
fetch: 'Name,FormattedID,Description,c_FoodCategory'
};
var rallyDataSource;
rallyDataSource = new rally.sdk.data.RallyDataSource('__WORKSPACE_OID__',
'__PROJECT_OID__',
'__PROJECT_SCOPING_UP__',
'__PROJECT_SCOPING_DOWN__');
rallyDataSource.findAll(queryConfig, processResults);
}
var processResults = function(results) {
var fruit_stories = results['fruit_stories'];
var veggie_stories = results['veggie_stories'];
var aDiv = document.getElementById("aDiv");
// Output fruit stories
for (var i=0; i<fruit_stories.length; i++) {
story = fruit_stories[i];
storyInfo += story.Name +
', ' + story.FormattedID +
', ' + story.c_FoodCategory + '<br>';
}
aDiv.innerHTML = '<strong>Name, State, Food Category</strong><br/>';
aDiv.innerHTML += storyInfo;
// Add break
aDiv.innerHTML += "<br/>
// Output veggie stories
for (var i=0; i<veggie_stories.length; i++) {
story = veggie_stories[i];
storyInfo += story.Name +
', ' + story.FormattedID +
', ' + story.c_FoodCategory + '<br>';
}
aDiv.innerHTML = '<strong>Name, State, Food Category</strong><br/>';
aDiv.innerHTML += storyInfo;
// Add break
aDiv.innerHTML += "<br/>
};
You may wish to consider using rally.sdk.ui.Table for displaying your output results - it provides a much cleaner visual display of columnar information.
Also, if you're interested in considering AppSDK 2.0, its Grid component contains a configuration attribute for automated grouping. See this example:
https://help.rallydev.com/apps/2.0/doc/#!/example/groupable-grid