Bigquery named parameters regex in Java - google-bigquery

I am looking for a way to use a regex as value in a named parameter in the Java SDK. According to the documentation, there is no datatype for that, and using a String parameter does not work.
Is there any way to use a regex as value in a named parameter?
QueryParameterValue Class has no datatype for that:
https://googleapis.dev/java/google-cloud-clients/0.91.0-alpha/com/google/cloud/bigquery/QueryParameterValue.html#of-T-com.google.cloud.bigquery.StandardSQLTypeName-
A regex in the query would e.g. look like this:
REGEXP_CONTAINS(some_attribute, r"^any\sregex\ssearchstring$")
and should be replaced by a named parameter like:
REGEXP_CONTAINS(some_attribute, #named_regex_parameter)
I tried different syntax in the query like
REGEXP_CONTAINS(some_attribute, r#named_regex_parameter)
etc. but none of them worked. The #named_regex_parameter is of type String. I tried to use values in the form of r"regex_expression" and just the regex_expression in the parameter value.
Seems like I need to build the query String without a named parameter for the regex part. Any hints to solve this with parameters would be really appreciated!
//Edit: added code example how the named parameters are used in the query config
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query)
.setDestinationTable(TableId.of(destinationDataset, destinationTable))
.setAllowLargeResults(true)
.setCreateDisposition(CreateDisposition.CREATE_IF_NEEDED)
.setTimePartitioning(partitioning)
.setWriteDisposition(WriteDisposition.WRITE_APPEND)
.setParameterMode("NAMED")
.addNamedParameter("regexExpressionParam", QueryParameterValue.string(someRegexExpressionStringVariable)) //this does not work
.addNamedParameter("someStringParam", QueryParameterValue.string(stringVariable))
.setPriority(Priority.BATCH)
.build();
The query should use the parameter #regexExpressionParam like so:
REGEXP_CONTAINS(theAttributeToQuery, #regexExpressionParam))

You need to pass the regular expression string without r'...'
I had a very similar problem with running parameterized queries on Python: it was something like this.
from google.cloud import bigquery
regex_input = "^begin_word.*end_here$"
# Construct a BigQuery client object.
client = bigquery.Client()
query = """
SELECT word, word_count
FROM `bigquery-public-data.samples.shakespeare`
WHERE REGEXP_CONTAINS(word, #regex)
ORDER BY word_count DESC;
"""
job_config = bigquery.QueryJobConfig(
query_parameters=[
bigquery.ScalarQueryParameter("regex", "STRING", f"r'{regex_input}'"),
]
)
query_job = client.query(query, job_config=job_config)
At first, I thought the input had to be wrapped by r'...'; just like how I normally write a regex on BQ explorer.
I tried to modify the string input to make it like a regular expression, which was this pard f"r'{regex_input}'" of the code.
but apparently BQ correctly escapes string without our help and I can just pass down the regex string like bigquery.ScalarQueryParameter("regex", "STRING", regex_input)

Related

How to create a correct filter string with OR and AND operators for django?

My app has a frontend on vue.js and backend on django rest framework. I need to do a filter string on vue which should do something like this:
((status=closed) | (status=canceled)) & (priority=middle)
but got an error as a response
["Invalid querystring operator. Matched: ') & '."]
After encoding my string looks like this:
?filters=((status%3D%D0%97%D0%B0%D0%BA%D1%80%D1%8B%D1%82)%20%7C%20(status%3D%D0%9E%D1%82%D0%BA%D0%BB%D0%BE%D0%BD%D0%B5%D0%BD))%20%26%20(priority%3D%D0%A1%D1%80%D0%B5%D0%B4%D0%BD%D0%B8%D0%B9)
which corresponds to
?filters=((status=closed)|(status=canceled))&(priority=middle)
How should look a correct filter string for django?
I have no problem if statement includes only | or only &. For example filter string like this one works perfect:
?filters=(status%3D%D0%97%D0%B0%D0%BA%D1%80%D1%8B%D1%82)%20%7C%20(status%3D%D0%9E%D1%82%D0%BA%D0%BB%D0%BE%D0%BD%D0%B5%D0%BD)
a.k.a. ?filters=(status=closed)|(status=canceled). But if i add an & after it and additional brackets to specify the order of conditions calculation it fails with an error.
I also tried to reduce usage of brackets and had string like this (as experiment):
?filters=(status%3D%D0%97%D0%B0%D0%BA%D1%80%D1%8B%D1%82%20%7C%20status%3D%D0%9E%D1%82%D0%BA%D0%BB%D0%BE%D0%BD%D0%B5%D0%BD)
a.k.a. ?filters=(status=closed | status=canceled). This one doesn't work - get neither error nor the data.
I need to have a mixed results in my case: both statuses (closed and canceled) and priority=middle, but a string format isn't correct. Please explain, which format would be Ok?
That doesn't look like a very uri friendly syntax you're trying to use there.
Try doing this instead:
?status[]=closed&status[]=cancelled&priority=middle
Then use request.GET.getlist('status[]') to get back the list and use the values for logical OR queryset filtering:
qs = qs.filter(status__in=request.GET.getlist('status[]', [])
and then add any additional filtering which works as logical AND.
If you're using axios, it should automatically format js status url param into proper format.

Drupal 7 - db_select: SQL function in where condition

I need to use this condition in my select statement:
WHERE YEAR(date) = YEAR(CURDATE())
but if I do, like this:
$query = db_select('table', 't');
$query->fields('t');
$query->condition('YEAR\(date\)', 'YEAR(CURDATE())', '=');
Drupal won't have it (even if I do not escape those parenthesis - it simply ignores them) because I get an error:
Column not found: 1054 Unknown column 'YEARdate' in 'where clause':
How to overcome this error?
Hmm.. just like this, it seems:
$query->where('YEAR(date) = YEAR(CURDATE())');
The where allows for the arbitrary SQL:
The where() method allows for the addition of arbitrary SQL as a conditional fragment. $snippet may contain any legal SQL fragment, and if it has variable content it must be added using a named placeholder. The $args array is an array of placeholders and values that will be substituted into the snippet. It is up to the developer to ensure that the snippet is valid SQL. No database-specific modifications are made to the snippet.
Hm, you could also use db_query, it allow you to write SQL queries "without Drupal".
I mean, you'll be able to add custom WHERE statements or any SQL-proper functions, like custom functions ;)
Eg.
$result = db_query('SELECT title FROM {node} WHERE type = "%s" AND title LIKE "%%%s%%"', 'type', 'title');
Use addExpression method :
https://api.drupal.org/api/drupal/includes!database!select.inc/function/SelectQuery%3A%3AaddExpression/7.x
$query = db_select('table', 't');
$query->fields('t');
$query->addExpression('YEAR(t.date) = YEAR(CURDATE())');
$result = $query->execute()->fetchAll();
var_dump($result);

postgresql replace function using pattern matching characters

I am having a table named "OptionsDetail" with column named "URL" in postgresql database. The "URL" column contain the following data
URL
http://www.site1.com/ebw/file1/detail.action?cid=1
http://www.another2.com/ebw/file1/detail.action?cid=11&code=MYCODE
http://www.anothersite3.com/ebw/file1/detail.action?cid=12&code=ANOTHERCODE&option=ROOM
Now I want to replace the data in URL to
URL
/file1/detail.action?cid=1
/file1/detail.action?cid=11&code=MYCODE
/file1/detail.action?cid=12&code=ANOTHERCODE&menu=ROOM
I wrote the following query to perform the above functionality
UPDATE "OptionsDetail" SET "URL" = replace("URL",'http://%/ebw/file1','/file1') WHERE "URL" LIKE '%/ebw/file1%';
And also another way I wrote like this
UPDATE "OptionsDetail" SET "URL" = replace("URL",'%/ebw/file1','/file1') WHERE "URL" LIKE '%/ebw/file1%';
Query is executing successfully saying like for ex: 200 rows affected but "URL" column data is not changing the way I need, the data is remaining as it is.
Please help me to resolve this issue
The problem is that replace doesn't support wildcards like %:
replace("URL",'http://%/ebw/file1','/file1')
^^^
You can use regexp_replace instead:
UPDATE YourTable
SET URL = regexp_replace(URL, 'http://.*/ebw/file1', '/file1')
WHERE URL LIKE '%/ebw/file1%'
Note that regexp_replace uses different wildcards than like. In regular expressions, "Any number of any character" is .* instead of %.
See it working at SQL Fiddle.

How to evaluate parameter variable stored in a file in jMeter?

Say, I have a jMeter variable var equals 123.
I also have a query stored in a sql file:
SELECT * FROM Table WHERE ID = ${var}
All I need is to read that query from the file and evaluate ${var} into actual value, and then execute it in JDBC Sampler. So I need to combine these two pieces into
SELECT * FROM Table WHERE ID = 123
and pass the query to JDBC sampler.
Though, jMeter doesn't evaluate the ${var} parameter stored in that sql file and all I can pass to JDBC sampler is (obvious one):
SELECT * FROM Table WHERE ID = ${var}.
Does anyone know how to make jMeter evaluate the stored variable into actual value?
I had a similar requirement.
You need to use Beanshell Preprocessor for the JDBC sampler. Copy the below script and put it in a .bsh file and call it. I assumed you have the query stored in 'SQLQuery' variable.
I tested the below script and it works.
import java.util.regex.Matcher;
import java.util.regex.Pattern;
String regex = "\\$\\{([^}]+)\\}";
SQLQuery = vars.get("SQLQuery");
Pattern pattern = Pattern.compile(regex);
Matcher matcher = pattern.matcher(SQLQuery);
while(matcher.find())
{
String token = matcher.group(); //${var}
String tokenKey = matcher.group(1); // var
SQLQuery = SQLQuery.replaceFirst(Pattern.quote(token), Matcher.quoteReplacement(vars.get(tokenKey)));
}
vars.put("SQLQuery", SQLQuery);
You need to use a "Prepared Select Statement" as Query Type. Prepared Statements
Then replace your ${var} with ?
SELECT * FROM Table WHERE ID = ?
And add the 2 parameters at the bottom.
Parameter values: ${var}
Parameter types: INT

SQL: Use a predefined list in the where clause

Here is an example of what I am trying to do:
def famlist = selection.getUnique('Family_code')
... Where “””...
and testedWaferPass.family_code in $famlist
“””...
famlist is a list of objects
‘selection’ will change every run, so the list is always changing.
I want to return only columns from my SQL search where the row is found in the list that I have created.
I realize it is supposed to look like: in ('foo','bar')
But no matter what I do, my list will not get like that. So I have to turn my list into a string?
('\${famlist.join("', '")}')
Ive tried the above, idk. Wasn’t working for me. Just thought I would throw that in there. Would love some suggestions. Thanks.
I am willing to bet there is a Groovier way to implement this than shown below - but this works. Here's the important part of my sample script. nameList original contains the string names. Need to quote each entry in the list, then string the [ and ] from the toString result. I tried passing as prepared statement but for that you need to dynamically create the string for the ? for each element in the list. This quick-hack doesn't use a prepared statement.
def nameList = ['Reports', 'Customer', 'Associates']
def nameListString = nameList.collect{"'${it}'"}.toString().substring(1)
nameListString = nameListString.substring(0, nameListString.length()-1)
String stmt = "select * from action_group_i18n where name in ( $nameListString)"
db.eachRow( stmt ) { row ->
println "$row.action_group_id, $row.language, $row.name"
}
Hope this helps!