I have a small issue, so here's a bit of background:
We are developing a Qlik Sense application and we normally write our expressions to an external script. We save these as variables, and then evaluate the variables in the application. The advantage of this is a) we can use better version control with GIT, and b) we can separate the queries from the application if we ever need to change platforms in future.
My Problem:
I have come across a situation where we need to concat a string to the result of an expression, which can be done easily in the application, but when you save the expression to an external file the single quotes around the expression interfere with the single quotes around the string.
I tried
using double quotes for the string only, but qlik doesn't evaluate it correctly.
same goes for the expression using double quotes only.
escaping the single quote inside the expression, eg. "\'" but same story.
What I was thinking of doing next was changing the quote to a rogue character so qlik would ignore it as text, then replacing it with a quote later so qlik would then try to evaluate it.
Example Code:
SET variable = 'if(isnull(month),'Month: ' & date(now(), 'MMM-YYYY'),'Month: ' & only({$<year={2016}, month={6}>}month)';
After some further research I found that Qlik has its own way of escaping characters without using the "\" character. I was able to solve this issue by escaping the inner single quotes like this:
SET variable = 'if(isnull(month),''Month: '' & date(now(), ''MMM-YYYY''),''Month: '' & only({$<year={2016}, month={6}>}month)';
Feels like a pretty silly oversight now, but hopefully this will save someone some time in the future.
Related
I have some automated workflow, which includes updating a column via SQL with HTML tags in it.
The basic SQL statement goes like this:
UPDATE content SET bodytext = '<div class="one two three">Here comes a whole lot of HTML with all special chars and double quotes " and single quotes ' and empty lines and all possible kind of stuff...</div>' WHERE pid = 10;
Is there a way to make MariaDB or MySQL to escape things automatically in SQL (without PHP)?
I'd suggest to use prepared statements. This way you separate the statement from it's parameters and don't need to care about additional escaping necessary in plain SQL.
Using functionality provided in PHP's MySQLi driver would simplify the process:
https://www.w3schools.com/php/php_mysql_prepared_statements.asp
Prepared statements are also possible in plain SQL, but I'm not sure if doing it manually would be worth the hassle
https://dev.mysql.com/doc/refman/8.0/en/sql-prepared-statements.html
Thank you for your input, but I think, I found a solution which works for me. It seems that you actually can tell the SQL server to accept a raw string by this kind of syntax:
SELECT q'[The 'end' of the day]'
(Source: https://www.databasestar.com/sql-escape-single-quote/)
So I did the following:
SELECT #html := '[<div class="one two three">Here comes a whole lot of HTML with all special chars and double quotes " and single quotes '' and empty lines and all possible kind of stuff...</div>]';
UPDATE content SET bodytext = #html WHERE pid = 10;
And it works that way without any escaping problems.
I am using VBA in Ms Access environment, to handle long string (memo field storing HTML originally).
After positioning by Instr(), I put the position into Mid(vStr,vStartPos,vEndPos-vStartPos+1) to extract the string, but the output doesn't match. I have already carefully checked this in immediate windows, as well as NotePad++. What I can say is Instr() and NotePad++ have given the same counting result, while Mid() is different. Mid()'s result are former than Instr()'s in some cases, and latter in other cases. I don't know the reason, and can just believe Mid() use different mechanism or have defeative (surprised!) in handling long string mixed with single-byte and bi-byte chars (but this is common in the world, and meet no problem before), and possibly some special characters.
I believe I need to custom-make a Mid() function. Any idea how to do it effectively and efficiently?
Thanks all for your reply. After I created a custom Mid() by RegEx and find that the problem has no change, I have found out the silly mistake I made. The Instr() and Mid() have no problem, but the string has been carelessly modified between them. So this case should be closed now.
I need to remove leading zeros from a string field in an Access database that is destroyed and recreated every time it is used within a C# program. Most string libraries (even SQL ones) include a Trim function to remove leading or following whitespace. Unfortunately, Access does not seem to have a LTrim(string s, char[] trimChars) or something similar. To get around this, I concocted this monstrosity:
Replace(LTrim(Replace(ADDRNO,'0', ' ')),' ', '0')
But this resulted in an undefined function reference for Replace, even though it is obviously an Access function.
What I am looking for is a way to trim these zeros, either by getting the JET engine to let me use the Replace function or by some other method entirely.
EDIT: Fixed syntax of Replace function. Problem still persists.
I suggest
Val(ADDRNO)
It will return the number portion without the leading zeros.
I think it's just the order of your parameters that is wrong:
debug.? Replace("My string", "i", "o") -> "My Strong"
You can use Trim and Replace.
I'm not sure what context you are running this but this seems to show the parameter order is different and uses double quotes instead of single quotes(I haven't used Access in awhile so maybe it doesn't matter), also try square brackets on column name:
http://www.techonthenet.com/access/functions/string/replace.php
Replace(LTrim(Replace([ADDRNO], "0", " "))," ", "0")
If that gives the same error just try the replace function by itself to narrow down the problem:
Replace ("alphabet", "a", "e")
If this works then you know the Replace function works, and there is some other issue.
Edit: If it doesn't work at all, then Replace is likely a VBA function available only in the Access application, and is not part of Jet. You could try some combination of Left/Right function and chop the string up, this can get quite ugly. I personally would just iterate over the record set and use C# code to modify the values. Hopefully you don't have such a large number of records that this would be a problem.
I would like a regex that would make this:
VALUES('Hit 'n Run')
into
VALUES('Hit ''n Run')
Is this possible?
No, this is not really possible. If you have VALUES('Hit 'n Run'), you already have an invalid mixture of delimiting apostrophes and literal apostrophes. String processing is like mixing sugar and salt: once you've mixed contexts without proper escaping there is no way of pulling them back apart.
If you are trying to rescue broken data, you could try something like (?<!\()'(?!\)) to match apostrophes that don't have a bracket next to them. It's a weak and easily fooled tactic but for simple data it might work.
If you are putting together dynamic SQL queries you must escape the ' before you put it into the query string, either using a simple string replace ' with '' if you're sure that's the only escape your DBMS requires, or — much better — using a dedicated SQL-string-literal-escaping function appropriate to your DBMS. Quite what that function would be depends on what platform (language, DBMS) you're talking about.
Any pattern that could be expressed in RegEx could then be exploited to create the very SQL injection issues you're trying to avoid.
Example nasty input:
VALUES(');DELETE * FROM customer;SELECT '
I am writing a backend application which needs to be able to send multiple SQL commands to a MySQL server.
MySQL >= 5.x support multiple statements, but unfortunately we are interfacing with MySQL 4.x.
I am trying to find a way (hint: regex) to split SQL statements by their semicolon, but it should ignore semicolons in single and double quotes strings.
http://www.dev-explorer.com/articles/multiple-mysql-queries has a very nice regex to do that, but doesn't support double quotes.
I'd be happy to hear your suggestions.
Can't be done with regex, it's insufficiently powerful to parse SQL. There may be an SQL parser available for your language — which is it? — but parsing SQL is quite hard, especially given the range of different syntaxes available. Even in MySQL alone there are many SQL_MODE flags on a server and connection level that can affect how basic strings and comments are parsed, making statements behave quite differently.
The example at dev-explorer goes to amusing lengths to try to cope with escaped apostrophes and trailing strings, but will still fail for many valid combinations of them, not to mention the double quotes, backticks, the various comment syntaxes, or ANSI SQL_MODE.
As bobince said, regular expressions are probably not going to be powerful enough to do this. They're certainly not going to be powerful enough to do it in any halfway elegant manner. The second link cdonner provided also does not address this; most answers there were trying to talk the questioner out of doing this without semicolons; if he had taken the general advice, then he'd have ended up where you are.
I think the quickest path to solving this is going to be with a string scanner function, that examines every character of the string in sequence, and reacts based on a bit of stored state. Rough pseudocode:
Read in a character
If the character is not special, CONTINUE
If the character is escaped (checking this probably requires examining the previous character), CONTINUE
If the character would start a new string or end an existing one, toggle a flag IN_STRING (you might need multiple flags for different string types... I've honestly tried and succeeded at remaining ignorant of the minutiae of SQL quoting/escaping) and CONTINUE
If the character is a semicolon AND we are not currently in a string, we have found a query! OUTPUT it and CONTINUE scanning until the end of the string.
Language parsing is not any of my areas of experience, so you'll want to consider that approach carefully; nonetheless, it's going to be fast (with C-style strings, none of those steps are at all expensive, save possibly for the OUTPUT, depending on what "outputting" means in your context) and I think it should get the job done.
maybe with the following Java Regexp? check the test...
#Test
public void testRegexp() {
String s = //
"SELECT 'hello;world' \n" + //
"FROM DUAL; \n" + //
"\n" + //
"SELECT 'hello;world' \n" + //
"FROM DUAL; \n" + //
"\n";
String regexp = "([^;]*?('.*?')?)*?;\\s*";
assertEquals("<statement><statement>", s.replaceAll(regexp, "<statement>"));
}
I would suggest seeing if you can redefine the problem space so the need to send multiple queries separated only by their terminator is not required.
Try this. Just replaced the 1st ' with \" and it seems to work for both ' and "
;+(?=([^\"|^\\']['|\\'][^'|^\\']['|\\'])[^'|^\\'][^'|^\\']$)