After I run a query and view the output, for example
select * from People
My output is as follows
First Last Email
Ray Smith raysmith#whatever.itis
How would I export this data so that it looks as follows?
"Ray","Smith","raysmith#whatever.itis"
Or is there a way to do this within SQL to modify records to contain quotes?
Because when you export, it's going to include the commas anyway, right?
If the columns you're interested in are 128 characters or less, you could use the QUOTENAME function. Be careful with this as anything over 128 characters will return NULL.
SELECT QUOTENAME(First, '"'), QUOTENAME(Last, '"'), QUOTENAME(Email, '"')
FROM People
select '"'+first+'","'+last+'","'+email+'"'
from people
This is the kind of thing best done in code however, you shouldn't query for presentation.
select concat(“\"”,first,“\"”,“\"”,Last,“\"”,“\"”,Email,“\"”) as allInOne
Modifying the records to contain quotes would be a disaster; you don't use the data only for export. Further, in theory you'd have to deal with names like:
Thomas "The Alley Cat" O'Malley
which presents some problems.
In Standard SQL, you'd use doubled-up single quotes to enclose single quotes (with no special treatment for double quotes):
'"Thomas "The Alley Cat" O''Malley"'
Some DBMS allow you to use double quotes around strings (in Standard SQL, the double quotes indicate a 'delimited identifier'; SQL Server uses square brackets for that), in which case you might write the string as:
"""Thomas ""The Alley Cat"" O'Malley"""
Normally, though, your exporter tools provide CSV output formatting and your SQL statement does not need to worry about it. Embedded quotes make anything else problematic. Indeed, you should usually not make the DBMS deal with the formatting of the data.
This worked best for me
SELECT 'UPDATE [dbo].[DirTree1] SET FLD2UPDATE=',QUOTENAME(FLD2UPDATE,'''')
+' WHERE KEYFLD='+QUOTENAME(KEYFLD,'''')
FROM [dbo].[Table1]
WHERE SUBSTRING(FLD2UPDATE,1,2) = 'MX'
order by 2
If you are using MS SQL Server, try something like:
SELECT '"'||Table.Column||'"'
FROM Table
-- Note that the first 3 characters between "SELECT" and "||" are:
' " '
-- The characters are the same after "||" at the end... that way you get a " on each side of your value.
Related
I'd like to generate a query like this:
select id, '=FKERES(B2;'D:\nyunyuka\[kotyog.xlsx]Munka1'!$A:$B;2;0)' as
excel_formula from table;
I learned that I have to escape the ' character using E but it seems not working. I assume that I have to escape all the \ as well. But It's getting complicated to me.
I am not sure if I understood the question correct. Hope that I did.
You need to add extra single quotes like this:
select id, '=FKERES(B2;''D:\nyunyuka[kotyog.xlsx]Munka1''!$A:$B;2;0)' as excel_formula
from table;
So, if there is a single quote ' in a string in a select clause then you have to add another one '' to make it work.
Besides doubling the quotes you can use dollar quoting instead.
So use
select id, $Q$=FKERES(B2;'D:\nyunyuka[kotyog.xlsx]Munka1'!$A:$B;2;0)$Q$ as excel_formula
from table;
Instead of
select id, '=FKERES(B2;''D:\nyunyuka[kotyog.xlsx]Munka1''!$A:$B;2;0)' as excel_formula
from table;
Where $Q$ in the above takes the place of the opening and closing quote ('). It is generally not worth it for just a couple instances but for long strings with lots of quotes it can sure comes in handy.
I am running below query in Teradata :
sel requesttext from dbc.tables
where tablename='old_employee_table'
Result:
alter table DB_NAME.employee_table,no fallback ;
I want to get below result using SQL:
DB_NAME.employee_table
Requesttext can be:
create set table DB_NAME.employee_table;
DB Name and table can occur anywhere in the result. Since .(dot) is joining them that's why i want to split with .(dot).
Basically I need sql which can result me surrounding values of .(dot)
I want DBName and Tablename in result.
I'm not a Teradata person, but this should work for both strings given so far, as long as teradata's regexp_substr() supports positive look-behind and positive look-ahead assertions (I might have the Teradata syntax wrong, so a little tweaking may be needed):
SELECT REGEXP_SUBSTR(requesttext, '(?<= )(\w+\.\w+)(?=[,$]?)', 1, 1)
FROM dbc.tables
WHERE tablename='old_employee_table'
See the regex101 example. Hopefully it translates to Teradata easily.
The regex looks for and returns the words either side of and including the period, when preceded by a space, and followed by an optional comma or the end of the line.
You could do this with either regexp_substr() or strtok().
As Jamie Zawinski said:
Some people, when confronted with a problem, think "I know, I'll use
regular expressions." Now they have two problems.
So I would go with the strtok() method. Also I'm lazy and regular expressions are hard.
Function strtok() takes three arguments:
The string being split
The delimiter to split the string
The number of the token to grab.
To get at the <database>.<table> from that string that is returned in your query, we can split by a space, grab the third token, then split that by a comma and grab the first token.
That would look like:
SELECT strtok(strtok(requestText,' ',3),',',1)
FROM dbc.tables
WHERE tablename='old_employee_table'
I have a string with value
'MAX DATE QUERY: SELECT iso_timestamp(MAX(time_stamp)) AS MAXTIME FROM observation WHERE offering_id = 'HOBART''
But on inserting into postgresql table i am getting error:
org.postgresql.util.PSQLException: ERROR: syntax error at or near "HOBART".
This is probably because my string contains single quotes. I don't know my string value. Every time it keeps changing and may contain special characters like \ or something since I am reading from a file and saving into postgres database.
Please give a general solution to escape such characters.
As per the SQL standard, quotes are delimited by doubling them, ie:
insert into table (column) values ('I''m OK')
If you replace every single quote in your text with two single quotes, it will work.
Normally, a backslash escapes the following character, but literal backslashes are similarly escaped by using two backslashes"
insert into table (column) values ('Look in C:\\Temp')
You can use double dollar quotation to escape the special characters in your string.
The above query as mentioned insert into table (column) values ('I'm OK')
changes to insert into table (column) values ($$I'm OK$$).
To make the identifier unique so that it doesn't mix with the values, you can add any characters between 2 dollars such as
insert into table (column) values ($aesc6$I'm OK$aesc6$).
here $aesc6$ is the unique string identifier so that even if $$ is part of the value, it will be treated as a value and not a identifier.
You appear to be using Java and JDBC. Please read the JDBC tutorial, which describes how to use paramaterized queries to safely insert data without risking SQL injection problems.
Please read the prepared statements section of the JDBC tutorial and these simple examples in various languages including Java.
Since you're having issues with backslashes, not just 'single quotes', I'd say you're running PostgreSQL 9.0 or older, which default to standard_conforming_strings = off. In newer versions backslashes are only special if you use the PostgreSQL extension E'escape strings'. (This is why you always include your PostgreSQL version in questions).
You might also want to examine:
Why you should use prepared statements.
The PostgreSQL documentation on the lexical structure of SQL queries.
While it is possible to explicitly quote values, doing so is error-prone, slow and inefficient. You should use parameterized queries (prepared statements) to safely insert data.
In future, please include a code snippet that you're having a problem with and details of the language you're using, the PostgreSQL version, etc.
If you really must manually escape strings, you'll need to make sure that standard_conforming_strings is on and double quotes, eg don''t manually escape text; or use PostgreSQL-specific E'escape strings where you \'backslash escape\' quotes'. But really, use prepared statements, it's way easier.
Some possible approaches are:
use prepared statements
convert all special characters to their equivalent html entities.
use base64 encoding while storing the string, and base64 decoding while reading the string from the db table.
Approach 1 (prepared statements) can be combined with approaches 2 and 3.
Approach 3 (base64 encoding) converts all characters to hexadecimal characters without loosing any info. But you may not be able to do full-text search using this approach.
Literals in SQLServer start with N like this:
update table set stringField = N'/;l;sldl;'''mess'
I have a query that I think should look like this:
select *
from Requesters
where CITIZEN_STATUS = 'OS-IE ';
The field CITIZEN_STATUS, whose data type is varchar(15), has a trailing space for this particular value. I have pasted it into Notepad++ and looked at it with a hex editor, and the final space is indeed 0x20.
For the query to work, I have to write it like this:
select *
from Requesters
where CITIZEN_STATUS like 'OS-IE%';
So, obviously, I have a workaround and the question is not urgent. But I would really like to know why the first query fails to do what I expect. Does anyone have any ideas?
I should mention I am using SQL Server 2005 and can provide more information about the configuration if needed.
In MySQL 5, this query works. However, it does not distinguish on trailing whitespace. The query matches 'OS-IE ' as well as 'OS-IE'. In SQL Server 2005 you can use a regular expression that defines the end of a line. The correct character for this is the dollar sign '$' to indicate that you do want the space. See http://msdn.microsoft.com/en-us/magazine/cc163473.aspx
I have been trying to learn more about SQL Injection attacks. I understand the principle, but when I actually look at some of these attacks I don't know how they are doing what they are doing.
There often seem to be uneven quotes, and a lot of obfuscation via HEX characters.
I don't get the HEX characters..., surely they are translated back to ASCII by the browser, so what is the point?
However, I am mainly confused by the odd quoting. I am having trouble finding an example right now, however it usually seems that the quoting will end at some point before the end of the statement, when I would have thought it would be at the end?
Perhaps an example is the common use of '1 or 1=1
What is such a statement doing?
I don't get the HEX characters...,
surely they are translated back to
ASCII by the browser
Nope.
However, I am mainly confused by the
odd quoting. I am having trouble
finding an example right now, however
it usually seems that the quoting will
end at some point before the end of
the statement, when I would have
thought it would be at the end?
Imagine you're building inline SQL instead of using parameter substitution as you should. We'll use a make-believe language that looks a lot like PHP for no particular reason.
$sql = "delete from foo where bar = '" + $param + "'";
So, now, imagine that $param gets set by the browser as such...
$param = "' or 1=1 --"
(We're pretending like -- is the SQL comment sequence here. If it's not there's ways around that too)
So, now, what is your SQL after the string substitution is done?
delete from foo where bar = '' or 1=1 --'
Which will delete every record in foo.
This has been purposefully simple, but it should give you a good idea of what the uneven quotes are for.
Let's say that we have a form where we submit a form with a name field. name was used in a variable, $Name. You then run this query:
INSERT INTO Students VALUES ( '$Name' )
It will be translated into:
INSERT INTO Students VALUES ( 'Robert' ); DROP TABLE STUDENTS; --')
The -- is a comment delimiter. After that everything will be ignored. The ' is used to delimit string literals.
To use hex chars in an attack has some reasons. One of the is the obfuscation, other one to bypass some naive security measures.
There are cases where quote marks are prohibited with sql injection. In this case an attacker has to use an encoding method, such as hex encoding for his strings. For instance '/etc/passwd' can be written as0x2f6574632f706173737764 which doesn't require quote marks. Here is an example of a vulnerable query where quote marks are prohibited. :
mysql_query("select name from users where id=".addslashes($_GET[id]));
If you want to use a mysql function like load_file(), then you have to use hex encoding.
PoC:/vuln.php?id=1 union select load_file(0x2f6574632f706173737764)
In this case /etc/passwd is being read and will be the 2nd row.
Here is a variation on the hex encode function that I use in my MySQL SQL Injection exploits:
function charEncode($string){
$char="char(";
$size=strlen($string);
for($x=0;$x<$size;$x++){
$char.=ord($string[$x]).",";
}
$char[strlen($char)-1]=")%00";
return $char;
}
I use this exact method for exploiting HLStats 1.35. I have also used this function in my php nuke exploit to bypass xss filters for writing <?php?> to the disk using into outfile. Its important to note that into outfile is a query operator that does not accept the output to a function or a hex encoded string, it will only accept a quoted string as a path, thus in the vulnerable query above into outfile cannot be used by an attacker. Where as load_file() is a function call and a hex encoding can be used.
As for the uneven quoting; sql injection occurs where the coder didn't sanitize user input for dangerous characters such as single quotes - consider following statement
SELECT * FROM admin WHERE username='$_GET["user"]' and password='$_GET["pass"]'
if I know that valid user is 'admin' and insert the 'or 1=1 I will get following
SELECT * FROM admin WHERE username='admin' and password='something' or 1=1
This will result in returning the query alway true, because the left side of the expression will always be true, regardless of the value of the password.
This is the most simple example of sql injection, and ofter you will find that the attacker won't need to use the quote at all, or maybe comment the rest of the query out with comment delimiter such as -- or /*, if there are more parameters passed after the injection point.
As for the HEX encoding, there may be several reasons, avoiding filtering, it is simply easier to use hex encoded values, because you don't need to worry about quoting all your values in a query.
This is for instance useful if you want to use concat to co-notate two fields together like so:
inject.php?id=1 and 1=0 union select 1,2,concat(username,0x3a3a,password) from admin
Which would providing 3rd row is visible one, return for isntance admin::admin. If I didn't use hex encoding, I would have to do this:
inject.php?id=1 and 1=0 union select 1,2,concat(username,'::',password) from admin
This could be problem with aforementioned addslashes function, but also with poorly written regex sanitization functions or if you have very complicated query.
Sql injection is very wide topic, and what I've covered is hardly even an introduction through.