Related
I have to write a select statement following the following pattern:
[A-Z][0-9][0-9][0-9][0-9][A-Z][0-9][0-9][0-9][0-9][0-9]
The only thing I'm sure of is that the first A-Z WILL be there. All the rest is optional and the optional part is the problem. I don't really know how I could do that.
Some example data:
B/0765/E 3
B/0765/E3
B/0764/A /02
B/0749/K
B/0768/
B/0784//02
B/0807/
My guess is that I best remove al the white spaces and the / in the data and then execute the select statement. But I'm having some problems writing the like pattern actually.. Anyone that could help me out?
The underlying reason for this is that I'm migrating a database. In the old database the values are just in 1 field but in the new one they are splitted into several fields but I first have to write a "control script" to know what records in the old database are not correct.
Even the following isn't working:
where someColumn LIKE '[a-zA-Z]%';
You can use Regular Expression via xQuery to define this pattern. There are many question in StackOverFlow that talk about patterns in DB2, and they have been solved with Regular Expressions.
DB2: find field value where first character is a lower case letter
Emulate REGEXP like behaviour in SQL
I see this:
Project.update_all("cost = cost * 3",
"lower(technology) LIKE '%microsoft%'")
as an example of update_all method in Active Record when I'm following The Rails 3 Way, very simple phrase, huh? But I just can't figure out what do parentheses mean in lower(technology) here.
So, Could you tell me some possible answers? Because I don't know if there are some different situations we can use parentheses like this.
thanks.
They call the SQL LOWER function to lowercase the string.
LOWER technology
would be a syntax error, because LOWER is a function, not a keyword.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Is there a good reason to use upper case for T-SQL keywords?
I personally find a string of lowercase characters to be more readable than a string of uppercase characters. Is some old/popular flavor of SQL case-sensitive or something?
For reference:
select
this.Column1,
case when this.Column2 is null then 0 else this.Column2 end
from dbo.SomeTable this
inner join dbo.AnotherTable another on this.id = another.id
where
this.Price > 100
vs.
SELECT
this.Column1,
CASE WHEN this.Column2 IS NULL THEN 0 ELSE this.Column2 END
FROM dbo.SomeTable this
INNER JOIN dbo.AnotherTable another ON this.id = another.id
WHERE
this.Price > 100
The former just seems so much more readable to me, but I see the latter way more often.
I agree with you - to me, uppercase is just SHOUTING.
I let my IDE handle making keywords stand out, via syntax highlighting.
I don't know of a historical reason for it, but by now it's just a subjective preference.
Edit to further make clear my reasoning:
Would you uppercase your keywords in any other modern language? Made up example:
USING (EditForm form = NEW EditForm()) {
IF (form.ShowDialog() == DialogResult.OK) {
IF ( form.EditedThing == null ) {
THROW NEW Exception("No thing!");
}
RETURN form.EditedThing;
} ELSE {
RETURN null;
}
}
Ugh!
Anyway, it's pretty clear from the votes which style is more popular, but I think we all agree that it's just a personal preference.
I think the latter is more readable. You can easily separate the keywords from table and column names, etc.
One thing I'll add to this which I haven't seen anyone bring up yet:
If you're using ad hoc SQL from within a programming language you'll have a lot of SQL inside strings. For example:
insertStatement = "INSERT INTO Customers (FirstName, LastName) VALUES ('Jane','Smith')"
In this case syntax coloring probably won't work so the uppercasing could be helping readability.
From Joe Celko's "SQL Programming Style" (ISBN 978-0120887972):
Rule:
Uppercase the Reserved Words.
Rationale:
Uppercase words are seen as a unit,
rather than being read as a series of
syllables or letters. The eye is drawn
to them, and they act to announce a
statement or clause. That is why
headlines and warning signs work.
Typographers use the term bouma for
the shape of a word. The term appears
in paul Saenger's book (1975). Imagine
each letter on a rectangular card that
just fits it, so that you see the
ascenders, descenders, and baseline
letters as various "Lego blocks" that
are snapped together to make a word.
The bouma of an uppercase word is
always a simple, dense rectangle, and
it is easy to pick out of a field of
lowercase words.
What I find compelling is that this is the only book about SQL heuristics, written by a well-known author of SQL works. So is this the absolute truth? Who knows. It sounds reasonable enough and I can at least point out the rule to a team member and tell them to follow it (and if they want to blame anyone I give them Celko's email address :)
Code has punctuation which SQL statements lack. There are dots and parentheses and semicolons to help you keep things separate. Code also has lines. Despite the fact that you can write a SQL statement on multiple physical lines, it is a single statement, a single "line of code."
IF I were to write English text without any of the normal punctuation IT might be easier if I uppercased the start of new clauses THAT way it'd be easier to tell where one ended and the next began OTHERWISE a block of text this long would probably be very difficult to read NOT that I'd suggest it's easy to read now BUT at least you can follow it I think
Mostly it's tradition. We like to keep keywords and our namespace names separate for readability, and since in many DBMSes table and column names are case sensitive, we can't upper case them, so we upper case the keywords.
I prefer lower case keywords. SQL Server Management Studio color codes the keywords, so there is no problem distinguishing them from the identifiers.
And upper case keywords feels so... well... BASIC... ;)
-"BASIC, COBOL and FORTRAN called from the eighties, and they wanted their UPPERCASE KEYWORDS back." ;)
I like to use upper case on SQL keywords. I think my mind skips over them as they are really blocky and concentrates on what's important. The blocky words split up the important bits when you layout like this:
SELECT
s.name,
m.eyes,
m.foo
FROM
muppets m,
muppet_shows ms,
shows s
WHERE
m.name = 'Gonzo' AND
m.muppetId = ms.muppetId AND
ms.showId = s.showId
(The lack of ANSI joins is an issue for another question.)
There is a psychology study that shows lowercase was quicker to read than uppercase due to the outlines of the words being more distinctive. However, this effect can disappear about with lots of practice reading uppercase.
What's worse it that as the majority of developers at my office believe in capitals for SQL keyword, so I have had to change to uppercase. Majority rules.
I believe lowercase is easier to read and that given that SQL keywords are highlighted in blue anyway.
In the glory days, keywords were in capitals because we were developing on green screens!
The question is: if we don't write C# keywords in uppercase then why do I have to write SQL keywords in uppercase?
Like someone else has said - capitals are SHOUTING!
Back in the 1980s, I used to capitalize database names, and leave SQL keywords in lower case. Most writers did the opposite, capitalizing the SQL keywords. Eventually, I started going along with the crowd.
Just in passing, I'll mention that, in most published code snippets in C, C++, or Java the language keywords are always in lower case, and upper case keywords may not even be recognized as such by some parsers. I don't see a good reason for using the opposite convention in SQL that you use in the programming language, even when the SQL is embedded in source code.
And I'm not defending the use of all caps for database names. It actually looks a little like "shouting". And there are better conventions, like using a few upper case letters in database names. (By "database names" I mean the names of schemas, schema objects like tables, and maybe a few other things.) Just because I did it in the 80s doesn't mean I have to defend it today.
Finally, "De gustibus non disputandum est".
It's just a matter of readability and helps you quickly distinguish SQL keywords.
Btw, that question was already answered:
Is SQL syntax case sensitive?
I prefer using upper case as well for keywords in SQL.
Yes, lower case is more readable, but for me having to take an extra second to scan through the query will do you good most of the time. Once it's done and tested you should rarely ever see it again anyway (DAL, stored procedure or whatever will hide it from you).
If you are reading it for the first time, capitalized WHERE AND JOIN will jump right at you, as they should.
It’s just a question of readability. Using UPPERCASE for the SQL keywords helps make the script more understandable.
I capitalize SQL to make it more "contrasty" to the host language (mostly C# these days).
It's just a matter of preference and/or tradition really...
Apropos of nothing perhaps, but I prefer typesetting SQL keywords in small caps. That way they look capitalized to most readers, but they aren't the same as the ugly ALL CAPS style.
A further advantage is that I can leave the code as is and print it in the traditional style. (I use the listings package in LaTeX for pretty-printing code.)
Some SQL developers here like to lay it out like this:
SELECT s.name, m.eyes, m.foo
FROM muppets m, muppet_shows ms, shows s
WHERE m.name = 'Gonzo' AND m.muppetId = ms.muppetId AND ms.showId = s.showId
They claim this is easier to read unlike your one field per line approach which I use myself.
I am coding in ColdFusion, but trying to stay in cfscript, so I have a function that allows me to pass in a query to run it with
<cfquery blah >
#query#
</cfquery>
Somehow though, when I construct my queries with sql = "SELECT * FROM a WHERE b='#c#'" and pass it in, ColdFusion has replaced the single quotes with 2 single quotes. so it becomes WHERE b=''c'' in the final query.
I have tried creating the strings a lot of different ways, but I cannot get it to leave just one quote. Even doing a string replace has no effect.
Any idea why this is happening? It is ruining my hopes of living in cfscript for the duration of this project
ColdFusion, by design, escapes single quotes when interpolating variables within <cfquery> tags.
To do what you want, you need to use the PreserveSingleQuotes() function.
<cfquery ...>#PreserveSingleQuotes(query)#</cfquery>
This doesn't address, however, the danger of SQL injection to which you are exposing yourself.
Using <cfqueryparam> also allows your database to cache the query, which in most cases will improve performance.
It might be helpful to read an old Ben Forta column and a recent post by Brad Wood for more information about the benefits of using <cfqueryparam>.
The answer to your question, as others have said, is using preserveSingleQuotes(...)
However, the solution you actually want, is not to dynamically build your queries in this fashion. It's Bad Bad Bad.
Put your SQL inside the cfquery tags, with any ifs/switches/etc as appropriate, and ensure all CF variables use the cfqueryparam tag.
(Note, if you use variables in the ORDER BY clause, you'll need to manually escape any variables; cfqueryparam can't be used in ORDER BY clauses)
ColdFusion automatically escapes single quotes quotes in <cfquery> tags when you use the following syntax:
SELECT * FROM TABLE WHERE Foo='#Foo#'
In case you would want to preserve single quotes in #Foo# you must call #PreserveSingleQuotes(Foo)#.
Be aware the the automatic escaping works only for variable values, not for function results.
SELECT * FROM TABLE WHERE Foo='#LCase(Foo)#' /* Single quotes are retained! */
In that light, the function PreserveSingleQuotes() (see Adobe LiveDocs) is not much more than a "null operation" on the value - turning it into a function result to bypass auto-escaping.
I voted up Dave's answer since I thought he did a good job.
I'd like to add however that there are also several different tools designed for ColdFusion that can simplify a lot of the common SQL tasks you're likely to perform. There's a very light-weight tool called DataMgr written by Steve Bryant, as well as Transfer from Mark Mandel, Reactor which was originally created by Doug Hughes and one I developed called DataFaucet. Each of these has its own strengths and weaknesses. Personally I think you're apt to consider DataFaucet to be the one that will give you the best ability to stay in cfscript, with a variety of syntaxes for building different kinds of queries.
Here are a few examples:
qry = datasource.select_avg_price_as_avgprice_from_products(); //(requires CF8)
qry = datasource.select("avg(price) as avgprice","products");
qry = datasource.getSelect("avg(price) as avgprice","products").filter("categoryid",url.categoryid).execute();
qry = datasource.getSelect(table="products",orderby="productname").filter("categoryid",url.categoryid).execute();
The framework ensures that cfqueryparam is always used with these filter statements to prevent sql-injection attacks, and there are similar syntaxes for insert, update and delete statements. (There are a couple of simple rules to avoid sql-injection.)
If I remove all the ' characters from a SQL query, is there some other way to do a SQL injection attack on the database?
How can it be done? Can anyone give me examples?
Yes, there is. An excerpt from Wikipedia
"SELECT * FROM data WHERE id = " + a_variable + ";"
It is clear from this statement that the author intended a_variable to be a number correlating to the "id" field. However, if it is in fact a string then the end user may manipulate the statement as they choose, thereby bypassing the need for escape characters. For example, setting a_variable to
1;DROP TABLE users
will drop (delete) the "users" table from the database, since the SQL would be rendered as follows:
SELECT * FROM DATA WHERE id=1;DROP TABLE users;
SQL injection is not a simple attack to fight. I would do very careful research if I were you.
Yes, depending on the statement you are using. You are better off protecting yourself either by using Stored Procedures, or at least parameterised queries.
See Wikipedia for prevention samples.
I suggest you pass the variables as parameters, and not build your own SQL. Otherwise there will allways be a way to do a SQL injection, in manners that we currently are unaware off.
The code you create is then something like:
' Not Tested
var sql = "SELECT * FROM data WHERE id = #id";
var cmd = new SqlCommand(sql, myConnection);
cmd.Parameters.AddWithValue("#id", request.getParameter("id"));
If you have a name like mine with an ' in it. It is very annoying that all '-characters are removed or marked as invalid.
You also might want to look at this Stackoverflow question about SQL Injections.
Yes, it is definitely possible.
If you have a form where you expect an integer to make your next SELECT statement, then you can enter anything similar:
SELECT * FROM thingy WHERE attributeID=
5 (good answer, no problem)
5; DROP table users; (bad, bad, bad...)
The following website details further classical SQL injection technics: SQL Injection cheat sheet.
Using parametrized queries or stored procedures is not any better. These are just pre-made queries using the passed parameters, which can be source of injection just as well. It is also described on this page: Attacking Stored Procedures in SQL.
Now, if you supress the simple quote, you prevent only a given set of attack. But not all of them.
As always, do not trust data coming from the outside. Filter them at these 3 levels:
Interface level for obvious stuff (a drop down select list is better than a free text field)
Logical level for checks related to data nature (int, string, length), permissions (can this type of data be used by this user at this page)...
Database access level (escape simple quote...).
Have fun and don't forget to check Wikipedia for answers.
Parameterized inline SQL or parameterized stored procedures is the best way to protect yourself. As others have pointed out, simply stripping/escaping the single quote character is not enough.
You will notice that I specifically talk about "parameterized" stored procedures. Simply using a stored procedure is not enough either if you revert to concatenating the procedure's passed parameters together. In other words, wrapping the exact same vulnerable SQL statement in a stored procedure does not make it any safer. You need to use parameters in your stored procedure just like you would with inline SQL.
Also- even if you do just look for the apostrophe, you don't want to remove it. You want to escape it. You do that by replacing every apostrophe with two apostrophes.
But parameterized queries/stored procedures are so much better.
Since this a relatively older question, I wont bother writing up a complete and comprehensive answer, since most aspects of that answer have been mentioned here by one poster or another.
I do find it necessary, however, to bring up another issue that was not touched on by anyone here - SQL Smuggling. In certain situations, it is possible to "smuggle" the quote character ' into your query even if you tried to remove it. In fact, this may be possible even if you used proper commands, parameters, Stored Procedures, etc.
Check out the full research paper at http://www.comsecglobal.com/FrameWork/Upload/SQL_Smuggling.pdf (disclosure, I was the primary researcher on this) or just google "SQL Smuggling".
. . . uh about 50000000 other ways
maybe somthing like 5; drop table employees; --
resulting sql may be something like:
select * from somewhere where number = 5; drop table employees; -- and sadfsf
(-- starts a comment)
Yes, absolutely: depending on your SQL dialect and such, there are many ways to achieve injection that do not use the apostrophe.
The only reliable defense against SQL injection attacks is using the parameterized SQL statement support offered by your database interface.
Rather that trying to figure out which characters to filter out, I'd stick to parametrized queries instead, and remove the problem entirely.
It depends on how you put together the query, but in essence yes.
For example, in Java if you were to do this (deliberately egregious example):
String query = "SELECT name_ from Customer WHERE ID = " + request.getParameter("id");
then there's a good chance you are opening yourself up to an injection attack.
Java has some useful tools to protect against these, such as PreparedStatements (where you pass in a string like "SELECT name_ from Customer WHERE ID = ?" and the JDBC layer handles escapes while replacing the ? tokens for you), but some other languages are not so helpful for this.
Thing is apostrophe's maybe genuine input and you have to escape them by doubling them up when you are using inline SQL in your code. What you are looking for is a regex pattern like:
\;.*--\
A semi colon used to prematurely end the genuine statement, some injected SQL followed by a double hyphen to comment out the trailing SQL from the original genuine statement. The hyphens may be omitted in the attack.
Therefore the answer is: No, simply removing apostrophes does not gaurantee you safety from SQL Injection.
I can only repeat what others have said. Parametrized SQL is the way to go. Sure, it is a bit of a pain in the butt coding it - but once you have done it once, then it isn't difficult to cut and paste that code, and making the modifications you need. We have a lot of .Net applications that allow web site visitors specify a whole range of search criteria, and the code builds the SQL Select statement on the fly - but everything that could have been entered by a user goes into a parameter.
When you are expecting a numeric parameter, you should always be validating the input to make sure it's numeric. Beyond helping to protect against injection, the validation step will make the app more user friendly.
If you ever receive id = "hello" when you expected id = 1044, it's always better to return a useful error to the user instead of letting the database return an error.