I'm trying to speed up a query that uses Contains Near with one that uses regexp_like. The initial Contains Near query takes about 45 minutes to run. Clob Column holds large "documents" and is domain indexed.
Initial query:
SELECT column1
FROM TEST
WHERE CONTAINS(column1,'{NEAR(quick,fox, lazy), 3, FALSE}')>0;
Proposed query:
SELECT column1
FROM TEST
WHERE REGEXP_LIKE(column1, '(\b(quick|fox|lazy)(?:\W+\w+){1,6}?\W(quick|fox|lazy)(?:\W+\w+){1,}?\W(quick|fox|lazy)\b)','i')
I got the original regexp syntax from here:
https://www.regular-expressions.info/near.html.
Problem:
I get the regexp code to work in html https://www.regextester.com, but when I put it in Oracle it doesn't find anything. What is wrong with my syntax? I can't figure it out. Does Oracle handle REGEXP differently?
Alex, you were exactly right. I don't see how to select your answer as correct though.
My problem was apparently that I was using regexp parameters that Oracle doesn't recognize. So, whereas it worked on https://www.regextester.com, it failed to work in Oracle because most of what I used isn't recognized as usable with regexp in Oracle. I really think Oracle should expand their regexp codes it recognized. This was really frustrating.
Related
This Statement:
SELECT id, units, cost FROM inventory_list WHERE cost <= 20;
Gives me:
ORA-00923: FROM keyword not found where expected
While this statement:
SELECT * FROM items WHERE ilt_id = 'il010230126' OR ilt_id = 'il010230128';
Gives me:
ORA-00933: SQL command not properly ended
Not sure about this and may be version dependent (below link is for oracle 10g... but you can see on this site
https://docs.oracle.com/cd/B19306_01/em.102/b40103/app_oracle_reserved_words.htm
That cost is an oracle reserved keyword, so it is not wise to use it as a column name.
If you have no control of the table I think you may be able to surround it in double quotes eg select "COST" to avoid oracle picking it up as a reserved word.
By default Oracle creates fields in uppercase so the field name will need to be in uppercase unless when the table was created it was forced into different case by surrounding it in Quotes.
Check that you don't have invisible characters in your file and that you're using the right encoding. I sometimes accidentally introduce them because I have a non english keyboard map and accidentally hit the wrong key combination.
Just type again one of your SQL statements and test them.
I have a HQL(Hive Query) file which has code like
select * ,'(submit_date)?+.+' from test
Table test has several other filed after submit date all of which are returned in the output of this query, but i couldn't understand how this thing works. ?
does any1 have any idea, i couldn't find any Doc related to this syntax
This is documented as the REGEX column specification:
A SELECT statement can take regex-based column specification.
We use java regex syntax. Try http://www.fileformat.info/tool/regex.htm for testing purposes.
The following query select all columns except ds and hr.
SELECT `(ds|hr)?+.+` FROM sales
So many times seen with and, so many times SQL Server ask that with has ; before it
How does ;with ... work??
;with coords(...) as (
SELECT * ...
)
Why must have ; before it?
The semicolon is used in SQL to end a query. Putting it before a query like that is just to make sure that the database understands that any previous query has ended.
Originally it was required after each query as they were entered line by line, so the database had to know when to run the query. When the entire query is sent in a single string, you only need semicolons in the case where the SQL syntax is not enough to determine where a query ends. As the with keyword has different uses a semicolon is sometimes needed before it to make sure that it's not part of the previous query.
Using WITH for CTEs requires the previous statement to be terminated with ;. Using it at the start like this guarantees correct syntax
So does MERGE in SQL Server 2008
See this SO question: Incorrect syntax near the keyword 'with'...previous statement must be terminated with a semicolon
It is best practise to terminate every SQL statement with a semicolon. The SQL Server docs (for example here) suggest doing so will be mandated in a future version to there's really no excuse for not getting into the habit now.
To answer the question: you see ;WITH... on Stackoverflow because EITHER the person answering is a sloppy coder OR the person answering assumes the person asking the question is a sloppy coder (and they'll claim it is the latter when it is the former :) The definition of "sloppy coder" here is someone who only uses a semicolon when they are forced to do so.
The use of WITH is for common table expressions (CTEs). They were trying to force the CTE to be defined as the first statement (i.e. cannot be linked with other parts of the query hence the ;)
The website i worked was recently attempted to be hacked by the following SQL injection script
boys' and 3=8 union
select 1,
concat(0x232425,ifnull(`table_name`,0x30),char(9),ifnull(`table_rows`,0x30), char(9),0x252423),
3,4,5,6,7,8,9
from `information_schema`.`tables`
where table_schema=0x62646B3032 limit 44,1 -- And '8'='8
This injection returned the mysql table name. This was reported by the error reporting system on that website and we managed to fix that part however I am not able to understand what does the above injection mean?
Anyone can explain this?
Penuel
They're using a select from the Information Schema views in mysql server :
http://dev.mysql.com/doc/refman/5.0/en/information-schema.html
They use some clever hacks to rout out simple sql injection prevention techniques.
According to this the MySQL concat()
Returns the string that results from
concatenating the arguments. May have
one or more arguments. If all
arguments are nonbinary strings, the
result is a nonbinary string. If the
arguments include any binary strings,
the result is a binary string. A
numeric argument is converted to its
equivalent binary string form
So 0x232425 is converted to #$% which is simply added to the begining and end of the table_name field. Maybe just to make it easier for them to pull out the Table names later using Regex.
Later on the char(9) is equivalent to a tab as you can see here and is just there to format the output nicer.
The 3,4,5,6,7,8,9 is just there so that the columns match the boys table that they are performing the Union on.
This injection returned the mysql table name.
Do you mean that your website displayed the table name when you gave it this input, or that the query returns that when run from the mysql client? If it showed on your website, then the attacker has the ability to inject much more harmful queries. Check your data.
This question already has answers here:
How to anticipate and escape single quote ' in oracle
(2 answers)
Closed 7 years ago.
I have a database with names in it such as John Doe etc. Unfortunately some of these names contain quotes like Keiran O'Keefe. Now when I try and search for such names as follows:
SELECT * FROM PEOPLE WHERE SURNAME='O'Keefe'
I (understandably) get an error.
How do I prevent this error from occurring. I am using Oracle and PLSQL.
The escape character is ', so you would need to replace the quote with two quotes.
For example,
SELECT * FROM PEOPLE WHERE SURNAME='O'Keefe'
becomes
SELECT * FROM PEOPLE WHERE SURNAME='O''Keefe'
That said, it's probably incorrect to do this yourself. Your language may have a function to escape strings for use in SQL, but an even better option is to use parameters. Usually this works as follows.
Your SQL command would be :
SELECT * FROM PEOPLE WHERE SURNAME=?
Then, when you execute it, you pass in "O'Keefe" as a parameter.
Because the SQL is parsed before the parameter value is set, there's no way for the parameter value to alter the structure of the SQL (and it's even a little faster if you want to run the same statement several times with different parameters).
I should also point out that, while your example just causes an error, you open youself up to a lot of other problems by not escaping strings appropriately. See http://en.wikipedia.org/wiki/SQL_injection for a good starting point or the following classic xkcd comic.
Oracle 10 solution is
SELECT * FROM PEOPLE WHERE SURNAME=q'{O'Keefe}'
Parameterized queries are your friend, as suggested by Matt.
Command = SELECT * FROM PEOPLE WHERE SURNAME=?
They will protect you from headaches involved with
Strings with quotes
Querying using dates
SQL Injection
Use of parameterized SQL has other benefits, it reduces CPU overhead (as well as other resources) in Oracle by reducing the amount of work Oracle requires in order to parse the statement. If you do not use parameters (we call them bind variables in Oracle) then "select * from foo where bar='cat'" and "select * from foo where bar='dog'" are treated as separate statements, where as "select * from foo where bar=:b1" is the same statement, meaning things like syntax, validity of objects that are referenced etc...do not need to be checked again. There are occasional problems that arise when using bind variables which usually manifests itself in not getting the most efficient SQL execution plan but there are workarounds for this and these problems really depend on the predicates you are using, indexing and data skew.
Input filtering is usually done on the language level rather than database layers.
php and .NET both have their respective libraries for escaping sql statements. Check your language, see waht's available.
If your data are trustable, then you can just do a string replace to add another ' infront of the ' to escape it. Usually that is enough if there isn't any risks that the input is malicious.
I suppose a good question is what language are you using?
In PHP you would do: SELECT * FROM PEOPLE WHERE SURNAME='mysql_escape_string(O'Keefe)'
But since you didn't specify the language I will suggest that you look into a escape string function mysql or otherwise in your language.
To deal quotes if you're using Zend Framework here is the code
$db = Zend_Db_Table_Abstract::getDefaultAdapter();
$db->quoteInto('your_query_here = ?','your_value_here');
for example ;
//SELECT * FROM PEOPLE WHERE SURNAME='O'Keefe' will become
SELECT * FROM PEOPLE WHERE SURNAME='\'O\'Keefe\''
Found in under 30s on Google...
Oracle SQL FAQ