I have been searching for a way of using Django ORM to use the SQL underscore wildcard, and do something equivalent to this:
SELECT * FROM table
WHERE field LIKE 'abc_wxyz'
Currently, I am doing:
field_like = 'abc_wxyz'
result = MyClass.objects.extra(where=["field LIKE " + field_like])
I already tried with contains() and icontains(), but that's not what I need, since what it does is adding parenthesis to the query:
SELECT * FROM table
WHERE field LIKE '%abc/_wxyz%'
Thanks!
You can use __regex lookup to build more complex lookup expressions than __contains, __startswith or __endswith (can add "i" character to beginning of each of these to make lookups case insensitive, like icontains). In your case, I think
MyClass.objects.filter(field__regex=r'^abc.wxyz$')
Would do what you are trying to do.
You can use the field__contains attribute.
for example:
MyClass.objects.filter(field__contains='abc_wxyz')
This is equivalent to:
SELECT * FROM MyClass WHERE field LIKE 'abc_wxyz'
Lord Elron's answer is incorrect. Django escapes all developer supplied wildcard characters to the LIKE-type lookups. The statement is equivalent to
SELECT * FROM MyClass WHERE field LIKE '%abc/_wxyz%'
(as the OP discovered) and the underscore has no effect.
See Escaping percent signs and underscores in LIKE statements
The field lookups that equate to LIKE SQL statements (iexact, contains, icontains, startswith, istartswith, endswith and iendswith) will automatically escape the two special characters used in LIKE statements – the percent sign and the underscore.
Related
How would I compare a file name with a naming convention thats saved as a windows wildcard pattern in a SQL column? For example:
I have a file HCLA_MCLA_20220308 and I want to check if there is a wildcard naming convention that matches this file name in my File_Naming_Convention column of my table. The match should be for "*HCLA_MCLA_**"
HCLA naming convention
All the naming conventions either use * or ?. The stars mean any number of characters before and/or any number or characters after. The ? means only 1 wild character in that position. Im not sure why there are two ** at the end of files in some cases.
My wildcard file naming convention column looks like this:
Section of column
Now I tried a query like below but this doesn't seem to be working. It appears that the wildcard tokens in the SQL table are windows wildcards. In my query I tried to replace them with SQL equivalents.
Select *
from MPM_FTP_Eligibility_Files_List
where replace(File_Naming_Convention,'*','%') like 'HCLA_MCLA_20220308'
Update:
Here is the fix to take care of both * and ?. Needed to swap positions between my file name and like pattern in my query, as jarlh accurately pointed out.
Select * from MPM_FTP_Eligibility_Files_List where 'FCS-CA25_1_20220301_1000N_Cigna_W_Trans.txt' like replace(replace(File_Naming_Convention,'*','%'),'?','_')
I have a query which is supposed to find matching rows ignoring case and special characters that may be present both in the query and the corresponding column. For that I use REGEXP_REPLACE like this:
SELECT *
FROM Order
WHERE REGEXP_REPLACE(reference, '[^a-zA-Z0-9äöüÄÖÜ]', '') LIKE %:search%
where search is the name of the parameter I want to use. That works, but doesn't yet sanitize the search parameter from unwanted special characters.
What I would like to do is something like the following, i.e. having the REGEXP_REPLACE on the right side as well:
SELECT *
FROM Order
WHERE REGEXP_REPLACE(reference, '[^a-zA-Z0-9äöüÄÖÜ]', '') LIKE %REGEXP_REPLACE(:search, '[^a-zA-Z0-9äöüÄÖÜ]', '')%
However that doesn't work and I get the following error:
42000][1064] You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '%REGEXP_REPLACE(
Is it not possible to use a function on the parameter or as part of a LIKE statement? Are there any workarounds?
It looks like you want to create a string starting and ending with '%' to use in your LIKE operator. To do that in MySQL's dialect of SQL you need to do your string manipulation explicitly using the built-in string manipulation functions.
You can use those functions anywhere your query needs a text string.
Try using CONCAT in an expression like this to generate that string. You'll be able to use it on the right side of your LIKE.
CONCAT('%', REGEXP_REPLACE(:search, '[^a-zA-Z0-9äöüÄÖÜ]', ''), '%')
I hope you don't want your query to be fast. It will be slow. It must examine every value of Order.reference in your table. It's slow because
it's not sargable due to WHERE f(column) LIKE whatever, and
column LIKE '%something%' requires looking at every value of column, rather than random-acccessing a BTREE index.
If you build a database to scale up, you design it so your queries can be sargeable. Sargability here might look like
WHERE cleaned_up_reference
LIKE CONCAT(REGEXP_REPLACE(:search, '[^a-zA-Z0-9äöüÄÖÜ]', ''), '%')
without the leading % on the right, and without evaluating any function on the column or columns being searched.
You can try this:
SELECT * FROM Order a
WHERE REGEXP_REPLACE(a.reference, '[^a-zA-Z0-9äöüÄÖÜ]', '') LIKE '%:search%'
Can I get an explanation to me how the "?" prevents sql injection?
Candy.where("taste ILIKE ?", "%#{term}%")
Also why is "%{term}%" used as opposed to just #{term}? What do the percentages represent?
Percentages are wild card characters that matches the string in any part of the value
You've actually asked two different questions there.
Neither of them are particularly related to Rails, so I am going to answer them generically (also because I'm not that familiar with Ruby!).
How does using '?' prevent SQL Injection
SQL Injection occurs when you use values provided from outside your program - user provided values - directly in SQL statements. For example, suppose you had this pseudo code:
sql="SELECT foo FROM bar WHERE name='"+name+"'"
where perhaps name was a variable containing user inputted data. However, if name contained a single quote (') then the SQL engine would think that single quote was the end of the value and continue parsing the remainder of the variable as SQL text.
Using placeholders (such as '?') avoid this because the value inside the placeholder does not need to be quoted - all of the content of the placeholder is treated as part of the value, none of it will be parsed as SQL, regardless of any embedded quotes.
Incidentally, the actual form of the placeholders used is somewhat dependent on the actual DB engine used and/or the client framework. Natively, Postgresql uses the $1, $2, etc for placeholders. Many frameworks extend this to allow '?' and other placeholder syntaxes.
Why is "%#{term}%" used as opposed to just #{term}
The SQL ILIKE operator uses '%' signs as wildcards. An expression such as:
taste ILIKE '%apple%'
would match 'apple', 'rottenApple', 'applesauce' or any other string containing 'apple' using a case insensitive match.
Note that the '%' signs are part of the right hand operand to ILIKE, within the quotes, so you cannot use placeholders like this:
Candy.where("taste ILIKE %?%", "#{term}")
An alternative would be:
Candy.where("taste ILIKE '%' || ? || '%'", "#{term}")
That works because || is the concatenation operator, so it concatenates the literal value % with the value of the placeholder and then the trailing literal value %.
When you use ? Rails will escape SQL control characters by itself.
How am I going to use BETWEEN Operator with Text Value or what is the right syntax when you will select all products with a ProductName for example ending with any of the letter BETWEEN 'C' and 'M'?
Most SQL dialects provide the RIGHT() function. This allows you to do:
WHERE RIGHT(TextValue, 1) BETWEEN 'C' AND 'M'
If your database doesn't have this function, you can do something similar with the built-in functions. Also, the exact comparison might depend on the collation of the column/table/database/server. Sometimes comparisons are independent of case and sometimes they are dependent on case.
In case you are interested in an alternative method (which does work with the w3schools SQL editor), you can also use the LIKE operator:
WHERE ProductName LIKE '%[c-m]'
This will get you all Product Names ending on any character between C and M.
(It does work with the w3schools SQL Editor.)
In this case, the LIKE operator is using two wildcard characters:
1.%
Any string of zero or more characters.
2.[c-m]
Any single character within the specified range ([a-f]) or set
([abcdef]).
You can find more information about the LIKE operator here:
https://msdn.microsoft.com/en-us/library/ms179859.aspx
Trying to select from table where the format can be either 1/2/2014, 1-2-2014 or 1_2_2014 in a text field. There's other text involved outside of this format but it shouldn't matter, but that's why this is text not a date type.
I tried '%[-,_,/]%[-,_,/]%' which doesn't work, and I've tried escaping the special characters in the brackets such as %[-,!_,/]%[-,!_,/]%' ESCAPE '!' which also doesn't work. Any suggestions?
I wanted to avoid using three searches like,
LIKE '%/%/%'
OR '%-%-%'
OR '%!_%!_%' ESCAPE '!'
EDIT: Using SQLite3
There is no regex like behavior in using the LIKE operator in SQL. You would have use two expressions and OR them together:
select * from table
where column like '%-%-%'
or column like '%/%/%'
Thanks for the information. I ended up switching to the GLOB operator which support [] in SQLite.
The Example was altered to GLOB '?[/-_]?[/-_]??*' Where * serves as % and ? serves as _ for the GLOB function.
Also thanks to Amadeaus9 for pointing out minimum characters between delimiters so that '//' isn't a valid answer.
If you're using T-SQL (AKA SQL Server) you don't want to have commas in the character set - i.e. LIKE '%[/_-]%[/_-]%'. However, keep in mind that this can match ANYTHING that has, anywhere within it, any two characters from the set.
EDIT: it doesn't looke like SQLite supports that sort of use of its LIKE operator, based on this link.
Relevant quote:
There are two wildcards used in conjunction with the LIKE operator:
The percent sign (%)
The underscore (_)
However, you may want to take a look at this question, which details using regex in SQLite.
It is not possible using the LIKE syntax.
However Sqlite3 would support the REGEXP operator; this is syntactic sugar for calling an user defined function that actually does the matching. If provided by your platform, then you could use for example
x REGEXP '.*[/_-].*[/_-].*'