How to turn on REGEXP in SQLite3 and Rails 3.1? - sql

I have the following statement in Rails 3 using an SQLite3 database:
word = 'Hello'
word_entry = Word.where("name REGEXP :word", {:word => "[[:<:]]#{word}[[:>:]]"})
However, when running this under SQLite3, I keep getting:
SQLite3::SQLException: no such function: REGEXP
I read in the SQLite3 documentation that it does indeed support the REGEXP function.
In my gemfile, I have the line
gem 'sqlite3'
And my database config file looks like this:
development:
adapter: sqlite3
database: db/development.sqlite3
pool: 5
timeout: 5000
Any ideas what's going on?
RESOLUTION:
I ended up finding this solution. Unfortunately, it doesn't work for Rails 3.
So to use regular expressions I ended up switching to MYSQL instead of SQLite3.

I ran into the same issue. I took the code used in the resolution, ported it to work with Rails 3+ and made a gem for easier use. I hope this helps.
https://github.com/sei-mi/sqlite3_ar_regexp

From the fine manual:
The REGEXP operator is a special syntax for the regexp() user function. No regexp() user function is defined by default and so use of the REGEXP operator will normally result in an error message. If a application-defined SQL function named "regexp" is added at run-time, that function will be called in order to implement the REGEXP operator.
So the grammar supports REGEXP but the default SQLite library does not provide an implementation for it. You'll have to hook up your own implementation through some C wrangling if you want or need such a thing.
Presumably the rationale is that the SQLite people want to keep SQLite as small and tight as possible but including a whole regular expression library would add weight that most people don't want. Also, they would have to choose a regular expression library and include it with the SQLite source or they'd have to put up with the vagaries of everyone's regular expression support in libc. I'm not one of the SQLite developers so this is pure speculation.
I'm guessing that you'll probably have to make do with LIKE and GLOB. Using LIKE will provide a more portable solution.

You may be intested in the sqlite3-pcre package, which implements REGEXP for SQLite.
See this comment on a similar issue.

I had a similar question, and found a Gem named wherex that is well documented and worked out of the box.
Your expression from above
Word.where("name REGEXP :word", {:word => "[[:<:]]#{word}[[:>:]]"})
would there be
Word.where(:name => Regexp.new("[[:<:]]#{word}[[:>:]]"))
Works like a charm for me :-)

From source of sqlite3_ar_regexp project, I extract this:
db = SQLite3::Database.open( database_name )
db.create_function('regexp', 2) do |func, pattern, expression|
func.result = expression.to_s.match(
Regexp.new(pattern.to_s, Regexp::IGNORECASE)) ? 1 : 0
end

From source of sqlite3_ar_regexp project, I extract this:
db = ActiveRecord::Base.connection.raw_connection
db.create_function('regexp', 2) do |func, pattern, expression|
func.result = expression.to_s.match(
Regexp.new(pattern.to_s, Regexp::IGNORECASE)) ? 1 : 0
end
Improved upon a previous answer with ActiveRecord::Base.connection.raw_connection so that db name isn't needed

Related

Use unaccent postgres extension in Knex.js Querys

I need make a query for a postgresdb without identify accents (á, í,ö, etc).
I'm already use Knex.js as query builder, and postgresql have a unaccent extension that works fine in sql querys directly to db, but in my code i use knex and unaccent function throws error in querys.
Can anyone help me, ¿is possible make querys with knex.js that use unaccent function of postgresql?
My solution is to process the string before submitting the query using the following code:
const normalize = (str) => str.normalize('NFD').replace(/[\u0300-\u036f]/g, '');
console.log(normalize('Ấ Á Ắ Ạ Ê')) -> 'A A A A A'.
Or if you use postgresql version 13 or later it already supports that functionality.
select normalize('hồ, phố, ầ', NFC) → 'ho, pho, a' -- NFC (the default), NFD, NFKC, or NFKD.
Document: https://www.postgresql.org/docs/13/functions-string.html

Possible sql injection

I'm using squeel gem in my project, and I have code something like this :
def self.search(query)
return self.scoped if query.blank?
self.joins(:supplier).where{lower(supplier.supplier_name).like_any(["%#{query}%"])}
end
My questions is this code vulnerable to SQL injection? And how do I fix it? I tried to do sanitize(query) but it just adds extra set of quotes and the SQL statement doesn't get generated appropriately
UPDATED:
Squeel will automatically escape strings, so your query is fine and won't open you up to injection. See question about sql injection - Squeel - Github
OLD (INCORRECT) ANSWER:
This is the active record version
Someone correct me if i'm wrong, but since you are passing in #{query} as a STRING and not an argument, then you are opening yourself up to injection. See the docs for passing in arguments
Using arguments will escape the 'query' STRING
Your query using arguments:
self.joins(:supplier).where{lower(supplier.supplier_name).like_any(["%"+?+"%"], query)}

IFNULL in Symfony2 Doctrine query builder

How is the IFNULL of SQL implemented in Symfony2 Doctrine Query Builder?
Let's say I have this query:
select * from ticket order by IFNULL(modified_date, '2000-01-01') DESC, created_date DESC
I have this DQL:
$this->qb->select("t, c.name")
->from("Ticket", "t");
$this->qb->orderBy("t.modifiedDate", "DESC");
$this->qb->addOrderBy("t.createdDate", "DESC");
Now how to add the IFNULL part?
Ok, done some research and found that there is no such implementation.
Googled a little more, and got that this kind of missing features can be added to Doctrine as own functions.
Found this extension on GitHub I think this will work. But wonder if ther would be any problems or conflicts with Doctrine versions...
This is the valid link with the DQL Extension
Edit with the solution explained:
Create the following directory under your project src path: /src/DoctrineExtensions/Query/Mysql
Put there the DQL Extension file (IfNull.php in this case)
Edit your src/config/packages/doctrine.yaml and insert this new lines:
doctrine:
...
orm:
...
dql:
numeric_functions:
IFNULL: App\DoctrineExtensions\Query\Mysql\IfNull
In your entity repository you can call this function like this:
$qb = $this->createQueryBuilder('tl')
->andWhere('IFNULL(tl.app,0) = 1');
Depending on your usecase you may be able to use the builtin "COALESCE" expression instead of installing the "IFNULL" extension.
The usage then is basically the same as with the IFNULL expression.
Just replace IFNULL with COALESCE in the example in https://stackoverflow.com/a/68827681/1707003.
Note: COALESCE might behave slightly different than IFNULL depending on your database. https://stackoverflow.com/a/18528590/1707003 contains some great explanations.
List of builtin case-expressions: https://www.doctrine-project.org/projects/doctrine-orm/en/2.13/reference/dql-doctrine-query-language.html#case-expressions

"Searching" a string on a MongoDB DB with MongoMapper

I'm making a webapp where I'm using MongoMapper and Sinatra. I wonder how could I implement a search feature against a DB's collection. I though something like SQL's:
SELECT * FROM posts WHERE match(title) against ("String to search");
How could I achieve this with MongoMapper? Thanks!
ok this is from my project and does work:
Post.where(:title => Regexp.new(/^string/i)) # Limit output with: .limit(10)
Maybe it's the Regexp?
You query for documents that match a case sensitive rooted regular expression:
Post.where(:title => /^stringtosearch/).first
MongoDB does not support full text search so this is the best you can do at the moment.

Rails 3 ActiveRecord query using both SQL IN and SQL OR operators

I'm writing a Rails 3 ActiveRecord query using the "where" syntax, that uses both the SQL IN and the SQL OR operator and can't figure out how to use both of them together.
This code works (in my User model):
Question.where(:user_id => self.friends.ids)
#note: self.friends.ids returns an array of integers
but this code
Question.where(:user_id => self.friends.ids OR :target => self.friends.usernames)
returns this error
syntax error, unexpected tCONSTANT, expecting ')'
...user_id => self.friends.ids OR :target => self.friends.usern...
Any idea how to write this in Rails, or just what the raw SQL query should be?
You don't need to use raw SQL, just provide the pattern as a string, and add named parameters:
Question.where('user_id in (:ids) or target in (:usernames)',
:ids => self.friends.ids, :usernames => self.friends.usernames)
Or positional parameters:
Question.where('user_id in (?) or target in (?)',
self.friends.ids, self.friends.usernames)
You can also use the excellent Squeel gem, as #erroric pointed out on his answer (the my { } block is only needed if you need access to self or instance variables):
Question.where { user_id.in(my { self.friends.ids }) |
target.in(my { self.friends.usernames }) }
Though Rails 3 AR doesn't give you an or operator you can still achieve the same result without going all the way down to SQL and use Arel directly. By that I mean that you can do it like this:
t = Question.arel_table
Question.where(t[:user_id].in(self.friends.ids).or(t[:username].in(self.friends.usernames)))
Some might say it ain't so pretty, some might say it's pretty simply because it includes no SQL. Anyhow it most certainly could be prettier and there's a gem for it too: MetaWhere
For more info see this railscast: http://railscasts.com/episodes/215-advanced-queries-in-rails-3
and MetaWhere site: http://metautonomo.us/projects/metawhere/
UPDATE: Later Ryan Bates has made another railscast about metawhere and metasearch: http://railscasts.com/episodes/251-metawhere-metasearch
Later though Metawhere (and search) have become more or less legacy gems. I.e. they don't even work with Rails 3.1. The author felt they (Metawhere and search) needed drastic rewrite. So much that he actually went for a new gem all together. The successor of Metawhere is Squeel. Read more about the authors announcement here:
http://erniemiller.org/2011/08/31/rails-3-1-and-the-future-of-metawhere-and-metasearch/
and check out the project home page:
http://erniemiller.org/projects/squeel/
"Metasearch 2.0" is called Ransack and you can read something about it from here:
http://erniemiller.org/2011/04/01/ransack-the-library-formerly-known-as-metasearch-2-0/
Alternatively, you could use Squeel. To my eyes, it is simpler. You can accomplish both the IN (>>) and OR (|) operations using the following syntax:
Question.where{(:user_id >> my{friends.id}) | (:target >> my{friends.usernames})}
I generally wrap my conditions in (...) to ensure the appropriate order of operation - both the INs happen before the OR.
The my{...} block executes methods from the self context as defined before the Squeel call - in this case Question. Inside of the Squeel block, self refers to a Squeel object and not the Question object (see the Squeel Readme for more). You get around this by using the my{...} wrapper to restore the original context.
raw SQL
SELECT *
FROM table
WHERE user_id in (LIST OF friend.ids) OR target in (LIST OF friends.usernames)
with each list comma separate. I don't know the Rails ActiveRecord stuff that well. For AND you would just put a comma between those two conditions, but idk about OR