Here is the original logic
(scrape_datas = ScrapeData.find(
:all, :conditions =>
"artist_status = 'NOT_FOUND'
AND blacklisted = 1
AND extracted = 0
and not EXISTS(
SELECT * FROM artist_name_suggestions where original = artist_name
)
I've been able to split up the first part better
scrape_datas = ScrapeData.where(
:artist_status => 'NOT_FOUND',
:blacklisted => 1,
:extracted => 0
)
Although having issues getting the "and not EXISTS" query into the mix
and not EXISTS(
SELECT * FROM artist_name_suggestions where original = artist_name
)
Thanks!
Firstly you can extract simple scopes:
scope :not_found, where(:artist_status => 'NOT_FOUND')
scope :blacklisted, where(:blacklisted => 1)
scope :extracted, where(:extracted => 0)
Then add a query method (assume artist_name is a column of scrape_datas):
def self.no_suggestions
scrape_datas = ScrapeData.arel_table
suggestions = ArtistNameSuggestion.arel_table
where(ArtistNameSuggestion.where(
suggestions[:original].eq(scrape_datas[:artist_name])
).exists.not)
end
Now you can do something like this:
ScrapeData.not_found.blacklisted.extracted.no_suggestions
Related
I'm new to zend 1.12 and I want to construct the following query, please advice:
SELECT tbl_user.first_name, tbl_user.last_name, tbl_user.email, tbl_user_group.group_id
FROM `tbl_user`, `tbl_user_group`
WHERE `organization_id` = 5
AND tbl_user.user_id = tbl_user_group.user_id
AND tbl_user_group.group_id = 11
Try this (not tested):
$select = $db->select()
->from(array('tu' => 'tbl_user'),
array('first_name', 'last_name', 'email'))
->join(array('tug' => 'tbl_user_group'),
'tu.user_id = tug.user_id',
array('group_id'))
->where('tu.organization_id = ?', 5)
->where('tug.group_id = ?', 11);
I am trying to do this using SQL in Ruby:
SELECT number*2 FROM dictionary;
I can use .select(:number) to get the number columns "as is", but I have no idea how to return them modified. I tried this (which obviously didn't work):
current_table.select(:number*2)
Try either of these
#newnumber = number.find(:all, :conditions => ["id = ?", #dictionary.id], :select => "number * 2 as newnumber")
#newnumber = number.all(:conditions=>"id='#{#dictionary.id}'",:select=>"number * 2 as newnumber")
Basically I crossed the same problem of Linq provider in this linq-to-nhibernate-produces-unnecessary-joins
List<Competitions> dtoCompetitions;
dtoCompetitions = (from compset in session.Query<FWBCompetitionSet>()
where compset.HeadLine == true
&& compset.A.B.CurrentSeason == true
select (new Competitions
{
CompetitionSetID = compset.CompetitionSetID,
Name = compset.Name,
Description = compset.Description,
Area = compset.Area,
Type = compset.Type,
CurrentSeason = compset.A.B.CurrentSeason,
StartDate = compset.StartDate
}
)).ToList();
Which leads to duplicated join in its generated SQL
SELECT fwbcompeti0_.competitionsetid AS col_0_0_,
fwbcompeti0_.name AS col_1_0_,
fwbcompeti0_.DESCRIPTION AS col_2_0_,
fwbcompeti0_.area AS col_3_0_,
fwbcompeti0_.TYPE AS col_4_0_,
fwbseason3_.currentseason AS col_5_0_,
fwbcompeti0_.startdate AS col_6_0_
FROM fwbcompetitionset fwbcompeti0_
INNER JOIN A fwbcompeti1_
ON fwbcompeti0_.competitionseasonid = fwbcompeti1_.competitionseasonid
INNER JOIN A fwbcompeti2_
ON fwbcompeti0_.competitionseasonid = fwbcompeti2_.competitionseasonid
INNER JOIN B fwbseason3_
ON fwbcompeti2_.seasonid = fwbseason3_.seasonid
WHERE fwbcompeti0_.headline = #p0
AND fwbseason3_.currentseason = #p1
Notice these joins, which are totally duplicated and also affect my SQL Server's performence.
INNER JOIN A fwbcompeti1_
ON fwbcompeti0_.competitionseasonid = fwbcompeti1_.competitionseasonid
INNER JOIN A fwbcompeti2_
ON fwbcompeti0_.competitionseasonid = fwbcompeti2_.competitionseasonid
Update1
In the NHibernate 3.2, this LiNQ bug is still valid, and I could not find a simple and reasonable Linq solution.
So I used QueryOver + JoinAlias + TransformUsing finishing the job, workds perfect to me.
FWBCompetitionSet compset = null;
FWBCompetitionSeason compseason = null;
FWBSeason season = null;
IList<Competitions> dtoCompetitions;
dtoCompetitions = session.QueryOver<FWBCompetitionSet>(() => compset)
.JoinAlias(() => compset.FWBCompetitionSeason, () => compseason)
.JoinAlias(() => compseason.FWBSeason, () => season)
.Where(() => compset.HeadLine == true)
.And(() => season.CurrentSeason == true)
.SelectList(
list => list
.Select(c => c.CompetitionSetID).WithAlias(() => compset.CompetitionSetID)
.Select(c => c.Name).WithAlias(() => compset.Name)
.Select(c => c.Description).WithAlias(() => compset.Description)
.Select(c => c.Area).WithAlias(() => compset.Area)
.Select(c => c.Type).WithAlias(() => compset.Type)
.Select(c => season.CurrentSeason).WithAlias(() => season.CurrentSeason)
.Select(c => c.StartDate).WithAlias(() => compset.StartDate)
)
.TransformUsing(Transformers.AliasToBean<Competitions>())
.List<Competitions>();
Yet Another Edit:
I think I finally found out what's going on. It seems that the LINQ to NHibernate provider has trouble navigating associations from the target to the source table and generates a separate join each time it encounters such an association.
Since you don't provide your mapping, I used the mapping from linq-to-nhibernate-produces-unnecessary-joins. This model has a Document with one Job and many TranslationUnits. Each TranslationUnit has many Translation entities.
When you try to find a Translation based on a Job, you are traversing the associations in the reverse order and the LINQ provider generates multiple joins: one for Translation -> TranslationUnit and one for TranslationUnit to Document.
This query will generate redundant joins:
session.Query<TmTranslation>()
.Where(x => x.TranslationUnit.Document.Job == job)
.OrderBy(x => x.Id)
.ToList();
If you reverse the navigation order to Document -> TranslationUnit -> Translation, you get a query that doesn't produce any redundant joins:
var items=(from doc in session.Query<Document>()
from tu in doc.TranslationUnits
from translation in tu.Translations
where doc.Job ==job
orderby translation.Id
select translation).ToList();
Given this quirkiness, QueryOver seems like a better option.
Previous Edit:
I suspect the culprit is compset.A.B.CurrentSeason. The first joined table (fwbcompeti1_) returns A.B while the next two (fwbcompeti2_ and fwbseason3_) are used to return A.B. The LINQ to NHibernate provider doesn't seem to guess that A is not used anywhere else and fails to remove it from the generated statement.
Try to help the optimizer a little by replacing CurrentSeason = compset.A.B.CurrentSeason with CurrentSeason = true from the select, since your where statement returns only items with CurrentSeason == true.
EDIT: What I mean is to change the query like this:
List<Competitions> dtoCompetitions;
dtoCompetitions = (from compset in session.Query<FWBCompetitionSet>()
where compset.HeadLine == true
&& compset.A.B.CurrentSeason == true
select (new Competitions
{
CompetitionSetID = compset.CompetitionSetID,
Name = compset.Name,
Description = compset.Description,
Area = compset.Area,
Type = compset.Type,
CurrentSeason = true,
StartDate = compset.StartDate
}
)).ToList();
I simply replace the value compset.A.B.CurrentSeason with true
How can you combine 2 different conditions using logical OR instead of AND?
NOTE: 2 conditions are generated as rails scopes and can't be easily changed into something like where("x or y") directly.
Simple example:
admins = User.where(:kind => :admin)
authors = User.where(:kind => :author)
It's easy to apply AND condition (which for this particular case is meaningless):
(admins.merge authors).to_sql
#=> select ... from ... where kind = 'admin' AND kind = 'author'
But how can you produce the following query having 2 different Arel relations already available?
#=> select ... from ... where kind = 'admin' OR kind = 'author'
It seems (according to Arel readme):
The OR operator is not yet supported
But I hope it doesn't apply here and expect to write something like:
(admins.or authors).to_sql
ActiveRecord queries are ActiveRecord::Relation objects (which maddeningly do not support or), not Arel objects (which do).
[ UPDATE: as of Rails 5, "or" is supported in ActiveRecord::Relation; see https://stackoverflow.com/a/33248299/190135 ]
But luckily, their where method accepts ARel query objects. So if User < ActiveRecord::Base...
users = User.arel_table
query = User.where(users[:kind].eq('admin').or(users[:kind].eq('author')))
query.to_sql now shows the reassuring:
SELECT "users".* FROM "users" WHERE (("users"."kind" = 'admin' OR "users"."kind" = 'author'))
For clarity, you could extract some temporary partial-query variables:
users = User.arel_table
admin = users[:kind].eq('admin')
author = users[:kind].eq('author')
query = User.where(admin.or(author))
And naturally, once you have the query you can use query.all to execute the actual database call.
I'm a little late to the party, but here's the best suggestion I could come up with:
admins = User.where(:kind => :admin)
authors = User.where(:kind => :author)
admins = admins.where_values.reduce(:and)
authors = authors.where_values.reduce(:and)
User.where(admins.or(authors)).to_sql
# => "SELECT \"users\".* FROM \"users\" WHERE ((\"users\".\"kind\" = 'admin' OR \"users\".\"kind\" = 'author'))"
As of Rails 5 we have ActiveRecord::Relation#or, allowing you to do this:
User.where(kind: :author).or(User.where(kind: :admin))
...which gets translated into the sql you'd expect:
>> puts User.where(kind: :author).or(User.where(kind: :admin)).to_sql
SELECT "users".* FROM "users" WHERE ("users"."kind" = 'author' OR "users"."kind" = 'admin')
From the actual arel page:
The OR operator works like this:
users.where(users[:name].eq('bob').or(users[:age].lt(25)))
I've hit the same problem looking for an activerecord alternative to mongoid's #any_of.
#jswanner answer is good, but will only work if the where parameters are a Hash :
> User.where( email: 'foo', first_name: 'bar' ).where_values.reduce( :and ).method( :or )
=> #<Method: Arel::Nodes::And(Arel::Nodes::Node)#or>
> User.where( "email = 'foo' and first_name = 'bar'" ).where_values.reduce( :and ).method( :or )
NameError: undefined method `or' for class `String'
To be able to use both strings and hashes, you can use this :
q1 = User.where( "email = 'foo'" )
q2 = User.where( email: 'bar' )
User.where( q1.arel.constraints.reduce( :and ).or( q2.arel.constraints.reduce( :and ) ) )
Indeed, that's ugly, and you don't want to use that on a daily basis. Here is some #any_of implementation I've made : https://gist.github.com/oelmekki/5396826
It let do that :
> q1 = User.where( email: 'foo1' ); true
=> true
> q2 = User.where( "email = 'bar1'" ); true
=> true
> User.any_of( q1, q2, { email: 'foo2' }, "email = 'bar2'" )
User Load (1.2ms) SELECT "users".* FROM "users" WHERE (((("users"."email" = 'foo1' OR (email = 'bar1')) OR "users"."email" = 'foo2') OR (email = 'bar2')))
Edit : since then, I've published a gem to help building OR queries.
Just make a scope for your OR condition:
scope :author_or_admin, where(['kind = ? OR kind = ?', 'Author', 'Admin'])
Using SmartTuple it's going to look something like this:
tup = SmartTuple.new(" OR ")
tup << {:kind => "admin"}
tup << {:kind => "author"}
User.where(tup.compile)
OR
User.where((SmartTuple.new(" OR ") + {:kind => "admin"} + {:kind => "author"}).compile)
You may think I'm biased, but I still consider traditional data structure operations being far more clear and convenient than method chaining in this particular case.
To extend jswanner answer (which is actually awesome solution and helped me) for googling people:
you can apply scope like this
scope :with_owner_ids_or_global, lambda{ |owner_class, *ids|
with_ids = where(owner_id: ids.flatten).where_values.reduce(:and)
with_glob = where(owner_id: nil).where_values.reduce(:and)
where(owner_type: owner_class.model_name).where(with_ids.or( with_glob ))
}
User.with_owner_ids_or_global(Developer, 1, 2)
# => ...WHERE `users`.`owner_type` = 'Developer' AND ((`users`.`owner_id` IN (1, 2) OR `users`.`owner_id` IS NULL))
What about this approach: http://guides.rubyonrails.org/active_record_querying.html#hash-conditions (and check 2.3.3)
admins_or_authors = User.where(:kind => [:admin, :author])
Unfortunately it is not supported natively, so we need to hack here.
And the hack looks like this, which is pretty inefficient SQL (hope DBAs are not looking at it :-) ):
admins = User.where(:kind => :admin)
authors = User.where(:kind => :author)
both = User.where("users.id in (#{admins.select(:id)}) OR users.id in (#{authors.select(:id)})")
both.to_sql # => where users.id in (select id from...) OR users.id in (select id from)
This generates subselets.
And a little better hack (from SQL perspective) looks like this:
admins_sql = admins.arel.where_sql.sub(/^WHERE/i,'')
authors_sql = authors.arel.where_sql.sub(/^WHERE/i,'')
both = User.where("(#{admins_sql}) OR (#{authors_sql})")
both.to_sql # => where <admins where conditions> OR <authors where conditions>
This generates proper OR condition, but obviously it only takes into account the WHERE part of the scopes.
I chose the 1st one until I'll see how it performs.
In any case, you must be pretty careful with it and watch the SQL generated.
I got this figured out. Here is the solution:
'conditions'=>array(
'OR' => array(
array('EavAttribute.attribute_code'=>'lastname'),
array('EavAttribute.attribute_code'=>'firstname')
),
'AND' => array(
array('UserEntityVarchar.entity_id'=>$id)
)
)
I am trying to convert this query into a cakephp query and I am having a bit of trouble with the conditions. No matter how I format the conditions, I always end up with the second query below.
Any help with this is greatly appreciated.
This is the query I am trying to replicate:
SELECT
`UserEntityVarchar`.`value_id`,
`UserEntityVarchar`.`attribute_id`,
`UserEntityVarchar`.`entity_id`,
`UserEntityVarchar`.`value`,
`EavAttribute`.`attribute_code`
FROM
`user_entity_varchars` AS `UserEntityVarchar`
LEFT JOIN `eav_attributes` AS `EavAttribute` ON(
`UserEntityVarchar`.`attribute_id` = `EavAttribute`.`attribute_id`)
WHERE
(UserEntityVarchar.entity_id = 1 AND
EavAttribute.attribute_code = 'firstname') OR
(UserEntityVarchar.entity_id = 1 AND
EavAttribute.attribute_code = 'lastname')
This is the query I keep getting no matter how I format my condition:
SELECT
`UserEntityVarchar`.`value_id`,
`UserEntityVarchar`.`attribute_id`,
`UserEntityVarchar`.`entity_id`,
`UserEntityVarchar`.`value`,
`EavAttribute`.`attribute_code`
FROM
`user_entity_varchars` AS `UserEntityVarchar`
LEFT JOIN `eav_attributes` AS `EavAttribute` ON(
`UserEntityVarchar`.`attribute_id` = `EavAttribute`.`attribute_id`)
WHERE
((`UserEntityVarchar`.`entity_id` = 1)
AND
(`EavAttribute`.`attribute_code` = 'firstname'))
AND
((`UserEntityVarchar`.`entity_id` = 1)
AND
(`EavAttribute`.`attribute_code` = 'lastname'))
This is the condition that I am using:
'conditions'=>array(
array(
array('UserEntityVarchar.entity_id'=>$id),
array('AND ' => array('EavAttribute.attribute_code'=>'firstname'))
),
array('OR' =>
array('UserEntityVarchar.entity_id'=>$id),
array('AND ' => array('EavAttribute.attribute_code'=>'firstname'))
)
)
You should be able to do it like this:
$this->Model->find('all', array('conditions' => array('OR' => array(
array('UserEntityVarchar.entity_id' => 1,
'EavAttribute.attribute_code' => 'firstname'),
array('UserEntityVarchar.entity_id' => 1,
'EavAttribute.attribute_code' => 'lastname')))));