ElasticSearch Tire two field conditional filter - ruby-on-rails-3

I'm doing a mutli-index query With Tire and rails 3 and I want to filter out Venues who have approved => false so I need some sort of combo filter.
Here is the query
query = params[:q]
from = params.delete(:from)
size = params[:size] || 25
Tire.search(
[Venue.index_name,
Performer.index_name, User.index_name], load: true) do |s|
s.query do
string(query, fields: [:_all, :name, :title], use_dis_max: true)
end
s.from from if from
s.size size if size
end.results.to_a
This line removes all Performers and Users because they don't have an :approved field.
s.filter(:term, :approved => true )
And this line obviously removes all non-venues which is no good.
s.filter(:term, { :approved => true, :index_name => 'venues'} )
Any ideas besides adding an approved: true field to all Users and Performers? I think something like this is what I want conceptually:
s.filter(:term, :approved => true, :if => {:index_name => 'venues'} )
EDIT Thanks to Mallox I was able to find the Should construct but I'm still struggling to implement it Tire. It seems like the below code should work but it return no results on any query. I also remove the "{:terms => { :index_name => ["performers", "users"]}}," to make sure it wasn't my use of index name or multiple lines of query that was the problem and still no luck. Can anybody shed some light on how to do this in Tire?
s.filter(:bool, :should => [
{:terms => { :index_name => ["performers", "users"]}},
{:term => { :approved => true}},
] )

So i have little knowledge about Ruby and Tire, but the ElasticSearch query that you want to build would be based on a bool filter, that contains some "should" entries (which would translate into inclusive OR).
So in your case something along the lines of:
"filter" : {
"bool" : {
"should" : [
{
"terms" : { "_type" : ["Performers","Users"] }
},
{
"term" : { "approved" : true }
}
]
}
}
Take a look at the documentation here, maybe that'll help:
:http://www.elasticsearch.org/guide/reference/query-dsl/bool-filter/

Related

cakephp4 get 1st record from containing table with order by a field

I have cakephp4 project
having 1 to many relationship between Portfolios and PSnaps
I want to show all 'Portfolios' with one associated record from PSnaps where its PSnaps.status=1 and order=>['PSnap.order_at'=>'ASC']
I tried many things but getting the correct result
below is giving 90% correct result only ordering on PSnaps.order_at is not working.
along with hasmany() i have created hasOne() association as shown in below model
Model
class PortfoliosTable extends Table
{
public function initialize(array $config): void
{
parent::initialize($config);
$this->setTable('portfolios');
$this->setDisplayField('id');
$this->setPrimaryKey('id');
$this->hasOne('FirstPSnaps', [
'className' => 'PSnaps',
'foreignKey' => 'portfolio_id',
'strategy' => 'select',//also tried join
//'joinType'=>'LEFT',//also tried inner,left,right
//'sort' => ['FirstPSnaps.order_at' => 'ASC'], //*******this is not working
'conditions' => function (\Cake\Database\Expression\QueryExpression $exp, \Cake\ORM\Query $query) {
$query->order(['FirstPSnaps.order_at' => 'ASC']);//*******also not working
return [];
}
])
;
$this->hasMany('PSnaps', [
'foreignKey' => 'portfolio_id',
]);
}
Controller
$pfolios = $this->Portfolios->find('all')
->select(['Portfolios.id','Portfolios.client','Portfolios.country','Portfolios.url'])
->where(['Portfolios.status'=>1])
->order(['Portfolios.order_at'=>'asc','Portfolios.id'=>'asc'])
->limit(8)
->contain([
'FirstPSnaps'=>function($q){
return $q
->select(['FirstPSnaps.portfolio_id','FirstPSnaps.snap'])
//->where(['FirstPSnaps.status'=>1])
//->order(['FirstPSnaps.order_at'=>'asc'])
;
}
])
->toArray();
it is returning correct porfolios with 1 p_snap record but ordering/sorting is not correct as I need first p_snap something like where p_snap.status=1 and p_span.portfolio_id= portfolios.id limit 1.

How to decode json data in sql for search?

need some advice to solve problem with yii grid/POstgreSql search.
In table I have some row let's call - some_important_data.
For showing in gridview i write some code in MyModel.php afterFind() method.
$someData = $this->some_data ? json_decode(urldecode($this->some_data)) :'';
if(!empty($someData)) {
$this->searcheeReason = $someData->csr_refuse_reason;
$this->witnessReason = $someData->csr_witness_reason;
}
........
if(!empty($this->starttime) && !empty($this->endtime) && !empty($csrPauseData->csr_colleague_reason)){
$this->status = "Rejected";
}
Now I want to write Search Part in search method in MyMethodSearch.php and Order in my controller =>
$dataProvider = $searchModel->search(Yii::$app->request->queryParams);
$dataProvider->setSort([
'attributes' => [
'searchNum' => [
'asc' => ['responses.number' => SORT_ASC],
'desc' => ['csresponses.number' => SORT_DESC]
],
'searcheeReason' => ???
I try a lot of things, but I don't find any optimal way. One of that ways
=>
SELECT substring(csresponses.some_data
from (POSITION('%22csr_refuse_reason%22%3A%22' in
csresponses.some_data) +
LENGTH('%22csr_refuse_reason%22%3A%22'))
for (POSITION('%22%2C%22' in
substring(csresponses.some_data
from (POSITION('%22csr_refuse_reason%22%3A%22' in
csresponses.some_data) )
for (LENGTH( csresponses.some_data) -
POSITION('%22csr_refuse_reason%22%3A%22' in csresponses.some_data))))
- LENGTH('%22use_firstname%22%3A%22')))
from csresponses
Code written ofcourse with yii rules, it's just sql interpretation

Ruby integration with mailchimp-api gem

I'm trying to subscribe a single email to multiple lists with RoR and the official mailchimp-api gem. It works, but the last four values (double_optin, update_existing, replace_interests, and send_welcome) are not updating and I get an error that the email "already exists" even though I'm trying to pass the update_existing as true. I've written Mailchimp several times and they feel they've reached the end of their assistance. They have said they are not experts in the wrapper--even if it is the "official" gem--and cannot help me further. My code looks like this:
responses << mailchimp_lists.each do |ml|
mailchimp.lists.subscribe(
ml,
{ "email" => order.customer_email,
"euid" => order.customer_id,
"leid" => ""
},
{ "FNAME" => order.customer_first_name,
"LNAME" => order.customer_last_name,
"COMPANY" => order.company_name,
"ADDRESS1" => order.billing_address_1,
"ADDRESS2" => order.billing_address_2,
"CITY" => order.billing_city,
"STATE" => order.billing_state,
"POSTALCODE" => order.billing_zip,
"SALUTATION" => ""
},
"html",
false,
true,
false,
false
)
end
I've tried sending the last four params in several different ways such as:
"email_type" => "html",
"double_optin" => false,
Or:
{"email_type" => "html"},
{"double_optin" => false}
At times, Mailchimp can see the params arrive in such a way that it seems it should not be triggering an "email already exists" error, but it just won't work. Any help is appreciated.
The mailchimp-api gem's documentation describes the subscribe method as:
#subscribe(id, email, merge_vars = nil, email_type = 'html', double_optin = true, update_existing = false, replace_interests = true, send_welcome = false)
While the batch_subscribe shows:
#batch_subscribe(id, batch, double_optin = true, update_existing = false, replace_interests = true)
Note that the batch method does not include a "send_welcome" param. When I removed it from my list of params for the subscribe method--essentially sending three booleans instead of four as suggested, the update_existing worked perfectly. Seems like an error in the documentation here: http://www.rubydoc.info/gems/mailchimp-api/2.0.4/Mailchimp/Lists#subscribe-instance_method
Hopefully this helps someone else!

How to setup ElasticSearch to do SQL LIKE "%" for email addresses?

In SQL, I can search email addresses pretty well with SQL LIKE.
With an email "stack#domain.com", searching "stack", "#domain.com", "domain.com", or "domain" would get me back the desired email address.
How can I get the same result with ElasticSearch?
I played with nGram, edgeNGram, uax_url_email, etc and the search results have been pretty bad. Please correct me if I'm wrong, it sounds like I have to do the following:
for index_analyzer
use "keyword", "whitespace", or "uax_url_email" tokenizer so the email don't get tokenized
but wildcard queries don't seem to work (with tire at least)
use "nGram" or "edgeNGram" for filter
I always get way too many unwanted results like getting "first#domain.com" when searching "first-second".
for search_analyzer
don't do nGram
One experiment code
tire.settings :number_of_shards => 1,
:number_of_replicas => 1,
:analysis => {
:filter => {
:db_ngram => {
"type" => "nGram",
"max_gram" => 255,
"min_gram" => 3 }
},
:analyzer => {
:string_analyzer => {
"tokenizer" => "standard",
"filter" => ["standard", "lowercase", "asciifolding", "db_ngram"],
"type" => "custom" },
:index_name_analyzer => {
"tokenizer" => "standard",
"filter" => ["standard", "lowercase", "asciifolding"],
"type" => "custom" },
:search_name_analyzer => {
"tokenizer" => "whitespace",
"filter" => ["lowercase", "db_ngram"],
"type" => "custom" },
:index_email_analyzer => {
"tokenizer" => "whitespace",
"filter" => ["lowercase"],
"type" => "custom" }
}
} do
mapping do
indexes :id, :index => :not_analyzed
indexes :name, :index_analyzer => 'index_name_analyzer', :search_analyzer => 'search_name_analyzer'
indexes :email, :index_analyzer => 'index_email_analyzer', :search_analyzer => 'search_email_analyzer'
end
end
Specific cases that don't work well:
emails with hyphen (eg. email-hyphen#domain.com)
query string '#' at the beginning or end
exact matches
searching with wildcard like '#' gets very unexpected results.
Suppose I have, "aaa#email.com", "aaa_0#email.com", and "aaa-0#email.com, searching "aaa" gives me "aaa#a.com" "aaa-0#email.com. Searching "aaa*" give me everything, but "aaa-*" gives me nothing. So, how should I do exact match wildcard queries? For these type of queries, I get pretty much the same results for different tokenizer/analyzer.
I do these after each mapping change:
Model.tire.index.delete
Model.tire.create_elasticsearch_index
Model.tire.index.import Model.all
References:
Configure ElasticSearch to use ngram by default. - SQL LIKE %% behavior
http://euphonious-intuition.com/2012/08/more-complicated-mapping-in-elasticsearch/
Considering what you are trying to accomplish, KeywordAnalyzer might be a reasonable choice of analyzer, though I don't see anything that would cause problems with a WhitespaceAnalyzer.
I suspect you are running into problems with the query parsing and analysis, although you haven't really described how you are querying. Simplest case would be to simply use term or prefix queries.
It does seem a bit like StandardAnalyzer would serve your purpose here, mostly (differentiating between "aaa_0" and "aaa-0" would be a problem), as long as it is applied consistently, and your query is correct.

conditional update_all with join tables in ActiveRecord?

The following query returns the collection of AR objects that I want to update:
Variant.all(:joins => { :candy_product => :candy }, :conditions => "candies.name = 'Skittles'")
I'm trying to do something like the following:
Variant.update_all(:price => 5, :joins => { :candy_product => :candy }, :conditions => "candies.name = 'Skittles'")
This should only update the price for the variants returned from original query. Is this possible with AR or will I have to write the SQL? This is a pretty large collection, so anything that iterates is out.
Using Rails 2.3.4.
As #François Beausoleil pointed correctly we should use scoped
Variant.scoped(:joins => { :candy_product => :candy }, :conditions => "candies.name = 'Skittles'").update_all(:price => 5)