Laravel TNTsearch custom index creation and usage for large SQL database table - sql

Here is my situation, context, and dilemma.
Situation
I'm fairly new to Laravel and still learning the ropes. I recently installed TNTSearch and Laravel Scout and was able to create a model index using the below config. I created the index using the console command php artisan tntsearch:import "App\Models\Product" and can fuzzy search successfully with App\Models\Product::search($keyword)->get().
config/scout.php
'tntsearch' => [
'storage' => storage_path() . '/index',
'fuzziness' => 'auto',
'fuzzy' => [
'prefix_length' => 2,
'max_expansions' => 50,
'distance' => 4,
],
'asYouType' => true
],
Context
I have an SQL database table with over 30k+ product records segmented per province (Canadian project), and instead of searching the whole index and later filter by market, I’d like to create one index per market and launch a search for a given market. I believe it will speed up the search and avoid returning results which will later be discarded! So basically having one product index file per province (i.e. products_on.index, products_qc.index, ...)
Dilemma/Issue
I am unable to find how to create such an index, have it update automatically and also how to use it. I scoured the Internet for tutorial/guidance and could only find scarce information I can hardly put together. I’d appreciate if someone could point me in the right direction or guide me on how to implement such a thing.
No answer is wrong, and any bits and pieces of information can help me greatly to “get up to speed.”
EDIT (July 30th, 2018):
I still haven't found the answer to my request but the more I search, the more I'm concluding search indexes are "tied" to a model, and it is not possible to have more than one index per model. So I would have to create one model extension per market from the original Listings model (Listings_QC, Listings_ON, ...). Then create an index per markets and search from those (Listings_QC::search(...)).
I'm not keen to create models based on data! Is this a good approach/practice?

RESOLVED !
My inexperience with Laravel search index in general lead me in the wrong direction!
I finally found a document explaining how to use searchBoolean() to search using "and". Modified my config as below to add the searchBoolean:
'tntsearch' => [
'storage' => storage_path() . '/index',
'fuzziness' => 'auto',
'fuzzy' => [
'prefix_length' => 2,
'max_expansions' => 50,
'distance' => 4,
],
'asYouType' => true,
'searchBoolean' => true
],
Then specify the market using the model's method toSearchableArray(), and add the market to any requested seach keyword.
For example, listing search with 'Alsace' for a QC market, I launch the search as
Listings::search('Alsace QC')->get().
Voilà! May help others hitting the same "wall"!

Related

Access objects from project in sandbox environment PHP

Is there a way I can access my project under Sandbox? I'm able to use the lookup method, find in order to fetch all the features from a project under the Yahoo! subscription, but how would I be able to do this for projects under Sandbox?
In your PHP code have you used Rally sandbox server URL?
https://sandbox.rallydev.com/
Here is a WebServices URL specific to Sandbox:
https://sandbox.rallydev.com/slm/doc/webservice/
I was able to figure it out. In the query to find specific features, I had to include the query parameter "workspace" to the sandbox reference which is (for 1.43) : "https://rally1.rallydev.com/slm/webservice/1.43/workspace/7189290105.js". I included the reference of the project as well which directly fetched all the features for my project. In addition, if you seek to only fetch features from your specific project and not from the ones on top of it, you have to include the "pageScopeUp" field into the query. You have to set this field to false:
$queryParams = array(
'query' => "",
'fetch' => 'true',
'pagesize' => 100,
'start' => 1,
'workspace' => "https://rally1.rallydev.com/slm/webservice/1.43/workspace/7189290105.js",
'project' => "whatever the project reference is",
'projectScopeUp' => false
);
$results = Connection::rally()->findWithQueryParameters('feature',
$queryParams);

Scope/Filter collection from route

I've got a standard resourceful route setup for 'Invoice' however I'm looking to add in the ability to filter the invoice records based on their state.
/invoices - shows all invoices
/invoices/unpaid - shows all unpaid invoices
/invoices/paid - shows all paid invoices.
/invoices/3 - shows invoice #3
I've gotten this working no problem with an explicitly defined match route.
match "/invoices/pending" => "invoices#index", :state => 'pending'
However, with a growing number of possible states this means modifying the routes regularly, and also means I'm repeating myself quite a lot.
My next attempt was to make this route a little more dynamic with named params in the match route.
match "/invoices/:state" => "invoices#index"
However this then negates the /invoices/id route and trying to look for /invoices/3 finds no records as it's searching based on the state parameter.
Can anyone help with defining a filter route such as this that'll work dynamically?
Add regular expression as constraint in the routes:
match "/invoices/:id" => "invoices#show", :id => /\d+/
match "/invoices/:state" => "invoices#index"
It should select the only-number id-s for show, and the rest for the index.
Try to use routing constraints . Something like
match "/invoices/:id" => "invoices#show", constraint: { id: /\d+/ }
match "/invoices/:state" => "invoices#index", constraint: { state: /\w+/ }
I decided to take a slightly different route (excuse the pun) with this, by adding a constraint that rather then using REGEX used a method to look at possible states direct on the state machine.
# State based routes, which match the state machine dynamicly.
match "/invoices/:state" => "invoices#index", constraints: lambda { |r|
# Get an array if potential state names.
states = Invoice.state_machine.states.map &:name
# See if the request state name matches or not.
states.include?(r.params[:state].parameterize.underscore.to_sym)
}
# Resource routes go here.
This basically means that if I now go looking for /invoices/paid or /invoices/unpaid I get my expected index action, with the state variable set as expecetd.
However, doing something like /invoices/pay_soon which is not a valid event returns a 404.
Pretty happy with that solution, but thanks to the other suggestions which got me on to the track of using a constraint.

Re-indexing an index in ElasticSearch to change the number of shards

I need to change the number of shards in my index. The index is quite big and i may have to change the configuration 10-15 times for testing purposes before i'm satisfied with the result. is there a tool offering out of the box this kind of functionality? or what's the easiest way of accomplishing this?
Both the Perl and Ruby clients directly support reindexing.
In Perl, you'd do:
my $source = $es->scrolled_search(
index => 'old_index',
search_type => 'scan',
scroll => '5m',
version => 1
);
$es->reindex(
source => $source,
dest_index => 'new_index'
);
Find more information in the post by Clinton Gormley.
In Ruby, you'd do:
Tire.index('old').reindex 'new', settings: { number_of_shards: 3 }
Find more information in the relevant Tire commit.

Is it possible to set certain product attributes for a different store view using the Magento API?

We are currently using the Magento API for importing a bunch of products into the store.
But we now run into a problem where some product attributes should be translated into a different language.
And I was wondering if it is possible to do this using the Magento API, because I can't seem to find anything related to that problem.
We currently have 2 store views, 1 for the Dutch version of the site and one for the French version of the site.
Our current import code looks something like this:
$store_id = $soapClient->call($soapSession, 'catalog_product.currentStore', array('nl'));
echo("store_id: $store_id");
$new_product_data = array(
'name' => 'NameInDutch',
'short_description' => 'DescriptionInDutch',
'price' => $price,
'weight' => $weight,
'websites' => array('base'),
'status' => '1'
);
$new_product_id = $soapClient->call($soapSession, 'catalog_product.create', array('simple', 4, $sku, $new_product_data)); // 4 => 'Default' attribute set
$localized_product_data = array(
'name' => 'NameInFrench',
'short_description' => 'DescriptionInFrench'
);
$store_id = $soapClient->call($soapSession, 'catalog_product.currentStore', array('fr'));
echo("store_id: $store_id");
$soapClient->call($soapSession, 'catalog_product.update', array($sku, $localized_product_data ));
Now, the output of the echo statements differs, the first time it's 1 and the second time it's 2, so that doesn't seem to be problem. But apparently it doesn't matter for the API if we set that value.
The result is that on the 'catalog_product.update' call, the name 'NameInFrench' overwrites the default name 'NameInDutch'.
So my question is if something like this is possible using the Magento API, and how one would accomplish this?
Ok, I found the answer, apparently I overlooked a certain line in the Magento API docs, because the solution was right there.
So: you don't need to set the currentStore each time, you just need to append the store id or code to the update array:
$soapClient->call(
$soapSession,
'catalog_product.update',
array($sku, $localized_product_data, 'fr')
);
This works perfectly.

Cakephp, i18n, SQL Error, Not unique table/alias

I get the following SQL error:
SQL Error: 1066: Not unique table/alias: 'I18n__name'
when doing a simple find query.
Any ideas on possible situations that may have caused this??
I'm using a bindModel method to retrieve the data is that related?
This is my code:
$this->Project->bindModel(array(
'hasOne' => array(
'ProjectsCategories',
'FilterCategory' => array(
'className' => 'Category',
'foreignKey' => false,
'conditions' => array('FilterCategory.id = ProjectsCategories.category_id')
))));
$prlist = $this->Project->find('all', array(
'fields' => array('DISTINCT slug','name'),
'conditions' => array('FilterCategory.slug !='=>'uncategorised')
))
You have not used or initialized the required table/model you are using in your controller. Use var $uses = array('your_table_name');
I do not have a direct answer to my problem. However after some research I came to the following conclusion.
I've decided to change my methodology and stop using the translate behavior in cakephp.
I've found another behavior called i18n which works much better when dealing with associated models. You can read about it http://www.palivoda.eu/2008/04/i18n-in-cakephp-12-database-content-translation-part-2/#comment-1380
In cakephp book it says :
"Note that only fields of the model
you are directly doing find on will
be translated. Models attached via
associations won't be translated
because triggering callbacks on
associated models is currently not
supported."
I'm not sure if I made the right choice however I have been struggling to get the translate behavior to work in cakephp and this solution at least makes the project I'm working functional.