How to search for text and filter? - redis

I am trying to search for filter by an indexed field and search for free text inside my objects. It seems that it is searching for the company name or the search term. How do I make this an and for SearchStream.freeText(String freeText)?
I need to use SearchStream.freeText since there are many fields I want to search on.
My data based on the roms-documents, I modified RomsDocumentsApplication
companyRepo.deleteAll();
Company redis1 = Company.of("Redis", "wwwabccom", new Point(-122.066540, 37.377690), 526, 2011, Set.of(CompanyMeta.of("Redis", 100, Set.of("RedisTag"))));
Company redis2 = Company.of("Redis", "wwwxyzcom", new Point(-122.066540, 37.377690), 526, 2011, Set.of(CompanyMeta.of("Redis", 100, Set.of("RedisTag"))));
Company microsoft1 = Company.of("Microsoft", "wwwabcnet", new Point(-122.066540, 37.377690), 526, 2011, Set.of(CompanyMeta.of("Redis", 100, Set.of("RedisTag"))));
Company microsoft2 = Company.of("Microsoft", "wwwxyznet", new Point(-122.066540, 37.377690), 526, 2011, Set.of(CompanyMeta.of("Redis", 100, Set.of("RedisTag"))));
companyRepo.save(redis1);
companyRepo.save(redis2);
companyRepo.save(microsoft1);
companyRepo.save(microsoft2);
#Data
#NoArgsConstructor
#RequiredArgsConstructor(staticName = "of")
#AllArgsConstructor(access = AccessLevel.PROTECTED)
#Document
#ToString
public class Company {
#Id
private String id;
#NonNull
#Indexed
private String name;
If I search, my first two searches seem fine, but my third search find values not for the company Microsoft, but for Redis
var result1 = entityStream.of(Company.class)
.filter(Company$.NAME.eq("Redis"))
.filter("*abc*")
.collect(Collectors.toList());
System.out.println("Search1:");
result1.forEach(System.out::println);
var result2 = entityStream.of(Company.class)
.filter(Company$.NAME.eq("Microsoft"))
.filter("*xyz*")
.collect(Collectors.toList());
System.out.println("Search2:");
result2.forEach(System.out::println);
var result3 = entityStream.of(Company.class)
.filter(Company$.NAME.eq("Microsoft"))
.filter("*co*")
.collect(Collectors.toList());
System.out.println("Search3:");
result3.forEach(System.out::println);
The output of the program:
Search1:
Company(id=01GR46XB7BMJXV5WK3AJ3FA4M9, name=Redis, tags=[], url=wwwabccom,
Company(id=01GR46XB974C3JJDAW70N20SFV, name=Microsoft, tags=[], url=wwwabcnet,
Search2:
Company(id=01GR46XB9388CRW0P71QM45K43, name=Redis, tags=[], url=wwwxyzcom,
Company(id=01GR46XB9ABBV3TN9BTQFP08C1, name=Microsoft, tags=[], url=wwwxyznet,
Search3:
Company(id=01GR46XB7BMJXV5WK3AJ3FA4M9, name=Redis, tags=[], url=wwwabccom,
Company(id=01GR46XB9388CRW0P71QM45K43, name=Redis, tags=[], url=wwwxyzcom,
I start redis with
docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:6.2.6-v2
module list
127.0.0.1:6379> module list
1) 1) "name"
2) "timeseries"
3) "ver"
4) (integer) 10805
2) 1) "name"
2) "graph"
3) "ver"
4) (integer) 21005
3) 1) "name"
2) "ReJSON"
3) "ver"
4) (integer) 20403
4) 1) "name"
2) "search"
3) "ver"
4) (integer) 20604
5) 1) "name"
2) "bf"
3) "ver"
4) (integer) 20403
Maven
<dependency>
<groupId>com.redis.om</groupId>
<artifactId>redis-om-spring</artifactId>
<version>0.6.4</version>
</dependency>

Can you post the output of modules list from the CLI? I suspect this has to do with Redis' rules for tokenization/Escaping https://redis.io/docs/stack/search/reference/escaping/ and prefix https://redis.io/docs/stack/search/reference/query_syntax/#prefix-matching and infix/suffix matches https://redis.io/docs/stack/search/reference/query_syntax/#infixsuffix-matching

The answer is actually quite simple.
The order of the filter matters. If you search in this order the first filter is ignored.
var result3 = entityStream.of(Company.class)
.filter(Company$.NAME.eq("Microsoft"))
.filter("*co*")
.collect(Collectors.toList());
System.out.println("Search3:");
If you switch the filter it works flawlessly
var result4 = entityStream.of(Company.class)
.filter("*co*")
.filter(Company$.NAME.eq("Microsoft"))
.collect(Collectors.toList());
System.out.println("Search4:");
result4.forEach(System.out::println);
I found this out by placing a break point on io.redisearch.client.Client
Somehow for Search3 the filter on Company name is missing, but is present for Search4.
Added a bug report.

Related

API call from google sheets is too long

I am trying to write a script in google sheets to update my 3commas bots. The API requires a number of mandatory fields are passed even when there's only 1 item that needs to be updated.
The code I have is below and it uses the values already read from the platform updating only the base_order_volume value. This works perfectly except for when the pairs value is long (more the 2k chars) and then I get an error from the UrlFetchApp call because the URL is too long.
var sheet = SpreadsheetApp.getActiveSheet();
var key = sheet.getRange('F4').getValue();
var secret = sheet.getRange('F5').getValue();
var baseUrl = "https://3commas.io";
var editBots = "/ver1/bots/"+bots[botCounter].id+"/update";
var patchEndPoint = "/public/api"+editBots+"?";
.
.
[loop around values in sheet]
.
.
var BaseOrder=Number(sheet.getRange(rowCounter,12).getValue().toFixed(2));
var botParams = {
"name": bots[botCounter].name,
"pairs": bots[botCounter].pairs,
"max_active_deals": bots[botCounter].max_active_deals,
"base_order_volume": BaseOrder,
"take_profit": Number(bots[botCounter].take_profit),
"safety_order_volume": bots[botCounter].safety_order_volume,
"martingale_volume_coefficient": bots[botCounter].martingale_volume_coefficient,
"martingale_step_coefficient": Number(bots[botCounter].martingale_step_coefficient),
"max_safety_orders": bots[botCounter].max_safety_orders,
"active_safety_orders_count": Number(bots[botCounter].active_safety_orders_count),
"safety_order_step_percentage": Number(bots[botCounter].safety_order_step_percentage),
"take_profit_type": bots[botCounter].take_profit_type,
"strategy_list": bots[botCounter].strategy_list,
"bot_id": bots[botCounter].id
};
var keys = Object.keys(botParams);
var totalParams = keys.reduce(function(q, e, i) {
q += e + "=" + encodeURIComponent(JSON.stringify(botParams[e])) + (i != keys.length - 1 ? "&" : "");
return q;
},endPoint);
var signature = Utilities.computeHmacSha256Signature(totalParams, secret);
signature = signature.map(function(e) {return ("0" + (e < 0 ? e + 256 : e).toString(16)).slice(-2)}).join("");
var headers = {
'APIKEY': key,
'Signature': signature,
};
try {
var params = {
'method': 'PATCH',
'headers': headers,
'muteHttpExceptions': true
};
var response = JSON.parse(UrlFetchApp.fetch(baseUrl + totalParams, params).getContentText());
I have tried to set the botParams as a payload in the params but when I do the signature is incorrect.
I anyone knows how to use sheets to make a call using extensive length of parameters I'd appreciate any help at all
Some sample data for the bots array would be
{
"name": "TestBot",
"base_order_volume": 0.001,
"take_profit": 1.5,
"safety_order_volume": 0.001,
"martingale_volume_coefficient": 2,
"martingale_step_coefficient": 1,
"max_safety_orders": 1,
"active_safety_orders_count": 1,
"safety_order_step_percentage": 2.5,
"take_profit_type": "total",
"stop_loss_percentage": 0,
"cooldown": 0,
"pairs": ["BTC_ADA","BTC_TRX"],
"trailing_enabled":"true",
"trailing_deviation":0.5,
"strategy_list": [{"strategy":"cqs_telegram"}]
}
Thanks in advance
I'd consider using a Cloud Function to either do the heavy lifting, or, if you're worried about costs, use it as a proxy. You can then call the cloud function from Google Sheets. Cloud Functions can be written in whatever language you're most comfortable with, including Node.
Check the GCP pricing calculator to see what the cost would be. For many cases it would be completely free.
This should give you a sense of how to use cloud functions for CSV creation:
https://codelabs.developers.google.com/codelabs/cloud-function2sheet#0
Here is a SO question with an answer that explains how to query cloud functions with authentication.

LUA script for modification object in Redis

I Need a short advise please about LUA script for modification object in Redis.
I Have Redis List with entities like that:
{
"#class": "com.myproject.model.Book",
"bookId": "someId",
"author": "someAuthor"
}
now, I need to change my entity to allow multiple authors for certain book, and create migration script for this:
{
"#class": "com.myproject.model.Book",
"bookId": "someId",
"authors": [
"java.util.ArrayList",
[
"someAuthor"
]
]
}
What I Think I need to do with LUA:
local book
local cacheName --cache name in my case
local authorId;
local size = redis.call('LLEN', cacheName)
if size == 0
then
return -1
end
while size > 0
do
book = redis.call('LPOP', cacheName)
-- modify entity here
affectedEntitiesCount = affectedEntitiesCount + 1
redis.call('RPUSH', cacheName, book)
size = size - 1
end
return affectedEntitiesCount
But I have no idea how to modify book according requirements.
Can someone take a look and suggest?
solved:
local book
local cacheName
local authorPattern= '"author":"[^"]*"'
local authorId
local replacementAuthors = '"authors":["java.util.ArrayList",["%s"]]'
local size = redis.call('LLEN', cacheName)
if size == 0
then
return -1
end
while size > 0
do
book = redis.call('LPOP', cacheName)
authorId = string.match(string.match(book, authorPattern), [["authorId":"([^"]+)]])
replacedAuthors = string.format(replacedAuthors , authorId)
book = string.gsub(book, authorPattern, replacedAuthors)
affectedEntitiesCount = affectedEntitiesCount + 1
redis.call('RPUSH', cacheName, book)
size = size - 1
end
return affectedEntitiesCount
Since it appears that you're storing JSON-encoded values in your list, you can use the cjson library that's embedded in Redis' Lua engine.
For example:
...
book = cjson.decode(redis.call('LPOP', cacheName))
book['bookId'] = book['bookId']..'foo' -- concat 'foo' to bookId
...
redis.call('RPUSH', cacheName, cjson.encode(book))
Note: also make sure that you use the KEYS input array to parameterize the script's input key names (e.g. local cacheName = KEYS[1])

How to duplicate an existing order (core, php) with Prestashop 1.6.x in FrontPage from client Account - My Orders?

if i use a function in classes\duplicateOrder, found on a previous answer, everytime i refresh the page order-history, all orders from my account->orders are duplicated
the function seems to be ok, so where is the problem?
public function duplicateOrder($id_order)
{
$order = new Order($id_order);
$duplicatedOrder = $order->duplicateObject();
$orderDetailList = $order->getOrderDetailList();
foreach ($orderDetailList as $detail) {
$orderDetail = new orderDetail($detail['id_order_detail']);
$duplicatedOrderDetail = $orderDetail->duplicateObject();
$duplicatedOrderDetail->id_order = $duplicatedOrder->id;
$duplicatedOrderDetail->save();
}
$orderHistoryList = $order->getHistory(Configuration::get('PS_LANG_DEFAULT'));
foreach ($orderHistoryList as $history) {
$orderHistory = new OrderHistory($history['id_order']);
$duplicatedOrderHistory = $orderHistory->duplicateObject();
$duplicatedOrderHistory->id_order = $duplicatedOrder->id;
//$duplicatedOrderHistory->save();
}
}
Where is your call? I think you call it in a foreach that lists the commands.
Regards
in history.tpl I inserted the folowing li option after Reorder button; whenever I press the refresh (F5) the orders are multiplicated (2, 4, 8, 16 and so on)
{l s='Duplicate Order'}
the function was saved in classes\Order\Order.php

Linqpad extension to plot graphs

I tried to plot some graphs in Linqpad with with "Util.RawHtml()" and "Dump()" but it is not working with this example from amcharts.com. I created a string variable including all the HTML source code but the result is not working.
string html = "";
using (System.Net.WebClient client = new System.Net.WebClient ())
{
html = client.DownloadString(#"http://pastebin.com/raw/pmMMwXhm");
}
Util.RawHtml(html).Dump();
Later versions of LinqPad 5 now support charting out of the box with Util.Chart. You can see the samples in the Samples Tab (next to My Queries) under
LINQPad Tutorial&Reference
Scratchpad Features
Charting with Chart
The following script is the Chart() - dual scale sample:
// Each y-series can have a different series type, and can be assigned to the secondary y-axis scale on the right.
var customers = new[]
{
new { Name = "John", TotalOrders = 1000, PendingOrders = 50, CanceledOrders = 20 },
new { Name = "Mary", TotalOrders = 1300, PendingOrders = 70, CanceledOrders = 25 },
new { Name = "Sara", TotalOrders = 1400, PendingOrders = 60, CanceledOrders = 17 },
};
customers.Chart (c => c.Name)
.AddYSeries (c => c.TotalOrders, Util.SeriesType.Spline, "Total")
.AddYSeries (c => c.PendingOrders, Util.SeriesType.Column, "Pending", useSecondaryYAxis:true)
.AddYSeries (c => c.CanceledOrders, Util.SeriesType.Column, "Cancelled", useSecondaryYAxis:true)
.Dump();
As I understand it, this will not work because the html contains scripts that will not be executed.
As an alternative, you can still use the old (and deprecated) google charts api, eg
var link = #"http://chart.apis.google.com/chart?chxt=y&chbh=a&chs=300x225&cht=bvg&chco=A2C180,3D7930&chd=t:10,20,30,40,50,60|30,35,40,45,55,60&chtt=Sample";
Util.Image (link).Dump();
or see
http://blog.divebomb.org/2012/11/dumping-charts-in-linqpad/
Not sure if it's the answer you're after but there may be value in looking at the DisplayWebPage method on the Util class in Linqpad. This correctly rendered your chart in the result window, (although there was a script error). Obviously, this may not solve your underlying issue.
I used version 5.10.00 to test this.

How to get/set Trello custom fields using the API?

I'm already in love with the Custom Fields feature in Trello. Is there a way to get and set custom fields via the API?
I tried using the get field API call to get a field (on a board with a custom field defined called "MyCustomField"):
curl "https://api.trello.com/1/cards/57c473503a5ef0b76fddd0e5/MyCustomField?key=${TRELLO_API_KEY}&token=${TRELLO_OAUTH_TOKEN}"
to no avail.
The Custom Fields API from Trello is now officially available (announcement blog post here).
It allows users to manipulate both custom field items of boards and custom field item values on cards.
Custom Fields API documentation
Getting customFieldItems For Cards
Setting & Updating CustomFieldItems
This is just to add to bdwakefield's answer. His solution involves hard coding the names of the field ids.
If you want to also retrieve the name of the fields themselves (for example get that "ZIn76ljn-4yeYvz" is actually "priority" in Trello, without needing to hard code it, call the following end point:
boards/{trello board id}/pluginData
This will return an array with the plugins information and in one of the array items, it will include a line along the lines of:
[value] => {"fields":[{"n":"~custom field name~:","t":0,"b":1,"id":"~custom field id that is the weird stuff at the card level~","friendlyType":"Text"}]}
So you just need to figure out the plugin for custom fields in your case, and you can retrieve the key -> value pair for the custom field name and the id associated with it.
It means that if you add / remove fields, or rename them, you can handle it at run time vs changing your code.
This will also give you the options for the custom field (when it is a dropdown like in bdwakefield's example above).
So I have a "sort of" answer to this. It requires some hackery on your part to make it work and there is more than a little manual upkeep as you add properties or values -- but it works.
I am doing this in powershell (I am NOT well versed in ps just yet and this my first really 'big' script that I have pulled together for it) since my intent is to use this with TFS Builds to automate moving some cards around and creating release notes. We are using custom fields to help us classify the card and note estimate/actual hours etc. I used this guys work as a basis for my own scripting. I am not including everything but you should be able to piece everything together.
I have left out everything with connecting to Trello and all that. I have a bunch of other functions for gettings lists, moving cards, adding comments etc. The ps module I linked above has a lot of that built in as well.
function Get-TrelloCardPluginData
{
[CmdletBinding()]
param
(
[Parameter(Mandatory,ValueFromPipelineByPropertyName)]
[ValidateNotNullOrEmpty()]
[Alias('Id')]
[string]$CardId
)
begin
{
$ErrorActionPreference = 'Stop'
}
process
{
try
{
$uri = "$baseUrl/cards/$CardId/pluginData?$($trelloConfig.String)"
$result = Invoke-RestMethod -Uri $uri -Method GET
return $result
}
catch
{
Write-Error $_.Exception.Message
}
}
}
You'll get data that looks like this:
#{id=582b5ec8df1572e572411513; idPlugin=56d5e249a98895a9797bebb9;
scope=card; idModel=58263201749710ed3c706bef;
value={"fields":{"ZIn76ljn-4yeYvz":2,"ZIn76ljn-c2yhZH":1}};
access=shared}
#{id=5834536fcff0525f26f9e53b; idPlugin=56d5e249a98895a9797bebb9;
scope=card; idModel=567031ea6a01f722978b795d;
value={"fields":{"ZIn76ljn-4yeYvz":4,"ZIn76ljn-c2yhZH":3}};
access=shared}
The fields collection is basically key/pair. The random characters correspond to the property and the value after that is what was set on the custom property. In this case it is an 'index' for the value in the dropdown. These two fields have a 'priority' (low, medium, high) and a 'classification' (Bug, Change Request, etc) for us. (We are using labels for something else).
So you'll have to create another fucntion where you can parse this data out. I am sure there are better ways to do it -- but this is what I have now:
function Get-TrelloCustomPropertyData($propertyData)
{
$data = $propertyData.Replace('{"fields":{', '')
$data = $data.Replace('}}', '')
$data = $data.Replace('"', '')
$sepone = ","
$septwo = ":"
$options = [System.StringSplitOptions]::RemoveEmptyEntries
$obj = $data.Split($sepone, $options)
$cardCustomFields = Get-TrelloCustomFieldObject
foreach($pair in $obj)
{
$field = $pair.Split($septwo,$options)
if (-Not [string]::IsNullOrWhiteSpace($field[0].Trim()))
{
switch($field[0].Trim())
{
'ZIn76ljn-4yeYvz' {
switch($field[1].Trim())
{
'1'{
$cardCustomFields.Priority = "Critical"
}
'2'{
$cardCustomFields.Priority = "High"
}
'3'{
$cardCustomFields.Priority = "Medium"
}
'4'{
$cardCustomFields.Priority = "Low"
}
}
}
'ZIn76ljn-c2yhZH' {
switch($field[1].Trim())
{
'1'{
$cardCustomFields.Classification = "Bug"
}
'2'{
$cardCustomFields.Classification = "Change Request"
}
'3'{
$cardCustomFields.Classification = "New Development"
}
}
}
'ZIn76ljn-uJyxzA'{$cardCustomFields.Estimated = $field[1].Trim()}
'ZIn76ljn-AwYurD'{$cardCustomFields.Actual = $field[1].Trim()}
}
}
}
return $cardCustomFields
}
Get-TrelloCustomFieldObject is another ps function that I set up to build an object based on the properties I know that I have defined.
function Get-TrelloCustomFieldObject
{
[CmdletBinding()]
param()
begin
{
$ErrorActionPreference = 'Stop'
}
process
{
$ccf = New-Object System.Object
$ccf | Add-Member -type NoteProperty -name Priority -value "None"
$ccf | Add-Member -type NoteProperty -name Classification -value "None"
$ccf | Add-Member -type NoteProperty -name Estimated -value ""
$ccf | Add-Member -type NoteProperty -name Actual -value ""
return $ccf
}
}