I am having an issue with the format EsStorage store tuples into Elasticsearch.
Let's take a concrete example:
X: {id: chararray,score: long,answers: {(id: chararray,score: long)}}
(4,385,{(7,287),(7263,48)})
I am using the following ESStorage configuration:
DEFINE EsStorage org.elasticsearch.hadoop.pig.EsStorage (
'es.http.timeout= 5m',
'es.index.auto.create = true',
'es.mapping.pig.tuple.use.field.names = true',
'es.nodes = $es_nodes',
'es.mapping.id = id'
);
And I store my tuples like this:
STORE X INTO '$es_index_name/$es_index_type' USING EsStorage;
The result is the following:
{
"_index": "stackoverflow_test3",
"_type": "post",
"_id": "4",
"_score": 1,
"_source": {
"id": "4",
"score": 385,
"answers": [
[
{
"id": "7",
"score": 287
}
],
[
{
"id": "7263",
"score": 48
}
]
}
}
I'am expecting to have something like this:
{
"_index": "stackoverflow_test3",
"_type": "post",
"_id": "4",
"_score": 1,
"_source": {
"id": "4",
"score": 385,
"answers": [
{
"id": "7",
"score": 287
},
{
"id": "7263",
"score": 48
}
]
}
}
Do you know how could I achieve this ?
Thank you
Related
if I get a response returning a list of elements like the following:
[
{
"c": {
"c1": 4,
"c2": "5",
},
"b": {
"b1": 2,
"b2": "3",
},
"a": "1"
},
{
"cc": {
"cc1": 4,
"cc2": "5",
},
"bb": {
"bb1": 2,
"bb2": "3",
},
"aa": "1"
}
]
Is there a way I can reverse the order of the properties in each element, while maintaining the element order? I am using JPA/Hibernate. The result should be as follows:
[
{
"a": "1",
"b": {
"b1": 2,
"b2": "3",
},
"c": {
"c1": 4,
"c5": "5",
}
},
{
"aa": "1",
"bb": {
"bb1": 2,
"bb2": "3",
},
"cc": {
"cc1": 4,
"cc2": "5",
}
}
]
Thanks in advance.
I would like to find name value at whatever level when searching by ID
I assume Lodash would be the best method, I have been searching and testing but no luck, is this possible?
{
"delegate": [
{
"id": "2",
"name": "David",
"balances": [],
"responsibility": [
{
"id": "18",
"name": "Lilly",
"balances": []
}
]
},
{
"id": "12",
"name": "Barb",
"balances": [],
"responsibility": [
{
"id": "3",
"name": "Amy",
"balances": []
},
{
"id": "9",
"name": "John",
"balances": []
}
]
}
],
"responsibility": [
{
"id": "14",
"name": "Shawn",
"balances": []
}
]
}
how can I fetch the name and team_name keys in this API data?
condition: here 18,1,17, etc are subject codes that change according to the subject and not fix this subject available in the next API call.
{
"18": {
"detail": {
"id": "18",
"name": "Hindi"
},
"list": [
{
"id": "5",
"team_name": "Gurpreet",
},
{
"id": "2",
"team_name": "Test1",
}
]
},
"17": {
"detail": {
"id": "17",
"name": "Punjabi"
},
"list": [
{
"id": "6",
"team_name": "Guru",
},
{
"id": "3",
"team_name": "Test",
}
]
},
"1": {
"detail": {
"id": "1",
"name": "History"
},
"list": [
{
"id": "7",
"team_name": "Gurpreet",
}
]
},
"19": {
"detail": {
"id": "19",
"name": "Math"
},
"list": [
{
"id": "4",
"team_name": "Gurpreet",
}
]
},
"status": true
}
Use this code. You can check keys getter to check dynamics key.
import 'dart:convert';
void main() async {
var f = {
"18": {
"detail": {"id": "18", "name": "Hindi"},
"list": [
{
"id": "5",
"team_name": "Gurpreet",
},
{
"id": "2",
"team_name": "Test1",
}
]
},
"17": {
"detail": {"id": "17", "name": "Punjabi"},
"list": [
{
"id": "6",
"team_name": "Guru",
},
{
"id": "3",
"team_name": "Test",
}
]
},
"1": {
"detail": {"id": "1", "name": "History"},
"list": [
{
"id": "7",
"team_name": "Gurpreet",
}
]
},
"19": {
"detail": {"id": "19", "name": "Math"},
"list": [
{
"id": "4",
"team_name": "Gurpreet",
}
]
},
"status": true
};
for (var o in f.keys) {
print(o);
if (f[o] is bool) {
print(f[o]);
} else { // check it is Map. I consider it always is Map
if ((f[o] as Map)['detail'] != null) {
print((f[o] as Map)['detail']['name']);
}
if ((f[o] as Map)['list'] != null) {
print((f[o] as Map)['list'][0]['team_name']); // you can use for here. please check array is not null
}
}
}
}
I am trying to get data from my database but when I show and make it become api response, I have some problem for grouping it based on productid.
I have response data that created based on golang like this:
[
{
"product_id": "1",
"product_name": "Cardigan",
"pitems": [
{
"id": "625ad1bc-66c5-440e-a527-d029d401ec2b",
"name": "Box",
"qty": 1
},
{
"id": "625ad1bc-66c6-440e-a527-d029d401ec2b",
"name": "test items1",
"qty": 1
},
{
"id": "625ad1bc-66c7-440e-a527-d029d401ec2b",
"name": "test items2",
"qty": 1
},
{
"id": "625ad1bc-66c8-440e-a527-d029d401ec2b",
"name": "test items3",
"qty": 1
}
]
},
{
"product_id": "2",
"product_name": "Polo",
"product_sku": "P01",
"items": [
{
"id": "625ad1bc-66c5-440e-a527-d029d401ec2b",
"name": "Box",
"qty": 1
},
{
"id": "625ad1bc-66c6-440e-a527-d029d401ec2b",
"name": "test items1",
"qty": 1
},
{
"id": "625ad1bc-66c7-440e-a527-d029d401ec2b",
"name": "test items2",
"qty": 1
},
{
"id": "625ad1bc-66c8-440e-a527-d029d401ec2b",
"name": "test items3",
"qty": 1
}
]
}
]
But This response is not my expected result, my expected result is like:
[
{
"product_id": "1",
"product_name": "Cardigan",
"pitems": [
{
"id": "625ad1bc-66c5-440e-a527-d029d401ec2b",
"name": "Box",
"qty": 1
},
{
"id": "625ad1bc-66c6-440e-a527-d029d401ec2b",
"name": "test items1",
"qty": 1
},
{
"id": "625ad1bc-66c7-440e-a527-d029d401ec2b",
"name": "test items2",
"qty": 1
}
]
},
{
"product_id": "2",
"product_name": "Polo",
"product_sku": "P01",
"items": [
{
"id": "625ad1bc-66c8-440e-a527-d029d401ec2b",
"name": "test items3",
"qty": 1
}
]
}
]
Can Anyone help me to solve my problem?
What does it mean to show detailed data?
the easy way is:
create 2 func like this:
func detail(id int)(result model.Struct)
{ return result }
func product()(result model.Struct_Result) {
for data.Next() {
// call func detail
data.Scan(&id, &product)
detailResult := detail(id)
// then put together with struct and mix append ()
outputLoop := model.Result{
"product_id": id,
"pitems": [
{
"id": detailResult.id,
"name": detailResult.name,
"qty": detailResult.qty
},
]
}
result = append(result,outputLoop)
}
return result
}
I have data about Product, which have some fields ( _id, Shop, ProductVerion...). It has indexed in Elastic Search. And I want to search products have max ProductVersion with Shop.
Ex:
Shop Amazon has 3 Version crawl product: 111,222,333.
Shop Ebay has 2 version: 222,444
Shop Alibaba has 2 version: 111, 444
Verions may may be the same.
And now, I want to get Products which have:
Shop Amazon and ProducVersion 333
or Shop Ebay and ProductVersion 444
or Shop Alibaba and ProductVersion 444.
But I don't know query that.
Help me, pls!!
I tried it out with some example documents. I kept version field as numeric field.
These are example documents with which I tried
[
{
"_index": "test",
"_type": "doc",
"_id": "12334",
"_score": 1,
"_source": {
"shopName": "amazon",
"version": 341
}
},
{
"_index": "test",
"_type": "doc",
"_id": "123",
"_score": 1,
"_source": {
"shopName": "amazon",
"version": 3412
}
},
{
"_index": "test",
"_type": "doc",
"_id": "1233",
"_score": 1,
"_source": {
"shopName": "amazon",
"version": 341
}
},
{
"_index": "test",
"_type": "doc",
"_id": "1238",
"_score": 1,
"_source": {
"shopName": "alibaba",
"version": 34120
}
},
{
"_index": "test",
"_type": "doc",
"_id": "1239",
"_score": 1,
"_source": {
"shopName": "alibaba",
"version": 3414
}
},
{
"_index": "test",
"_type": "doc",
"_id": "123910",
"_score": 1,
"_source": {
"shopName": "alibaba",
"version": 124
}
}
]
As #demas had specified I went ahead with terms aggregation and top hits aggregation
indexName/_search
{
"size": 0,
"aggs": {
"shop": {
"terms": {
"field": "shopName.keyword"
},
"aggs": {
"product": {
"top_hits": {
"size": 1,
"sort": [
{
"version": {
"order": "desc"
}
}
]
}
}
}
}
}
}
This should provide you with the document that contains the highest product version number for each shop as shown below.
{
"took": 8,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"shop": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "alibaba",
"doc_count": 3,
"product": {
"hits": {
"total": 3,
"max_score": null,
"hits": [
{
"_index": "test",
"_type": "doc",
"_id": "1238",
"_score": null,
"_source": {
"shopName": "alibaba",
"version": 34120
},
"sort": [
34120
]
}
]
}
}
},
{
"key": "amazon",
"doc_count": 3,
"product": {
"hits": {
"total": 3,
"max_score": null,
"hits": [
{
"_index": "test",
"_type": "doc",
"_id": "123",
"_score": null,
"_source": {
"shopName": "amazon",
"version": 3412
},
"sort": [
3412
]
}
]
}
}
}
]
}
}
}
It will be better if you provide reproducible example, but it looks like you need to use aggregation framework: term aggregation and top hits aggregation:
GET /_search
{
"aggs": {
"shops": {
"terms": { "field": "Shop" }
},
"aggs": {
"tops": {
"top_hits": {
"size": 100
}
}
}
}
}