Related
I need to update the MongoDB field with the array of objects where JSON object to be updated with as an array
if I have something like this in MongoDB
"designSectionContents" : [
{
"_id" : "5bae17ecbd7595540145ec98",
"type" : "subSection",
"columns" : [
{
"0" : {
"itemId" : "5b7465980783d9a37058f160",
"type" : "field"
}
},
{
"0" : {
"itemId" : "5b7465630783d9a37058f15c",
"type" : "field"
}
},
{
"0" : {
"itemId" : "5b7465810783d9a37058f15e",
"type" : "field"
}
}
],
"subSectionContentLayout" : {
"labelPlacement" : "Top",
"columns" : 3
}
}
]
I want to change the above snippet to below in MongoDB
"designSectionContents" : [
{
"_id" : ObjectId("5bae17ecbd7595540145ec98"),
"type" : "subSection",
"columns" : [
[
{
"itemId" : "5b7465980783d9a37058f160",
"type" : "field"
}
],
[
{
"itemId" : "5b7465630783d9a37058f15c",
"type" : "field"
}
],
[
{
"itemId" : "5b7465810783d9a37058f15e",
"type" : "field"
}
]
]
}
]
curly braces opening and closing tag has to be changed to array
This should work:
db.collection.aggregate([
{
"$project": {
"designSectionContents": {
"$map": {
"input": "$designSectionContents",
"as": "designSectionContent",
"in": {
"_id": "$$designSectionContent._id",
"type": "$$designSectionContent.type",
"columns": {
"$map": {
"input": "$$designSectionContent.columns",
"as": "inp",
"in": [
"$$inp.0"
]
}
}
}
}
}
}
}
]);
Here's the working link.
I am trying to create a BQ Table from AVRO file. I am getting this error when i run the BQ load job:
"Error while reading data, error message: The Apache Avro library
failed to parse the header with the following error: Unexpected type
for default value. Expected long, but found null: null"
The Schema of the AVRO file is:
{
"type" : "record",
"name" : "Pair",
"namespace" : "org.apache.avro.mapred",
"fields" : [ {
"name" : "key",
"type" : "int",
"doc" : ""
}, {
"name" : "value",
"type" : {
"type" : "record",
"name" : "CustomerInventoryOrderItems",
"namespace" : "com.test.customer.order",
"fields" : [ {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "inventory_order_items",
"type" : {
"type" : "map",
"values" : {
"type" : "array",
"items" : {
"type" : "record",
"name" : "CustomerInventoryOrderItem",
"fields" : [ {
"name" : "order_item_id",
"type" : "int",
"default" : null
}, {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "created_at",
"type" : "long"
}, {
"name" : "product_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "type_id",
"type" : "int",
"default" : null
}, {
"name" : "event_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "price",
"type" : [ "null", "double" ],
"default" : null
}, {
"name" : "tags",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "estimated_ship_date",
"type" : [ "null", "long" ],
"default" : null
} ]
}
}
}
} ]
},
"doc" : "",
"order" : "ignore"
} ]
}
I am not sure what is wrong with the schema or anything else, because of which I am unable to load the data.
The problem is most likely the fields that have type int but you have null as the default value. For example:
"name" : "type_id",
"type" : "int",
"default" : null
The default should either be changed to be an integer or the type should be changed to be a union that includes null (like many of the other fields).
----------Query i have tried----------
db.getCollection('rates').aggregate([
{ $match: { "userId" : "5d4c4f69341b7b1746c80d13"}},
{ $unwind: '$ratewithdate.daywiserates'},{$match : {"$and" :
[{"ratewithdate.daywiserates.date" :{$gte :new ISODate("2019-09-
23T00:00:00.000Z")} },
{"ratewithdate.daywiserates.date" :{$lte :new ISODate("2019-09-
27T00:00:00.000Z")}}]}}])
-------------------------------------------------------------
In My query is here i want to know the query between two dates ,i want
get the data between two dates in the array? am unable to do that ,can
any one send me query,
here my question is i want get the date range from given date to next 30
days,i have tried with aggregate but data become slow..can any one suggest
any better solution for the making query formation
i have tried with aggregation as well find queries am not able find any
results,my goal here is find get the data between two dates,for example
if i selected 2019-09-23T10:43:14.239Z thi date from this date i wanna i
want to show the data.
please send me your value able suggestions to, am not bale to query with
double nested array queries in mongodb,please send me your value able
suggestions to,am not bale to query with double nested array queries in
mongodb please send me your value able suggestions to,
am not bale to query with double nested array queries in mongodb.
please send me your value able suggestions to, am not bale to query with
double nested array queries in mongodb,please send me your value able
suggestions to,am not bale to query with double nested array queries in
mongodb please send me your value able suggestions to,
am not bale to query with double nested array queries in mongodb
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1bff"),
"alloted_roomid" : [],
"name" : "working_rate3",
"description" : "bitcpin",
"type" : "room",
"value" : null,
"inclusive" : "General",
"refundable" : {
"cancellationWindow" : "",
"outsideWindowPenalty" : "",
"insideWindowPenalty" : ""
},
"nonRefundable" : true,
"cancellationWindow" : "",
"daysWiseRate" : "30",
"insideWindowPenalty" : "",
"outsideWindowPenalty" : "",
"deviations" : 980,
"policy" : "",
"funds" : "nonRefundable",
"vat" : 890,
"other_tax" : 90,
"roomRates" : [
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c00"),
"roomId" : ObjectId("5d7c8f2950a6c766c64b2a46"),
"roomName" : "Basic",
"rate" : 9888
}
],
"userId" : "5d4c4f69341b7b1746c80d13",
"hotelCode" : 10034,
"ratewithdate" : [
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c01"),
"roomCategory" : "Basic",
"roomId" : ObjectId("5d7c8f2950a6c766c64b2a46"),
"createdAt" : ISODate("2019-09-21T10:43:14.243Z"),
"daywiserates" : [
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c1f"),
"date" : ISODate("2019-09-21T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c1e"),
"date" : ISODate("2019-09-22T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c1d"),
"date" : ISODate("2019-09-23T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c1c"),
"date" : ISODate("2019-09-24T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c1b"),
"date" : ISODate("2019-09-25T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c1a"),
"date" : ISODate("2019-09-26T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c19"),
"date" : ISODate("2019-09-27T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c18"),
"date" : ISODate("2019-09-28T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c17"),
"date" : ISODate("2019-09-29T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c16"),
"date" : ISODate("2019-09-30T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c15"),
"date" : ISODate("2019-10-01T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c14"),
"date" : ISODate("2019-10-02T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c13"),
"date" : ISODate("2019-10-03T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c12"),
"date" : ISODate("2019-10-04T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c11"),
"date" : ISODate("2019-10-05T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c10"),
"date" : ISODate("2019-10-06T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c0f"),
"date" : ISODate("2019-10-07T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c0e"),
"date" : ISODate("2019-10-08T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c0d"),
"date" : ISODate("2019-10-09T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c0c"),
"date" : ISODate("2019-10-10T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c0b"),
"date" : ISODate("2019-10-11T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c0a"),
"date" : ISODate("2019-10-12T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c09"),
"date" : ISODate("2019-10-13T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c08"),
"date" : ISODate("2019-10-14T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c07"),
"date" : ISODate("2019-10-15T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c06"),
"date" : ISODate("2019-10-16T10:43:14.239Z"),
"rate" : 9888
},
{
"_id" : ObjectId("5d85fec2e8652a5c20ae1c05"),
"date" : ISODate("2019-10-17T10:43:14.239Z"),
"rate" : 9888
},
]
}
],
"id" : "rat-W262IxTjk",
"__v" : 0
}
Try this:
db.getCollection('rates').aggregate([
{ $match: { "userId" : "5d4c4f69341b7b1746c80d13"}},
{ $unwind: '$ratewithdate'},
{ $unwind: '$ratewithdate.daywiserates'},
{ $match : {
"$and" :[
{ "ratewithdate.daywiserates.date" :{$gte :new ISODate("2019-09-23T00:00:00.000Z")} },
{ "ratewithdate.daywiserates.date" :{$lte :new ISODate("2019-09-27T00:00:00.000Z")} }
]
}
},
{ $addFields: {result: "$ratewithdate.daywiserates"}},
{ $project: {result: 1, _id: 0}}
])
I'v tried to perform the following query :
db.getCollection('fxh').find({"username": "user1", "pf.acc.accnbr" : 915177},{userid: true, "pf.pfid": true, "pf.acc.accid":true})
and my collection is the following :
{
"_id" : ObjectId("5932fd8f381d4c0a7de21942"),
"userid" : 1496513894,
"username" : "user1",
"email" : "user1#gmail.com",
"fullname" : "User 1",
"pf" : {
"acc" : [
{
"cyc" : [
{
"det" : {
"status" : "New",
"dcycid" : 1496513941
},
"status" : "New",
"name" : "QPT202017_M1",
"cycid" : 1496513940
}
],
"status" : "New",
"accnbr" : 915177,
"accid" : 1496513939
},
{
"cyc" : [
{
"det" : {
"status" : "New",
"dcycid" : 1496552643
},
"status" : "New",
"name" : "QPT202017_S8",
"cycid" : 1496552642
}
],
"status" : "New",
"accnbr" : 73497,
"accid" : 1496552641
}
],
"pfid" : 1496513935,
},
"lastupdate" : ISODate("2017-06-03T18:18:55.080Z"),
"__v" : 0
}
When I execute the query the result is the following :
{
"_id" : ObjectId("5932fd8f381d4c0a7de21942"),
"userid" : 1496513894,
"portfolio" : {
"acc" : [
{
"accid" : 1496513939
},
{
"accid" : 1496552641
}
],
"pfid" : 1496513935
}
}
And my problem is that I need to see only the concerned accid and the result returns the all accid !.
Any idea how just to return the selected accid of accnbr ?
NB : I have also tried to add $ sign at the end of my query , it
selects the right acc but it returns the all objects or I need just
only ONE returned object.
On 6/5/17
I also used the aggregate command instead of find and it get result by using this :
db.getCollection('fxh').aggregate([ { $unwind : "$pf.acc"} , { $match : {"username":"adh1", "pf.acc.accbr": 915177 } }, {$project : {_id:0, accid: "$pf.acc.accid"}}])
But could NOT get a lower level result, when I ran this :
db.getCollection('fxh').aggregate([ { $unwind : "$pf.acc.cyc"} , { $match : {"username":"adh1", "pf.acc.accbr": 915177, "pf.acc.cyc.name": "QPT202017_M1" } }, {$project : {_id:0, cycid: "$pf.acc.cyc.cycid"}}])
Any idea ?
You can try the below aggregation pipeline.
The idea is to $unwind one nested level at a time, starting from the outermost to the innermost.
For each nested level unwinding, you can apply the$match to limit the documents and continue till you have the desired shape.
You can $group it together at the end to get back to the original shape.
db.getCollection('fxh').aggregate([
{ $match : {"username":"adh1"} },
{ $unwind : "$pf.acc"} ,
{ $match : {"pf.acc.accbr": 915177 } },
{ $unwind : "$pf.acc.cyc"},
{ $match : {"pf.acc.cyc.name": "QPT202017_M1" } },
{$project : {_id:0, accid: "$pf.acc.accid", cycid: "$pf.acc.cyc.cycid"}}])
My query in lucene 6.2.0 goes like:
query query = new PhraseQuery.Builder()
.add(new Term("country","russia"))
.setSlop(1)
.build();
Basically among all my documents which are:
{
"_id" : ObjectId("586b723b4b9a835db416fa26"),
"name" : "test",
"countries" : {
"country" : [
{
"name" : "russia"
},
{
"name" : "USA china"
}
]
}
}
{
"_id" : ObjectId("586b73f24b9a835fefb10ca5"),
"name" : "nitika jain",
"countries" : {
"country" : [
{
"name" : "russia and denmrk"
},
{
"name" : "USA china"
}
]
}
}
{
"_id" : ObjectId("586b744f4b9a835fefb10ca7"),
"name" : "arjun",
"countries" : {
"country" : [
{
"name" : "russia pakistan"
},
{
"name" : "india iraq"
}
]
}
}
I want a document which has only russia. Ideally it should be the one highest scored, but instead I get something like "Found 3 hits."
Document<stored,indexed,tokenized<id:586b723b4b9a835db416fa26> stored,indexed,tokenized,omitNorms,indexOptions=DOCS<name:test> stored,indexed,tokenized,omitNorms,indexOptions=DOCS<countries:{ "country" : [ { "name" : "russia"} , { "name" : "USA china"}]}> stored,indexed,tokenized<country:russia> stored,indexed,tokenized<country:USA china>>**0.12874341**
Document<stored,indexed,tokenized<id:586b73f24b9a835fefb10ca5> stored,indexed,tokenized,omitNorms,indexOptions=DOCS<name:nitika jain> stored,indexed,tokenized,omitNorms,indexOptions=DOCS<countries:{ "country" : [ { "name" : "russia and denmrk"} , { "name" : "USA china"}]}> stored,indexed,tokenized<country:russia and denmrk> stored,indexed,tokenized<country:USA china>>**0.12874341**
Document<stored,indexed,tokenized<id:586b744f4b9a835fefb10ca7> stored,indexed,tokenized,omitNorms,indexOptions=DOCS<name:arjun> stored,indexed,tokenized,omitNorms,indexOptions=DOCS<countries:{ "country" : [ { "name" : "russia pakistan"} , { "name" : "india iraq"}]}> stored,indexed,tokenized<country:russia pakistan> stored,indexed,tokenized<country:india iraq>>**0.12874341**
All 3 results are equally scored. How can I get the document with only russia to be highest scored?
In Phrase queries, the slop is zero by default, requiring exact matches. that means that if you modify your query in this way:
query query = new PhraseQuery.Builder()
.add(new Term("country","russia"))
.build();
you'll get what you're looking for.